Impact measurement, a generative flow On line lecture conference Culture Care Mai, 14 2020
Dr. Anke Coumans
Hi Everybody. This will be my first zoom presentation. Difficult not to have an audience in front of me, but instead a screen full of small images that represent real people. I hope I will be able to truly connect with you this way.
My name is Anke Coumans. I am professor Image in Context at the Knowledge Centre Art and Society at Minerva Art Academy in Groningen. I am involved in new roles for artists and designers in society and I develop and study new relational artistic practices in the contexts of health, the elderly and wellbeing. Impact measurement is one of my focus points and for many years I have worked with my colleague Ingrid Schuffelers on this issue.
We recently created a small collective with some young artist and designers called Demenslab.
What I like to do in my presentation is to give you an example of how Ingrid and I did an alternative impact measurement a few years ago in Groningen when we were asked to measure all the participatory art projects in which the elderly were involved. I want to tell you very precisely what our purpose was, and how we worked it out. We all learned a lot from this exercise. And I like to share that with you.
It will explain to you the four principles that are important in our view for every impact measurements activity.
In general one could say that impact measurement refers to the procedural obligation we have when accepting money from a funding organisation. This obligation means that we have to prove to the funding organisations after the project is completed that something good and valuable came out of the project they financed. Through the Impact measurement we prove that the objectives worked out fine and that we did what we said we should do.
In most cases we measure it because we are obliged to do it and because we want to propagate our effectiveness in the hope to get more funding in the future.
I also guess that we all experienced that the funding organisations are most of all interested in who we reached with our participative art projects, and what the effect of our art project was. And I also guess we all use qualitative and quantitative questionnaires to find out more about our participants and to find out what the project meant to them. Just because it seems the most logical and feasible thing to do. And I suppose we all found out how difficult those
questionnaires are, especially when it comes to the appreciation of the project. What else can they say when asked How did you evaluate the project: yes I liked it a lot?
So when we were asked by the council and the province of Groningen to do an impact measurement on a number of art projects with elderly people, called Pronkjewail, we decided to do it differently.
The so called pronkjewail projects consisted of a list of very different projects. I will give you some examples to give you have an idea what kind of projects we had to measure.
In one of the projects Samen in beeld tv (together in the picture television) the television makers made television programmes about elderly people in care homes, and these items were meant to be broadcasted within the elderly facility. But during the project the elderly asked to be included in the making of the items: the projects started as a project about the elderly and it ended as a project with the elderly included in the making of it.
In another project called Groningen plus, a group of elderly people created for themselves a kind of art academy for the elderly. They invited a theatre maker to help them cross the border of the obvious and dive into new artistic
In a third project rock and roll dance was offered to the elderly and vulnerable people of a small neighbourhood in Groningen. The organiser was a resident of that neighbourhood where she had a small space for people to make art. She was a rock and roll dancer herself and she became the leading lady of a variety of people who had never thought they were able to dance rock and roll and have fun.
When we were asked to do the impact measurement of these projects, as I already said, we wanted to do it differently. These were our ambitions in the design of the impact measurement.
1 to measure impact is to create it
We wanted to do an impact measurement that would be of real benefit to the artists, the participants and the facilitating organisation. An Impact
Measurement that would help the province and council to develop a policy that would really be of benefit to elderly people. We wanted an impact measurement that had impact by itself.
2 show don’t tell
We wanted to develop tools that would help us to know what really happened in those projects and we wanted to hear it from the participants and
stakeholders themselves. We decided NOT to work with a questionnaire and to invite them all at the same moment to do the impact meeting together as an open invitation to find words to describe the project and to share these words in a non measuring context.
3 a learning community
We wanted to make use of the impact the people of the different projects would have on each other during the measurement. The impact measurement as the start of a kind of learning environment that could continue after the meetings were over. Perhaps new liaisons (collabarations?) would arise out of it.
4 to bridge the distance between real people and data
We wanted all stakeholders, including the people who were responsible for the money, to be present at the measurement. We wanted to confront them not only with the outcome, but also with the people who were committed to the projects, with the people who participated and with the methodology we used. We wanted to bridge the distance between real people and data.
We were very aware of these principles while designing the measurement as an event itself and we tried to develop context and measuring tools that would make it possible for all kind of unforeseen information to emerge.
This meant that the methods had to be open and encourage people to contribute information that was in a way also new to themselves.
This also meant that we had to develop a protocol for the day that provoked a lot of interaction between the different participants and stakeholders.
Let me explain to you how we tried to do that.
We invited the artists who executed the projects and asked them to bring along some participants of the projects they were responsible for. When
organizations as older people facilities were involved we also asked them to send a representative. When they arrived we told them that we had several tables with playful exercises were they could share their input where they wanted and in the order they would prefer.
A On one of the tables we asked everybody who was present to write postcards about the project to people they know. We promised to post them after the session was over. In that sense the postcards were not just an exercise but also a real intervention. By offering them this opportunity we gave them the space to express whatever they wanted to express and to advocate the project to whomever they wanted to advocate it to. Some wrote postcards for friends and family, some wrote a card to the organisation to thank them and others to the council and the province to ask for more attention and money. The postcards represent the direct voices of the participants. It gave us authentic information about the value of a project. It helped us to see the new possibilities and the new groups that could be approached. And while sitting on a table they spoke with other participants and grew their own/personal network.
B In a second exercise we asked people to draw where they came from and what their trajectory was on a map of Groningen. The participants of each group drew lines and commented, while other groups were watching.
It showed us not only where people came from, but also the importance of the journey in itself. We learned that an art project already starts the moment people leave their homes. As they travelled in the same car, they made a connection and already exchanged all kind of interesting topics during the trip to the location of the art project.
C In a third exercise the stakeholders of each project gathered around a table and monitored by Ingrid Schuffelers they drew a diagram which represented the development of their project including the context and the stakeholders. Ingrid led them through the process by asking them who did what and what happed next and what provoked the happening. In fact, they drew some kind of actor network (for the ones among you who know the actor network theory of Bruno Latour).
Afterwards we redrew the drawings in illustrator and together with the outcomes of the other exercises they gave us information to analyse.
This exercise was the most powerful of all the exercises. Not only did it visibly show the power of the collective, the exercise also enabled the stakeholders of the project to develop a shared idea of how the project went and what the future possibilities could be. In fact: they measured themselves, they didn’t feel measured by some external organisations that would draw conclusions based on their input.
In all of these exercises the applicants not only brought in information, they also got something out of it: new insights, new ownership over their project, new relations to work on for the future.
Based on the information of the impact measurement we wrote a report in which we showed how the exercises provided qualitive and quantative data concerning the focus points the funding organisations brought in. They wanted information about the level of co-creation (yellow in the scheme), about the experience of the people involved (blue in the scheme) and about the way the projects evolved (green in the scheme).
In our report we organized the visual material of the measurements: the data produced by the participants
.
We showed our interpretation:
We gave our advice based on these schedules. For example, we advised to focus on organisations who were eager to develop considering the ambitions of the elderly: organisations that would listen and act accordingly.
I cannot go into this information due to lack of time. I hope that’s okay?
I will now continue with the lessons we learned from this Impact measurement. If I look back at this event it taught me some things about impact
measurements that I would like to share with you. In the past week the
members of the Demenslab coalition have worked with me on a plan that holds together what Impact measurement should be.
1 IM should be about what is emergent.
It should not just be about how it was, and how it is, but also about how it should or could be. Part of measuring is being open for what emerges, what has opened itself due to the interventions of the project.
What more can be done? What opportunities are ahead?
We saw in the project of Groningen plus that they asked for interventions to make things possible. It was the impact measurement that made them aware that they were actually working on a sort of Art Academy of the Elderly, and that their main force was the fact that the initiative of the project came from the participants, who were both organisers and participants. To transfer the whole project to another location would not be possible, because it would then impose something on the participants of that location. The only thing that could really be transferred was the energy and the trust that elderly people can develop the projects they wanted and how they wanted it. It just takes courage and faith.
2 IM should be about measuring autonomy
The impact one measures in art projects is always some kind of autonomy. What results from art projects is in essence the creation of autonomous individuals with their own voice and with new ownership over their
(professional) lives. This autonomy comes into being as a result of unexpected individual findings of the participants. They suddenly realise that they are somebody else than they always thought they were, that they are able to do much more, that they know much more, etc.
How did the interventions in the project procreated forms of autonomy, and how can this autonomy be described in terms of inclusiveness, ownership, having a voice etc. and (1) what is possible as a consequence of this emerging autonomy?
We saw in the project of Samen in Beeld tv that the elderly felt the urge to become part of the television team, to make the films themselves. There was something in the project that made them feel empowered to be more than the subject in the television programme or the subject looking at television, they also wanted to be involved making the programmes themselves. This growing autonomy created new emerging possibilities for the programmes themselves. 3 IM should be an ongoing feedback loop during the project
Impact measurement is not only important when the project is completed, it should be an essential ingredient during each participatory art project. We call this dynamic in which each action is followed by an awareness of and a
reflection on the consequences a feedback loop. To measure impact is to create it means that I.M can create a feedback loop in which one is in constant
dialogue with the people of the project.
How was impact measured and taken serious during the project? How can we see that creating autonomy and creating new possibilities was already part of the project?
Due to the openness of the woman who organised the project Samen in beeld tv, the project organisation developed and changed.
4 I.M. should be situational dialogue
If we do an impact meeting when the project is completed it should not (just) be an objective procedural measurement but it should also be a situational exercise including the all the partners, a joint and situated exercise, aimed at future possibilities. A dialogue which can create a generative flow. That flow should also be measured from time to time to understand the dynamics that can come from forward? while measuring impact as an ongoing feedback loop.
Conclusion
Perhaps you already notice that all the characteristics I gave you, are the characteristics of an art process that aims at an artistic intervention.
Every art process is characterized as a process in which the artist is open to what emerges and a process in which they find new possibilities based on what they witnessed.
In that sense each art process can be characterised by an ongoing feedback loop
Specific for art in general and certainly for participatory art: it is always situational, it is dealing with what is present: material, people, previous exercises.
And finally: Art is about creating autonomy by giving the audience a new perspective, new possibilities be it in the form of a painting, an installation, or an intervention
