Participation process(es) explored : within ZonMw programs working with patient panels

Participation

process(es)

explored

Within ZonMw programs working with

patient panels.

Thesis Concept

Master Medical Anthropology and Sociology- UVA

Student: A.Vis (10269843)

Supervisor: A.J. Pols

Second Reader: J.C. Aceros

10th of August 2014

Amsterdam

3

Summary

The term 'participation' seems to have acquired mythical proportions; nowadays it is a very fashionable word to use when discussing issues like healthcare, education, development, and cooperation. It is a discourse full of optimism: everybody gets the chance to have a say in the decision-making. However, participation is not simply a new way of working, a method that just has to be installed. Giving patients a voice implies a major change in the organizing, thinking and doing of the health research system and its actors. It is a value, a way of looking at how people relate to each other, and who has a say in what. This value is embodied in dialogue with the different actors and in the different contexts and situations that they find themselves (Dedding 2009). But little is known regarding this interactive process, about how participation within organizations is shaped, nor about the effects of participatory processes on the different participants and organizations, and how you can assess these effects (Dedding 2009; Blume & Catshoek 2003). On the basis of a interview study, I have tried to answer the question: How patient participation is realised and embedded within ZonMw

programs during the appraisal procedure in subsidy granting?

Patient participation offers an opportunity to include the experience of patients into the research system. If patients become involved as equal partners, encouraged to think about the relevance and conduct of research, important themes of the life world of patients will be added to the research agenda (De Wit 2013). Creating such a dialogue for patients to contribute their knowledge proofs to be a challenge at ZonMw. Due to the lack of knowhow by both patients and professionals, little interaction within the partnerships, collaboration was suboptimal. Through experimentation, combined with reflection and learning, more insight can still be gained into effective approaches, appropriate programs and useful tasks for participating patients at ZonMw. It is important that support is offered to all parties involved to strengthen the competences of all actors to collaborate effectively. An unambiguous concept of participation in terms of task, roles and responsibilities shared between these actors is lacking. A change in the current culture, structure and practices of health research is needed to effectively and structurally involve patients and embed their perspective successfully.

4

Preface

The thesis before you "Participatory process(es) explored ', is the concluding piece of the Master Medical Anthropology and Sociology.

With this thesis, I hope to have given some insights into the possibilities and limitations of patient participation in various ZonMw programs. It aims to perceive patient participation as a highly context dependent concept, in order to raise awareness about what is really needed to participate meaningfully. This interview study may provide insights in how to make use of the characteristics and challenges of the context, when designing suitable forms of participation in an organization. On a more general and theoretical level, this thesis contributes to the knowledge on the possibilities and limits of participatory discourse in health research.

On a personal level, I hope that the presented cases will inspire researchers, patients and policy makers to continue on the path of dialogue, evaluation and mutual learning.

It is appropriate to address a word of gratitude to the respondents who have told their story and thus have contributed to the realization of this thesis, particularly I thank the program coordinator on participation within ZonMw for her positive support. I am also indebted to my friends and relatives for their ongoing support. Finally, my thanks go out to supervisor A.J. Pols, a passionate source of inspiration with a soft spot for participation.

For me, nothing more remains to do then to thank you as a reader for taking up this thesis. As closure, I would offer you the following message:

5

Index

Summary ... 3

Preface ... 4

Ch.2: Methodology & Ethical precautions ... 9

2.1 Interview Study ... 9

2.2 Data collection ... 9

2.3 Recruitment and selection ... 10

2.4 Definitions ... 10

2.5 Ethical precautions ... 11

Ch.3: Theoretical background ... 13

3.1 Patient participation in health research ... 13

3.2 Patients’ Knowledge ... 14

3.3 Participatory-Models ... 15

3.4 The (down) sides of professionalism ... 15

3.5 The power of the professional body ... 16

Ch.4: Participatory process ... 17

4.1 General structure of a ZonMw program ... 17

4.2 Patient-perspective in subsidy request... 18

4.3 Assessment procedure ... 19

4.4 Example of application timeline ... 20

4.5 Additional opportunities to participate ... 20

Ch.5: Goals and views on participation ... 22

5.1 Arguments to include patient participation ... 22

5.2 Addition to professional perspective ... 22

5.3 Studies have more societal value ... 23

5.4 Fulfilment of fundamental right ... 24

Ch.6: How do they do it? ... 26

6.1 Scoring Relevance & Utility ... 26

6

Ch.7: Results of the participatory process(es) ... 33

7.1 Level of influence ... 33

7.2 Influence relating to structure ... 34

7.3 Double Assesment/Overruling ... 35

7.4 Acknowledgment ... 36

7.5 Cultural shift ... 37

Ch. 8: Challenges and bottlenecks ... 39

8.1 Time pressure ... 39

8.2 Time as a counterproductive factor for embedding participation ... 39

8.3 Representativeness ... 40

8.3 Absence of guidelines and knowledge on participation ... 42

8.4 Burden on participating patients ... 42

8.5 Weighing ... 43

8.6 Ad hoc evaluation ... 44

8.7 Transmission of information and feedback ... 45

Ch. 9: Chances to improve ... 47

9.1 Knowledge sharing between programs ... 47

9.2 Cooperation on general framework ... 48

9.3 Participation at appropriate level ... 48

9.4 Next step ... 50

Ch. 10: Discussion ... 52

Ch. 11. Conclusion ... 54

Annexes ... 57

7

Ch.1. Introduction

ZonMwi_{, the Dutch Organization for Health Research and Development, funds health research,}

innovation and stimulates use of developed knowledge to help improve health and healthcare in the Netherlands. Strengthening the position of health consumers, patients and their organizations, while stimulating participation in the health care system, is one of the priorities in the work of ZonMw (ZonMw 2013). In recent years, ZonMw took a considerable number of initiatives in this area. Patients are being asked as (official) reviewers to comment on submitted proposals at ZonMw programs in so called “patient panels”, consulted by researchers and participate in program- and project committees (Van den Bovenkamp 2010). It appears to be a challenge to obtain panel members who feel like they contributed successfully in the participatory process, and whose input is perceived as added value to the healthcare and research by researchers/policymakers (Keizer 2012). That process is difficult, especially when the perception of researchers and patients diverges. Striking enough there is little research done about the effect of these panel and its participants. Since the start of the programs it became compulsory to include a paragraph about patient participation in all research proposals. The risk exists that researchers consider the paragraph merely as a necessary requirement for funding. Although the panels receive preparation from the Federation of Patients and Consumer Organisations in the Netherlands (NPCF) or involved patient organizations, patients can often say nothing about the rest of the research proposal. They do not have the tools and (medical) knowledge to assess the full proposal (ZonMw 2013). Therefore the question arises at ZonMw: How do you achieve meaningful participation?

Participation is seen in most (management) literature as a solution to shortcomings identified with policy-making and implementation. Input of patients is not only seen as important in the case of individual doctor-patient contact (Guadagnoli and Ward 1998; Bensing 2000; Gattellari, Butow et al. 2001), but is advocated on a variety of other levels of decision-making; for example guideline development and research assessment. Patient participation in decision-making is part of the development towards a more demand-driven health care system. With the emphasis on choice options for individual patients, the present Dutch health care system is increasingly aimed at suiting the care according to individual needs (Grit, van de Bovenkamp et al. 2008). Research, shows that it is not so self-evident and effortless for professionals to identify with patients/to perceive them as the starting point for everything that goes on in the organization (Bate 2007). This notion of patient centeredness is not new. Previously the patient was also central to the decision-making process, but there was more discussion about the patient then with the patient. Through the years the notion

8

arose that patients should be involved in decision-making, because other parties proved to be insufficient to speak on their behalf. Due to their special concern with their disease, it is seen legitimate in Western healthcare to grant patients involvement in decisions regarding their situation (Baker 2007). Patients’ organizations have become acknowledged as stakeholder and patients are in the Netherlands indicated, next to health providers and health insurance companies, as the ‘third party’ in the health field (Van de Bovenkamp, Grit et al. 2008). "Meaningful participation" is generally defined as participation that is influential and actually leads to change (Dedding 2009). Patients can bring a unique perspective to the table on account of their experiences, but participation isn’t an easy task for patients - effective participation often requires expertise and perseverance (Trappenburg 2008; Van der Kraan et al. 2008). Patient participation is seen as a way to improve the quality of care through the input of experiences. This however implies that ‘experiential knowledge’ of participating patients is regarded valid by professionals and researchers. But, are patients truly taken serious in the participation process? Harrison and Mort (1998) mention the risk that participation could be used instrumental and in that case it will be reduced to a symbolic role (Harrison and Mort 1998).

The core of the discussion does no longer have to be about the question whether patients have sufficient skills that enable them to participate, because it has been shown in multiple studies that they are willing and able to participate (Dedding 2009). However, the question remains how participation of patients can be meaningful. Is there any chasm at ZonMw between professionals and patients within the panels? And is this chasm getting smaller or is it increasing?

This leads to the following research question:

How patient participation is realised and embedded within ZonMw programs during the appraisal procedure in subsidy granting?

This requires the following sub-questions to be answered through data gathering & analysis:

1) How is the participation of patients within the ZonMw programs shaped?

2) How is participation of patients perceived by researchers and policymakers at ZonMw?

3) What are the results, bottlenecks and challenges of these participatory processes?

4) What recommendations can be given to improve the current patient participation within ZonMw?

9

Ch.2: Methodology & Ethical

precautions

The study is in line with the current evaluation on participation at ZonMw. A qualitative approach does justice to the complexity and dynamics that characterizes participatory projects. This study is a process evaluation of the use of patient participation, instead of an impact assessment. Given the limited timeframe it would have been hard to measure the results of the participatory process.

2.1 Interview Study

In this research ZonMw is identified as the main research unit. This study was intended to acquire knowledge and insights from the practice of patient participation at ZonMw; therefore interviews have been conducted with program coordinators/secretaries, participants and (experts on participation related to six ZonMw programs (See Annex). The topics and the method of reduction were derived from publications on ZonMw programs, literature on participation and information on quality research. The data from internal documents and literature was input for the sub-question on the design of the participatory process.

2.2 Data collection

The study started in early 2014 with a document study, including scientific literature and policy documents. Also, interviews were conducted with experts on participation in research assessment to hear the dilemmas they face. Then the interviews with patient referents were scheduled.

The researcher chose to conduct semi-structured interviews, face-to-face on location. The interviews were digitally recorded, transcribed, and by encryption analysed (Creswell 2003). The collected information from the interviews was coded in relation to the research and sub-questions. The main themes that emerged are worked out in the analytic chapters of this thesis. To identify these coding themes, several techniques where used (Gery and Bernard 2003). First off all, the most obvious themes in the corpus of data were those “topics that occur and reoccur”. Next to repetition, occurring shifts in the interviews, for example pauses and changes in voice tone (cynicism, a raised voice), were used in the coding process. Finally, the transcribed interviews were scanned for linguistic connecters, such as: “because,” “since,” “as a results”. When analysing the themes and corresponding phrases, the researcher searched for similarities and difference across the data (interviews, theory, documents).

10

During the execution of the study the validity and reliability of the data was taken into account; through data- and method triangulation, data- and identity anonymisation of respondents, defining concepts at use and a systematic-way to conduct the data-analysis.

2.3 Recruitment and selection

All respondents were invited by email/telephone to participate. The program secretaries/coordinators, patient referents, and other respondents are experts in their field, chosen for their (contrasting) opinion, role in the debate, and position within their organization.

The respondents have received fictitious names to secure their privacy:

1) Ms. H. Van Leeuwen (coordinator participation ZonMw)

2) Mr. M. Van der Putten (program secretary ZonMw)

3) Ms. V. Spelt (program secretary ZonMw)

4) Ms. B. Sieverts (program secretary ZonMw)

5) Ms. J. Helderman (program secretary ZonMw)

6) Ms. K. Koza (patient referent in panel)

7) Ms. D. Van der Kraan (patient referent in panel)

8) Ms. J. Vogelaar (program secretary ZonMw)

9) Mr. P. Duijmelinck (patient referent in program-commission)

10) Ms. W. Koster (external program coordinator)

11) Mr. W. Horváth (trainer/ experiential expert)

12) Mr. B. Kirchyünger (external program coordinator)

2.4 Definitions

In this thesis participation is defined as: “a situational and interactive process involving actors in

research, quality or policy who enter into a dialogue, which does justice to the experience, knowledge and skills of the actors throughout the entire process. This process aims to achieve improvements in research/care” (Dedding & Slager 2013).

11

Therefore, the concept of participation is a way of looking at how people relate to each other. Just like, Hart (1992), I consider participation as an active form of involvement, with the possibility that this involvement is effectuated in decision making. During the data-gathering of this research, what was considered as patient participation was based on the views of the respondents (they were asked where the described dialogue takes places) and on the definition above.

During the interviews different terms where used to describe the persons who contributed from a patient’s perspective: end-users, patients, experiential experts etc. The original phrasing by the respondents has been used in the quotes. The readability of the text could have benefited from using one term, but had also worked euphemistically and made the differences in views and experiences with patient participation less sharply visible. Participation is such a term that is never used for people who fall within the standard of "ordinary citizens". When expressing my own thoughts on participatory processes I therefor deliberately use the term: patient. This will remind everyone that the people concerned are ill and/or limited in their lives, which might imposes specific requirements on the chosen form of participation at ZonMw. Where in this thesis is spoken of professionals, it will refer to people who professionally work in health care. This is not to suggest that patients are not professionals, but I only want to connect to prevalent language usage.

2.5 Ethical precautions

The ethical responsibilities lie firmly with the researcher; he is responsible not to harm the participants. Because of the risk of causing something to happen in relation to the lives of the participants, he needs to consider the impact of his study.

If any ethical issues related to the methods of use manifested themselves, they predominantly occurred during the interviews or observations. Participants might have said more than they really wanted to or formulated there opinion in an undesired manner. Due to a possible conflict of interest between the researcher and the researched, the researcher could be tempted to use or deliberately obtain such data (Blaxter et al. 2001). Because of these ethical concerns, the respondents needed to be informed about the fact that they are part of an evaluation on participation (Van de Sande 1999). The researcher has asked, and was appointed, informed consent from the research participants prior to the interviews. The use and storages of the data has been determined in dialog with the participant. None of the organisations insisted on consent forms, neither on a formal ethical clearance procedure

12

The risks of participating in this research was more likely to be psychological than physical. Respondents could be confronted with an undesired (re)evaluation on their motivation and/or contribution to the participatory processes at ZonMw.

The researcher has evaluated with his supervisor before and after the data collection if all risks were justified by the importance and relevance of the study. In absolute terms, the level of risk in this study is minimized when a high level of anonymity and confidentiality is ensured. Therefore, the respondents have been given fictitious names and their connection to the particular programs is not specified.

13

Ch.3: Theoretical background

Participation is not a blueprint that policymakers can impose on any practice, but a process that forms in relations of dialogue and joint decision-making (Thompson 2007). These relationships and dialogue are influenced by the degree in which people want to participate, but also by the environment and institutions in which the process occurs (Dachler & Wilpert 1978). In the next sections the ideas behind the participation discourse, the details of the program and the context dependency are further elucidated.

3.1 Patient participation in health research

Roughly speaking, there are two interpretations of the term participation possible. Participation can simply mean 'taking part' or 'being present'. Secondly it can also have the meaning of a changing power relation in which the views of participants actually affect decision-making. Both interpretations are valid, but problems arise when the first interpretation is confused with the second: Tokenism (Boyden & Ennew 1997). According to Hart (1992), tokenism occurs when policy makers or researchers want patients to participate, but have not thought the implications through (Dedding 2009). This will ultimately leads to a lack of room for the ideas and needs of the participating patients, or they will simply be overruled. To emphasize the difference with tokenism, the second form of participation is sometimes called meaningful participation, active or effective participation.

However, in practice there is still confusion among actors about how participation is interpreted and validated. Although participation can thus have several meanings, these definitions and the choices made therein are rarely communicated. This is problematic, because the use of the same terminology in different contexts suggests consensus, whereas in reality that is rarely the case. As long as the various meanings of participation among actors are made transparent (clearly communicated and understood as such) they don’t have to be an issue. Naturally, this also applies to this research, that is why I clarified a working definition in Chapter 2.

Scientific research without active involvement of patients is hard to image nowadays (Smit et al 2006). More and more institutions and funds perceive participation of the people for whom the research is intended therefore as a prerequisite. ZonMw states that the patient perspective should standard be taken into account in every research and invites patients to become a member of program committees and to assess the relevance and execution of research applications (ZonMw 2013). As with patient participation in the actual health care, participation in health (care) research is a difficult process. Patients are often indignant about the instrumental way they are involved in

14

research, with a lack of attention for their experience and expertise (ZonMw 2013). There is also indignation among patients about receiving no feedback on the results of their participation or the end results of the research. And if the research results are disseminated, accessibility could be compromised because of the use of scientific jargon (Blume & Catshoek 2003). Patients are increasingly aware of the scientific value of their knowledge and experience deprived from everyday life. Not only do they want their knowledge to be recognized in research, but also that their participation in research results in favourable outcomes for themselves. At the same time, patients and patient organizations still struggle with questions about the desirable form of participation in research. They wonder what role they can and are supposed to play. Simultaneously, professionals must learn to rely on the knowledge and expertise of patients, share the control on the execution of research and to perceive it as an equivalent collaboration (ibid). Besides the setting of research agenda's and methods, attention needs to be paid to the way patients and scientists work together (ibid).

3.2 Patients’ Knowledge

Intuitively it is clear that people with (chronic) illnesses or disabilities have knowledge about how to cope with their condition in daily life. However, it is difficult to formulate what kind of knowledge they exactly possess. There are several outlooks on this type of knowledge. First, the practical knowledge that people require and use to live with their illness can roughly be seen as equivalent/similar to medical knowledge. Concepts such as self-management are based on this notion. Secondly, this knowledge could be interpreted as completely alien from medical knowledge, because people give culturally specific meanings to their condition or tell stories about personal experiences. With this experiential knowledge they provide scientists complementary expertise or develop knowledge for their peers (Dedding & Slager 2013). The danger of considering patient’ knowledge as entirely different is that it is likely to be romanticized and not seen as useful/relevant for professionals “who do not know how it is like to be a patient”. The consequences of this often unnoticed ambiguity about what patients' knowledge actually is, is that well-intentioned policy interventions fail and patients never really become a partner in research. What they contribute does not fit in the form of medical science: it is not perceived as evidence (Van de Bovenkamp & Trappenburg 2009). Either way, people with (chronic) illnesses or disabilities can participate due to their specific knowledge. Participation in scientific research can be aimed at articulating (verbalize) and transporting this patients knowledge (Dedding & Slager 2013).

15

3.3 Participatory-Models

But who influences the participatory processes? Arnstein (1996) makes us aware of the political character of participation. She defines participation as 'the redistribution of power that enables the

have-not-citizens, presently excluded from the political and economic processes, to be deliberately included in the future' (Arnstein 1969:216). This claim isn't the reason Arnstein is widely cited in

participation literature. For implementation and evaluation of participation processes various participatory-models have been developed. Arnstein’s description of the participation process (1969) lies at the basis. For more than 40 years, Arnstein’s ladder of citizen participation has been a touchstone for policy makers and practitioners promoting user involvement. In the literature on participation in health research the model has dominated as well. The ladder consists of 8 steps, ascending from manipulation to citizen control. A useful starting point for categorizing forms of participation, but it could simultaneously be criticized for the division in homogenous blocs. Patient groups encompass a host of divergent points of view, competing interest and splintered subgroups. The ladder also fails to recognize the claim of other reviewed researchers (Van de Bovenkamp et al. 2008; Harrison & Mort 1998) that for some patients, participation itself may be a goal.

According to Gustafsson & Driver (2005) the problems with the use of participants emanate from concerns with power and legitimacy. Similar to Arnstein (1969) these authors identify decision-making as a continuum. The key issues concerning such a hierarchical definition are centred on the usefulness of measuring participation out of context. The authors believe that there is not simply ‘more’ participation because a method is considered to fit higher up a theoretical ladder. Participants do not necessarily take aims of their counterparts, and the knowledge embedded in the used policy, at face value. It may well be contested, resisted and re-interpreted. The production and dissemination of information opens up new forms of knowledge and subsequently affects participants’ contribution (Gustafsson & Driver 2005).

3.4 The (down) sides of professionalism

The review study of Harrison & Mort (1998) concerns two practices, public consultation and user involvement, and describes examples of related problems through a study on health panels in the UK. The authors illustrated that professionals do not always see professionalism of participating patients as an aspiration. In this study patients were not placed in the centre of decision-making, they were constructed as just another interest group. Their view has to be balanced with physicians, managers and others. According to the authors it makes sense in situations where officials do not agree with panel members to challenge their legitimacy; they might be dismissed as extremist or unrepresentative. They could also be criticized for being badly organized or they would be patronized

16

as sick people who cannot give attention to their duties.

3.5 The power of the professional body

Like mentioned above, patient involvement is traditionally seen as a driver for quality and is usually described in terms of ‘patient-centeredness’. For Baker this concept implies a style of consultation by patients and forms of shared decision-making. The author states that patient involvement in a professional body like a committee is unlikely by itself to be a useful mechanism for delivering greater professional accountability. On problem is that an individual patient cannot easily challenge assumptions or question priorities. It’s difficult for one person to introduce new perspectives unless they have exceptionally skills of persuasion and are well respected in the setting. “The process through

which the patient voice is heard, the issues to which patients can contribute and the extent to which they have an influence, are all determined by the members of the professional body.”(Baker 2007:

465). Thus, Baker addresses the already discussed context dependency similar, but she elaborates more on the role of the professional body in this relationship. This study shows the limitation of individual patient participation and that “meaningful” participation demands efforts from all parties.

17

Ch.4: Participatory process

In addition to programs for healthcare development and –innovation, ZonMw also develops research programs. These range from fundamental research to applied research on healthcare and -services. Most ZonMw research programs stem from assignments of the Ministry of Health, Welfare and Sport (VWS). The interviewees derive from six ZonMw programs who let patients participate in the subsidy procedure (See Annexii_{). In this chapter the general structure of - and the assessment procedure within}

- these programs are presented.

4.1 General structure of a ZonMw program

Program Council

ZonMw has program councils established in which experts and representatives of involved (funding) parties have a seat. The program council meets at least once a year to discuss the progress, new developments and possible bottlenecks. In most cases the program council will have no formal responsibility in the implementation of the program and subsidy round, but primarily functions as sounding board and advisory body.

Program Committee

The control of the program rests largely on independent experts, for example prof. dr.'s of university med. centres, organized in a program committee. The ZonMw board installs the members of the program committee. The experts participate in their personal capacity, and are bound by ZonMw's code of interest. The program committee recommends the ZonMw board and funding parties on the remuneration of projects. In most of programs with direct relevance to the patient, there is a member who represents the patient perspective in the committee. Depending on the scope of the program the program committee will consist of approximately 10-30 experts, supported by a secretary at ZonMw.

Many of the respondents in this research are program secretaries/coordinators. Their role is to support the program committee and carry out the necessary management activities in the field of information management, document management, and progress reporting. Often they are also the, internal and external, initial point of contact.

Patient panel

ZonMw state that they find the participation of patients important. Therefore, ZonMw establishes patient panels, consisting of representatives of patients (organizations) in the field of the co-financing

18

fund(s). During the funding round such a panel will assess the projects on certain aspects. These may include: the importance of the problem, burden on cooperating participants, end-users involvement and contributing to the improvement of clinical practice.

Program coordinator participation:

The program coordinator is the first point of contact within ZonMw for questions related to participation by employees. Furthermore the coordinator tries to construct and implement policy on how ZonMw could improve participation.

Patient organisation(s)

Patient organisations are active in decision-making on health research in various ways. They are increasingly consulted when composing research agendas and sometimes have a (intermediary) role in the research proposal assessment. In the examined programmes they facilitated en supervised the patient panels which were used.

4.2 Patient-perspective in subsidy request

Based on the aims off a research program a call for proposals is published, an appeal to the field to submit research proposals. Attention to the patient perspective could be shaped by targeting outcomes/publications on patient organizations. It could also be formally enforced by ZonMw through including participation as an assessment criterion. Depending on the topic, the committee could also decide to ask researchers to develop their proposal in consultation/collaboration with the target group.

Even when it is not mandatory, ZonMw can still steer on the patient perspective through the questions on the application form to which researchers submit their project. The research applicants are asked to mention in their action plan how they collaborated with patients. This is how ZonMw tries to stimulate researchers to reflect on, and take to account, the potential significance of their research for (future) users.

Research proposals are then reviewed by a patient panel. The researchers must often add a layman's summary of the project to their proposal. However, occasionally the summary appeared to be inadequate (Klop et al. 2004). In some programs it was therefore decided to replace the summary or to expand it with a so-called informed consent form (patient information sheet). With this form, patient referents can better assess how stressful the research and the chosen methods are for participating patients. The referents will receive a copy of the entire application and relevant

19

passages in the project application are pointed out to them, such as: the Dutch summary; questions about the patient population, (received) standard care and health benefits; study design, patient selection criteria, description of the intervention and patient outcome analysis; all forms of patient information.

In addition to patient referents, clinical and methodological referents are asked to assess the policy relevance and scientific quality of the proposal. The committee shall, on the basis of these combined assessments, decide on funding of applications (Klop et al. 2004).

4.3 Assessment procedure

The following procedure is used in one of the programs and provides a fair indication of general assessment proceedings at ZonMw.

Project Ideas (brief pre-applications)

The program uses a pre-selection through project ideas (call for ideas). With these project ideas, up to a maximum of three A4 text, the ideology behind a project will be expressed to the program committee. The purpose of the project idea-step is to select the most suitable and promising ideas. A project idea also offers the possibility to give some constructive feedback on the development from ideas to actual applications. Furthermore, this phase will make sure that the number of pending subsidy applications does not grow too large in relation to the available resources.

Relevance score of project ideas

The program committee will first assess project ideas themselves, with an emphasis on relevance. If the amount of subsidy applications exceeds budget, the committee will decide to perform a pre-selection: based on the relevance of the projects, and to a lesser extent on the quality of the ideas. The project ideas are also tested on various relevance criteria by the co-financing fund(s) and the patient panels. The funds review the relevancy of the project ideas on the fit with their research agenda's. The patient panels base their assessment on the earlier mentioned criteria.

Based on all the reviews (the program committee, the co-financing fund(s) and the patient panels), the program committee will formulate an opinion on which project ideas can be developed into a subsidy application.

Developed applications

In the full application the applicant indicates the actual steps that will be taken to apply the results of research into patient care. These (English) proposals are presented to a minimum of three

20

(inter)national scientific referents. The applicant will also be invited to respond to the referees' reports (adversarial procedure). The quality assessment is based on the judgment of the referees and the rebuttal of the applicant. In the assessment of the quality, both the quality of the underlying science and the quality of the translation are of importance.

The opinion on the quality and relevance results in a remuneration advice from the program committee. Afterwards the patient panels assesses these applications that are still eligible for funding again on the relevancy criteria. In addition, the co-financing funds asses if these projects are still relevant within their own research agenda. The ZonMw board decides in consultation with the co-financing fund(s) which applications eventually receive funding.

4.4 Example of application timeline

The first call for one off the examined programs opened in October 2013. The first projects to be awarded grants will start in the summer of 2014. A total time lapse of 8-14 months from submission till execution. A second call will be issued later in 2014.

4.5 Additional opportunities to participate

Funded Projects

Before officially starting, each funded project could be required to set up a specific user committee to facilitate the implementation of the project results. Both a representative of the patient panel or from the co-financing fund could serve on this committee. This way, participants can be informed on the progress of the project and the implementation possibilities.

21

Monitoring

Project managers prepare an annual, written progress report. ZonMw can decide to have these reports presented orally in the presence of (parts of) the program committee, patient panel and program council.

22

Ch.5: Goals and views on

participation

The general necessity of patient participation in health, research is not under discussion in this thesis. However, for a better understanding of the findings in this study it is good to know what the interviewees from the different programs aim for with participation. Do the respondents within the ZonMw programs share a similar perspective on participation?

5.1 Arguments to include patient participation

The arguments that legitimate participation within health research are threefold. First, numerous authors claim that the quality and relevance of research outcomes increases by including the patient perspective (Shea et al. 2005; Abma & Broerse 2010; Thornton 2006). Second, participation could be used to fulfil a (fundamental) right. Who has an interest in health care and research – like patients – must have the opportunity to be heard. Finally, arguments relate to participation in an instrumental/strategic manner. Because potential benefits of participation are the gaining of trust and access to institutions, enhance chances for fund raising and creating support for the implementation of outcomes (Schipper 2011). All of the mentioned arguments are present within the examined ZonMw programs and some are even noted by ZonMw in its external publications on participation.

5.2 Addition to professional perspective

Patients are the ones that need to deal with the effects of their condition in their daily lives. As a result, they can help detect topics which would otherwise had remained unnoticed in research. Such as fatigue, comorbidity, living with pain and side effects of medications (ZonMw 2013). The most expressed added value of participation by patients lies in the execution of an assessment on the burden and relevance of research. Various respondents claim that participation helps professionals to look from the perspective of the target group when searching for solutions for health care problems and to coordinate the communication towards them. These respondents mention a chasm between the life world of a researcher/professional and a patient. Or as Van der Kraan expressed this: “A

physician must once have been ill to truly know what it is to be patient.” Koza emphasizes that this

patient perspective really is a valuable addition, which enforces researchers to take the needs of the target group into consideration:

23

"Researchers have relevant clinical experience, but seeing therefore only a part of the palette. They do not spent evenings at home together with the person with the disorder and his informal carers. So they do not observe relevant things like pain or sleeping problems from practice. The fact that the experiential expert or informal carer can point this out to them, forces researchers to have an eye for the daily lives of people with a specific condition and to align their research questions/agenda accordingly.” - Ms. K. Koza (patient referent in panel)

Therefore, the general assumption among the respondents is that through a relevance assessment the overall quality of the research and its outcomes will increase. One could state that all respondents are, in one way or another, convinced of the added value of the knowledge and perspective of patients. This occasionally reaches beyond initial thoughts on the value of participation:

“I myself believe that participation also offers added value beyond the patient-perspective. This process has taught me that you do not merely have to ask experiential experts to evaluate the burden of studies on participating patients. The panel of patients proved to bear specific and substantive ideas on various elements of the research proposals, even about the methodology. My previous notion was that only scientist, people who are working in the field, could say something meaningful about research. After this positive experience of participation I would perceive my view as obsolete.” - Mr. B.

Kirchyünger (external program coordinator)

5.3 Studies have more societal value

When patients are involved in the decisions on research, not only studies that fit to interests of researchers and professionals are carried out. The research will also get significant relevance to the patients themselves. It connects to their questions, needs and problems. That legitimizes the research and increase its social impact (ZonMw 2013). Van der Putten compares the need for including the patient perspective in health research to a company where the management never visited the workplace and gets out of touch with it. According to Van der Putten, one could not come to research with societal value, if one has no idea of the problems and needs of end users. The argumentation put forward by multiple respondents is that research is best put in practice, when it corresponds to the social world of patients or their informal carers. If the target group does not want to participate in the research or is not keen on the (potential) outcomes, a lot of effort is depicted as useless. Van Leeuwen explicitly indicated that the legitimacy of the participation of patients (citizens) coincides with the financing of ZonMw, as it is financed from public funds through the Ministry of Health, Welfare and Sport.

24

5.4 Fulfilment of fundamental right

That participation is gaining importance at ZonMw is emphasized by the respondents. Next to this interest in participation at an institutional level, the choice for participation derives mainly from the objectives of the programs. “All of these programs are ultimately about them: the end-users. For

example, they revolve about the funding for studies on the development of a treatment they themselves or future patients, eventually could undergo.” - Ms. B. Sieverts (program secretary

ZonMw). This is driven by a moral imperative; who has an interest in health care and research must have the opportunity to be heard. However, simultaneously one could question if all stakeholders should have a say in research? For example the pharmaceutical companies who have a clear financial interest?

According to Van der Putten you need to take this fundamental rights perspective, as a central starting point when developing a project and keep trying to embed it during further phases. Even when it is challenging to do, for example when patients suffer severe mental illnesses or are too ill to participate actively or think alongside researchers. One can still involve interest groups or loved ones to represent the patient perspective: “There is always a direct or indirect way to which we can get

that perspective surfaced.”- Mr. M. Van der Putten (program secretary ZonMw). However, the

answers to the question whether participating in research or policy really is a right of patients, are not unequivocal: “Hmm a right... maybe more a necessity? If you really want the outcomes of your study

to align to a target group, then you need to give them a saying in one way or another.” - Ms. J.

Helderman (program secretary ZonMw)

According to Koster, it is important that participation in programs will also take "real, ideological shape". Even if you perceive participation as a right, you have to think of the implementation. To her this entails the involvement of professionals and participants “... into a dialogue about what they

want to achieve together. Second, they need to think about the impact of this answer on the desired shape of their collaboration.” - Ms. W. Koster (external program coordinator). Koster points out that

in retrospect she has the feeling that participation within her program had been shaped from thoughts on structure above content: “In the program design we needed representation of the

involved parties, therefore we had to position figurines in the right place and then we felt ready to start. Sometimes we were lacking that joint reflection on what we want to achieve, why we work together and what we need for good cooperation.” - Ms. W. Koster (external program coordinator)

To sum up this chapter, one could state that the arguments to include participation of patients in the ZonMw programs vary and a general consensus is lacking. The dialogue about the necessity of participation and the desired outcome(s) does not take place. This appears to be a complex process,

25

because according to Koster it is easier for professionals and participants to discuss the content of projects/subsidy requests, then to pronounce mutual expectations and necessities. Nevertheless, the respondents mutually agree that assuring the patient perspective within these ZonMw programs is valuable. Most of the respondents considered participation as a process in which the primary purpose is not focused on shared decision making, but on integrating the patient perspective in research. In the following chapters the effects and the achieved level of participation will be discussed in detail.

26

Ch.6: How do they do it?

In previous chapters, general structures, assessment procedures and various perspectives on participation were described. In this chapter the experiential world of the patient referents is deepened, as well as the decisions the other respondents faced with when implementing participation.

6.1 Scoring Relevance & Utility

As mentioned in Chapter 4 the patient referents are asked to review the projects on relevance for the target group, the benefits for the target group and the design of patient participation within the project. “In principle it's about patient participation and you have related criteria that you should

asses. You do not need to verify if the research methodology is correct or whether the study is cost-effective. There are a number of enthusiastic members who occasionally do make remarks on these matters, which is not prohibited but not in line with our task.” – Ms. K. Koza (patient referent in

panel). The respondents indicate that precisely for this last aspect the external scientific referents are asked. Next to quality, the committee then looks at relevance; relevance to the program and the Dutch healthcare as such. Before giving their final verdict, the committee is highlighted on the assessment of the patient panel: "That can really make a difference, especially when it comes to

projects that scored similar on other points. In such a case, the opinion of the patient panel may tip the balance in favour of one project over another."– Mr. M. Van der Putten (program secretary

ZonMw)

6.2 Recruitment

“We work with a patients organization, which has a pool of patients on which they can make an appeal.. Furthermore, they also recruit through calls on forums, twitter and through relevant newsletters. They then make a selection of who is or is not suitable. They had so much responses that people had to be put on a reserve list.” – Mr. M. Van der Putten (program secretary ZonMw)

This option to select participants is not ubiquitous among the examined programs, and recruitment sometimes happened under restricting circumstances. “I am under the impression that this was done

under great time pressure”, recalls external program coordinator Koster, “so who was able to participate was allowed.” Because in some programs more proposals were submitted at a given

moment than expected, immediate expansion of the panels was needed. Otherwise the workload for the panel would have been too much, according to Koza. Not even all respondents are familiar with the details of the recruitment process and inclusion criteria: “I think they just polled who would like to

27

ZonMw) Even panel members themselves did not always know if there were requirements that they needed to match.

The external program coordinator who was involved in one recruitment process acknowledges that referents are sometimes searched for in environment of the recruiter: “We, as a patient organization,

have a lot of contacts with patients. I myself went looking for people within my own network. So, I contacted patients who I thought would be potentially suitable for the panel and who would also like to do it.” – Mr B. Kirchyünger (external program coordinator) Thus candidates were sometimes

recruited in the direct environment of the patient organisation. Ms. Koza states that all referents in her panel are patients themselves, some of which are attached professionally to patient organisations. She is one of those panel members, and was recruited for the panel in an unusual way.

“I was approached to participate in the panel during my interview for a position within the patient association, which indeed was a pretty special way of recruiting. During the interview my chronic illness was mentioned, after which they spontaneously indicated that I would fit the panel perfectly. So I first went into the panel, when I did not even know that I would come to work at the association.”–

Ms. K. Koza (patient referent in panel)

First and foremost, the main selection criterion according to the respondents was the need to have patient experience. For some panels the required type of disease/illness was also defined. Those disease-specific panels are related to the topics that are depicted within the project proposals to evaluate.

A specific level of education was not always indicated in vacancies/recruitment texts. However, it was quite common for referents to be obligated to hand in their résumé during the selection process. Both the program secretaries as the referents claimed that a referent needed to possess a certain “level” in order to be able to participate. Argumentation is two-folded, first this level consisted off practical competences like English proficiency:

“That's just very practical requirement, because the documents are in English. As far as I know, there are no requirements in terms of age or level of education, but you should be able to participate. This is a thing you'll find out soon enough when you start as a panel member. After a few rounds, you know in terms of content and level of skills what you can expect. If that level is too high, I guess you just cannot participate. Because the same contribution is expected from everyone..”- Ms. K. Koza (patient

28

Sieverts expressed that this is why the funds she cooperates with pursue a university or college level when recruiting panel members, this is also recognized by other respondents: “I think there are a lot

of people end up stuck with high education”, says Koza, “not because it is so selected, but because of the practical work demands.” Koza says that her degree in health science was not a requirement, but

a personal gain which made it easier to participate; she knows what scientific research contains, can read proposals quite fast an understands what arguments ZonMw can use to reject projects and which not.

The second frequently heard argument is that referents need to be able to think from the vantage point of others. “It is often the question whether people project it all on themselves or whether they

also notice the bigger picture/the public interest.”- Ms. J. Helderman (program secretary ZonMw). This

ability is also linked to the structure and intensity of the form of participation.

“I believe that if you want to be a full member of the program committee you will need to have a little more of a transcending view then patients who would only reflect everything on themself. Also, because otherwise the discussions that play in program committee are not sufficiently understood. When participation on a less intensive level, it is possible more suitable to talk mainly from their own experiences.” – Ms. J. Helderman (program secretary ZonMw)

6.3 Reimbursement

Van der Putten explains that the participation of patients within ZonMw programs, on a panel or committee, are paid positions. Just like regular committee members or referents they receive a fee. The reimbursement varies by program, all programs cover travel expenses. In many cases there was also a form of attendance fee. Finally, in a few cases an additional reimbursement for the preparation time is granted.

However, there is uncertainty about the exact policy on fees and the amount that will be earned by the referents for each evaluation round. Prior to assessment round, these panel members know with certainty that travel expenses are reimbursed and that there will be an attendance fee. However, they do not know in advance what the exact amount will be that they receive per survey:

“What should be changed is that you never know in advance what you will earn. It sounds trivial, but one time you will get X amount per survey and in other rounds it is a fixed amount that does not depend on the quantity. Now I have 5 studies to assess, but no idea what the reimbursement will be.”

29

6.4 Support

Several panels arranged working sessions (up to 4 hours), where the referents were able to assess the project ideas or-proposals in f.e. groups of three. These working sessions were not mandatory, some (groups of) referents choice to do the assessment at other locations. “The referents told us themselves

that they valued a meeting where they could discuss the proposals together. Which I then organized for them. We turned this meeting into a diner party”. – Mr. B. Kirchyünger (external program

coordinator). At such arranged sessions a fixed contact of the involved patient organization was present to clarify questions of the referents. She/he facilitates these sessions and stood in direct contact with ZonMw or the researchers who wrote the subsidy requests. This individual was also concerned with necessary documentation and practical issues surrounding the participation of the referents (sending invitations, confidentiality obligations, assessment forms, disbursement of fees). He/she will sent the proposals to the referents and subsequently their assessments towards the researchers.

Helderman indicates that researchers in the program also receive support by ZonMw and patient organisations. They provide meeting and guidelines on the elements that should be included in the grant application. Furthermore, researchers can always call the patient organization for advice on how to embed participation in their research. Helderman does not think ZonMw programs can act completely without the support of patient organizations in matters of participation. Within the programs these organizations offer support to referents and provide training about what role referents should take in a panel. "That's really been a great investment. Both in terms of time and

finances. They also have expertise and experience with panels and focus groups. We do not have that, nor do we have the capacity, network and time for it.”- Ms. J. Helderman (program secretary ZonMw)

Hence, the patient organizations are mainly responsible for recruiting, supporting and training patients who participate as referents.

6.5 Training

“Most of those patients have never seen a research proposal, let alone written or evaluated one” that

is why Van der Putten believes training of referents is necessary. When starting out as panel member, the patients often received such an introductory training. Herein is discussed what the panel members need to look for in the project proposals, on basis of group discussions on previous proposals. “And during this group reflection you also hear things you had not seen yourself. Then you

notice that people focus on other things.” – Ms. K. Koza (patient referent in panel). Thus, one becomes

30

these trainings also focus on empowering both the patients as the researchers on “...how to ensure

their voice gets heard and end up in the project.”

The respondents indicated that as one goes through more funding rounds and gain more experience in assessing, you learn better what you need to pay attention to as a referent. This learning process starts at the first gathering/training:

“We got a research proposal and we got the normal questions there; the information that you needed pay attention to. The funny thing is that it was the first time I read a proposal and I really loved the research. Logically, I gave it a high rating. During the reflection in the group, and with instructions from the trainer, I noticed that I gained a more critical opinion on the proposal. So you learn to look with different eyes an investigation, which is very useful.” -Ms. D. Van der Kraan (patient referent in

panel)

The training and education for referents does not stop after the introductory period. Every year there is a range of educational materials and training provided which programs can use. However, Koster perceived the time it took to develop these materials as a great deficit:

“In the first instance, there were very few concrete tools on how you should design participation within a ZonMw program. We decided to develop a training manual and a basic-training. Ideally, we would have liked to be able to offer these matters when the funding rounds were launched. In the beginning, we were overtaken by events. At the time we had finished our support materials, the participants had already came up with these insights and best practices themselves, through trial and error.” - Ms. W.

Koster (external program coordinator)

In recent years, therefore, there is an increasing focus on tailored training. Now it is more common to ask for the specific needs of a panel/network or delegation, and to look at what is required to secure participation for the future.

6.6 Jargon

Because some of the proposals concern complex (fundamental) studies, much attention is paid to the lay summary. For example one research proposals is about finding networks in the brains. “Which is

already hard to understand for scientific advisory boards”, claims Kirchyünger, “...so imagine what they are like for people who are not experts in the field. You depend on lay summaries to provide information about what the research means for patients.” These mandatory lay summaries were seen

as a positive feature by all respondents. However, special meetings needed to be organized by the programs/ZonMw to educate researchers on the construction of such a lay summary. Mainly because

31

program officers noticed that “...some researchers inserted their English summary into Google

translate en used copy paste to put it in the proposal. Some were even surprised that the experiential experts looked ad these summaries and formed an opinion about them.” - Ms. B. Sieverts (program

secretary ZonMw)

Jargon within the proposals was not labelled as bothersome by the referents, but perceived as an extra burden. Referents had to look up (medical) terms and abbreviations.

“These proposals are clearly not written for us. But that is not necessary. Logically, these texts are full of jargon and abbreviations e.g. medical conditions and chemicals. So I look up these medical terms. Then I do know what the research is about, without having prior medical knowledge.”- Ms. K. Koza

(patient referent in panel)

The referents like to receive the entire application for the reviewing process, so that they can get a notion of the bigger picture. "You look at the summaries and use the other documents for

clarification. It is not worthwhile to read all the texts, otherwise I would rapidly have to read 1200 pages each round. " - Mr. P. Duijmelinck (patient referent in program-commission) When they

participated as committee member receiving the full application was useful as well, that way they could get involved in discussions on parts that transcended the lay summary.

Receiving patient information was perceived as a valuable asset during the participatory project. The referents appreciate this inclusion of information letters for patients, because it is an indication on the ability of the researchers to translate their studies.

6.7 Enforcement

As mentioned, researchers were often required to include patient participation in their application within the process of the project application, as well as formulating a lay summary for the referents. For Koza this way of working leads to effective participation. The mere fact that including patient participation is a requirement in the studies, is a huge deal for her. Without this inclusion, the research will not be approved. She believes that researchers underrated the strictness of this requirement in the beginning of the program. Koza described another requirement that was enforced, a patient organization needed to be contacted and involved by the researchers in the execution of the study. Otherwise the referents would take this into the most serious account or even put forth a negative advice. The respondents think that setting conditions for participation compels the researchers to do “something” with the subject. It is perceived as a strength. That it is a requirement to think about the implementation of participation, does not necessarily lead to actual

32

participation within the study. “Researchers can always claim that patient participation in their

particular research does not apply”, clarifies Sieverts, “... but then they must explain why.”

6.8 Co-funding

In some programs, the involved patient organizations also contribute to the financing. Because they are co-financing the funding round, they often also have input in the course of the round, a say in which projects are eligible and how many projects should be honoured. Sieverts programs revolves around translational research, which is also a priority of the collaborating health funds. Therefore they opted for co-financing. Besides sharing knowledge, experience and networks, the collaboration revolved around (financial) economies of scale. By appointing the funds as co-financier and laying the money together, a larger program with a larger budget emerged. "That's for the funds also

advantageous, because we could do the entire process of plotting the funding round and take care of all logistics. For us, the benefits lie with the provision of more subsidies” - Ms. B. Sieverts (program

secretary ZonMw)

Logically, the available budget influences the number of subsidies that could be granted. Kirchyünger indicates that his organization had budget reserved for one research: “With a directive of 1:3, it

meant that only three projects could go past the proposal-stage of which one got honoured”.

So, although the selection criteria for participants are not formally expressed/rigid, there appears to be a common understanding of what the job of being a referent requires of patients. Patients are selected on the base of these competences or obtain them through the act of assessing in practice. Furthermore they receive support and compensation for their efforts. By means of enforcement and funding is the input of patients structurally embedded in the appraisal procedure.