“Improving the Quality of the IAF Methods Database”
“The development of a dashboard”
Written By: Mariska C. Baanstra Student nr.: S1583492
Rijksuniversiteit Groningen Faculty of Economics and Business Master International Business and Management
First supervisor: Dr. B.J.W. Pennink Second supervisor: Dr. J.A. Neuijen
Preface
Before you lays my Master Thesis which is carried out in order to graduate from the Master International Business and Management at the University of Groningen.
I would like to thank my supervisor Dr. Bartjan Pennink for connecting me to the IAF Methods Database in order to carry out this research, and thank him for his support and enlightening ideas about how to carry out the research. I would also like to thank my second supervisor Bram Neuijen for his support.
I am grateful to Peter Bootsma and Jon Jenkins of the IAF Methods Database for sharing their knowledge and thoughts about group facilitation with me, and giving me new inspiration to write after each meeting.
Thanks go out the research team that helped me evaluate all group facilitation methods.
Sarah, Gerda, Suzanne, Carina and Stefanie, it would have been a lot more work without them.
Mariska C. Baanstra Groningen, June 2008
Summary
This research tries to answer the research question:
In what way can the group facilitation methods of the IAF Methods Database be measured to improve the quality of the database and be more useful to the users?
In chapter one the reason for the research is explained and the dashboard elements validity, reliability, usability, applicability and cultural sensitivity are chosen to be the measurement tools to evaluate the quality of the methods on.
In chapter 2 a literature research is done to find theory about these dashboard elements regarding group facilitation. With this theory several indicators are made which can be used to evaluate the methods. Based on these indicators a questionnaire is made and an evaluation form is created. This are the two ways in which the quality of the IAFMD will be estimated as described in chapter 3.
In chapter 4 we analyze the results of the questionnaire. The indicators are statistically tested to find out what are the solid indicators for measuring the quality of a group facilitation method. The results of these solid indicators and the results of the evaluation done by a special research team can be found in chapter 5. This chapter gives an overview of the total quality perception of the group facilitation methods. In chapter 6 we take closer look at the most popular methods. We also see which methods score the highest and the lowest scores based on the indicators.
In chapter 7 the dashboard element cultural sensitivity is analyzed. Since this element is measured in a different way, no indicators were selected. The methods are classified according to the dimensions of Hofstede, of which the results can be found in this chapter.
Finally a conclusion is given and some critical reviews about the thesis in the discussion.
Table of contents
Preface ...2
Summary...3
Table of contents ...4
1. Introduction...5
2. Finding the dashboard indicators ...12
2.1 Validity...12
2.2 Reliability...17
2.3 Usability ...20
2.4 Applicability...24
2.5 Cultural Sensitivity ...27
2.6 Summary ...33
3. Research Design...34
3.1 Questionnaire...34
3.2 Evaluation of methods by research team ...38
3.3 Conclusion...40
4. Indicator Analysis...41
4.1 Method ...41
4.2 Construction of dashboard element: Validity...41
4.3 Construction of dashboard element: Reliability...44
4.4 Construction of dashboard element: Usability ...45
4.5 Construction of dashboard element: Applicability...47
4.6. Summary ...49
5. Results of the questionnaire and evaluation per dashboard element ...50
5.1. Validity...50
5.2. Reliability...54
5.3. Usability ...57
5.4 Applicability...60
5.5 Summary ...64
6. The Dashboard Scores...65
6.1 Questionnaire and evaluation scores per method ...65
6.2. The quality of the database indicated by the dashboard ...67
6.3. The dashboard in practice ...69
6.4 Summary ...69
7. Cultural Sensitivity...71
7.1 Culture in general ...71
7.2 Results per method ...75
7.3 Conclusion...77
8. Conclusion and Recommendations ...78
9. Discussion...81
References...84
1. Introduction
This thesis is dedicated to develop a dashboard for the website www.iaf-methods.org.
This website facilitates all kinds of group facilitation methods available for users all over the world. To make the database of the methods more usable, a dashboard with several elements is developed to give a good overview of the quality of each method. The dashboard will be displayed at the website and contain information about the validity, reliability, usability, applicability and cultural sensitivity of the group facilitation method.
In this thesis we will find indicators for these five dashboard elements which are then used to construct the final dashboard.
To understand more about the reason for this research we will first explain more about the International Association of Facilitators Method Database (IAFMD) and about group facilitation methods. With this information the research question is formed, sub questions are selected and a description is given about how this research will be carried out. We will also explain why we chose the five dashboard elements validity, reliability, usability, applicability and cultural sensitivity.
The International Association of Facilitators (IAF)
The International Association of Facilitators was created by a group of professionals desiring a place for interchange, professional development, trend analysis and peer networking. A formal association was proposed and adopted at a networking conference in Alexandria, Virginia, in January, 1994. More than 70 people signed on as charter members. Since then the IAF has grown to over 1500 members in more than 63 countries.
The mission of the IAF is to promote, support and advance the art and practice of professional facilitation through methods exchange, professional growth, practical research, collegial networking and support services. This is accomplished through peer- to-peer networking, professional development and annual conferences. Methods exchange happens mostly through the methods database. (www.iafworld.com)
The International Association of Facilitators Methods Database
The IAF methods database (IAFMD) is an online methods resource and knowledge sharing community with 403 methods for enhancing group facilitation listed (June 2008).
The database is dedicated to providing managers, team leaders, and facilitators with online and face-to-face tools for creating, leading and following up group meetings. To use the methods, a free membership is required to get access to all the information about the methods. It has over 3825 members (June 2007) all over the world using these methods and adding new methods. Jon Jenkins is the founder of the database and has written the book “The 9 Disciplines Of A Facilitator” (Jenkins & Jenkins 2006). The IAFMD uses the name of the bigger organization IAF, but operates independently.
Group facilitation methods
Groups handle a range of tasks and take a variety of forms. Among these are committees, boards of advisors and directors, councils, quality improvement teams, research and development teams, and task forces. Work groups can be organized by type (e.g., executive, negotiation, production, advisory, service), function (plan, direct, integrate, display, treat), and settings (corporate, medicine, transportation, fast food, law). Groups may engage in problem solving, decision making, planning, and/or implementation (London, 2007).
Facilitation is making a process easier or more convenient. It is about process rather than content. How do you do something instead of what do you do. Facilitation is about the movement from A to B, to make it easier to get to a destination. You can facilitate yourself, someone else, are a whole group (Hunter, Bailey & Tailor, 1995). This research is about facilitating a group.
To enhance group facilitation, many group facilitation methods are designed. These methods concern all different facets of group facilitation. Many of these methods are in the IAFMD, and will be looked at in this research.
Reason for research
Said before, many methods are present in the IAF Methods Database. Though, at this moment there is no ranking or scientific information about the methods in the database, they only have been categorized according to subject. This is for example done in the master thesis of van Dijken (2007).On the website the methods have been categorized by the facilitative area. One can think of categories like decision making, team building, strategic planning and many more.
The database gives a good overview of many methods, but gives no additional information about the quality of these methods. Additionally, everyone can add methods, as long as they are registered as a supplier. There is no control over the quality of the method added. Therefore a facilitator has to rely on the description of the method to know if it is suitable for the situation he or she is in. Is the method valid? Is it reliable? Is it easy to use and applicable in many situations? Can it be used in different cultures?
These are aspects that are of importance for a facilitator before choosing a method. In this research we want to develop a tool to evaluate the quality of the methods with, and to find out what the quality of the methods in the IAFMD is.
Research objective
This paper will focus on creating a qualitatively higher methods database by creating a set of indicators to measure the methods on. The indicators will measure the validity, reliability, applicability, and usability of the methods and examine the cultural sensitivity.
The results of the research will be put in a dashboard. These dashboards will be applied to all methods and put on the IAFMD website. The dashboard will contain information about all five dashboard elements.
Research question
In what way can the group facilitation methods of the IAF Methods Database be measured to improve the quality of the database and be more useful to the users?
Conceptual model
In the conceptual model is shown how the research will develop. First the indicators that will measure the dashboard elements will be extracted from the literature about group facilitation. Then these indicators will be tested and the strong indicators will be extracted.
This will be the basis for the dashboard. The cultural sensitivity will be measured differently. The dimensions will be extracted from the literature and some test will take place. The results of these tests will be in the dashboard.
Figure 1 – Conceptual Model
Sub questions
To make the research question operational, sub questions are formed. These questions help to answer the research question.
Group Facilitation Literature
Indicators Validity
Indicators Reliablity
Indicators Usability
Indicators
Applicability Culture
Testing indicators
Good Indicators
Validity
Good Indicators Reliablity
Good Indicators
Usability
Good Indicators Applicability
Testing
Dashboard Quality of IAFMD Methods
1. How can the group facilitation methods be measured on validity 2. How can the group facilitation methods be measured on reliability 3. How can the group facilitation methods be measured on usability 4. How can the group facilitation methods be measured on applicability 5. How can the group facilitation methods be measured on cultural sensitivity
These five sub questions resemble the five dashboard elements that will be looked at closer in this research. Together they will be used to evaluate the quality of the group facilitation methods.
Why these dashboard elements are important
Many elements can be thought of to evaluate the quality of a group facilitation method on.
The most important for a group facilitation method is that it does what it promises to do and that it works in many different situations. Group facilitation does not always take place in the same setting, and a method is used to acquire a certain goal. Both reasons can be split up in several elements, explained here.
When you want to use a group facilitation method you want to know beforehand if it will give the results you want to have. If it will not give the result you want to have another method should be selected. This will influence the choice of a method in different situations. When a research is done, two standard criteria for a good quality research are reliability and validity. This is because they measure if the intended result is obtained, and if the results are not a coincidence (Baarda & De Goede, 2006). These two criteria are easily transferred to group facilitation methods. The same criteria count for the methods; to obtain the required result, a method needs to be valid and reliable. Both dashboard elements measure the quality of a method in a given situation. The meaning of these two dashboard elements and what their influence on group facilitation is will be explained in the next chapter.
Not every method is used in the same situation. Therefore we have to dashboard elements that measure if the method can be used in different situations. Again we can take the example of a research. In a good research the gained knowledge should be able to be
applied in reality (Swanborn, 1987). This is part of the theory that scientific ideas should hold in practice. As De Leeuw (2003) explains, ‘quality is usability’, and since we are looking for the quality of the group facilitation methods, the choice to evaluate the methods in usability is easily made. Applicability is part of usability. Though, it is important enough to be a separate dashboard element. This is because with usability we mainly focus on the effort that has to be put in to make the method a success in a variety of situations, and applicability means that it really works in a variety of situations.
The last dashboard element, cultural sensitivity has a high relation with applicability.
Only, the applicability dashboard element focuses on applicability in general and cultural sensitivity on the cultural applicability. This is important for the database as it is used all over the world. It gives the users of the database information about the usefulness in their specific culture.
Taken all together, it is important for a group facilitation method to be valid and reliable, because they guarantee the required result. It is important for a method to be usable and applicable, so that it can be used in a variety of situations, and cultural sensitivity will tell about the applicability in a variety of cultures.
In the next chapters we will find the indicators that will measure the dashboard elements and therefore answer the sub questions. The indicators will be used in a questionnaire and analyzed on their quality of measuring the dashboard element trough a factor analysis and Cronbach’s alpha. Additionally the methods are evaluated by a research team that scores all the methods on these dashboard elements. The results of the questionnaire and the evaluation will be looked at in another chapter and finally some conclusions will be drawn. The results of the research will be used to put a dashboard on the IAFMD website which displays the dashboard elements per method.
Summary
This research tries to answer the question how the methods of the IAF Methods Database can be measured to improve the quality of the database. This is done on 5 dashboard elements; validity, reliability, usability, applicability and cultural sensitivity. All five elements will be displayed in the dashboard on the website. With this research we will
find indicators to measure the group facilitation methods by for the first four dashboard elements and evaluate the methods on their cultural sensitivity.
In the next chapter We will look at the literature written about group facilitation to find the indicators that we can measure the dashboard elements on.
2. Finding the dashboard indicators
In this chapter we will find the indicators needed to measure the dashboard elements on.
This will be done trough a search through the literature written about group facilitation.
Indicators will be found according to the 5 dashboard elements in the form of validity, reliability, applicability, usability and cultural sensitivity. All dashboard elements will be described separately and explained. A description is given why the elements are of importance to test the method on, and which group facilitation theories are of importance for the dashboard elements so that the indicators for the group facilitation methods can be constructed. The results of these measurements will be put in the dashboard on the IAFMD website.
2.1 Validity
Validity is most of the time used in statistical measures. In the book of De Leeuw (2003) it is used for the quality of definitions, the quality of a conceptual model and on the quality of measuring procedures. Therefore it can be said that validity is a good measure to find out the quality of the methods in de IAF Methods Database. First we will explain about validity, then why it is important for the group facilitation methods and then how it can be linked to the methods. A method can be compared to a measuring procedure or observations methods, therefore it is useful to apply the validity rules for measuring procedures to the methods.
Validity in general
A method is valid when it actually measures what it is supposed to measure (De Leeuw, 2003). It has to measure the thing that it is supposed to measure and not some related variable. When referred to the group facilitation the method should create the effect in a group what the method promises to create.
There are 4 levels of validity; face validity, content validity, criterion validity and construct validity (Litwin, 1995). In face and content validity the measures are reviewed on their appearance by individuals and with respectively no formal training or with expertise about the subject. Criterion validity can be split up in two parts; concurrent and
1. Is the method theoretically founded 2. Does the method have positive critics
predictive. With concurrent validity it is measured on a “gold standard” and evaluated by that standard, and with predictive validity it is measured how well the measure can predict expected future observations. Construct validity measures how meaningful an instrument is which is usually determined after years of experience by numerous investigators.
For this research we will look at criterion validity, the concurrent part, since it measures how well one instrument stacks up against another instrument or predictor. This is what we want to do with the methods. We want to find out, with several indicators, which method scores high on validity. The “gold standard” will be defined according to the literature in the next paragraph.
Why is it important
It is of importance to the database that the methods will be measured on their validity level because it will increase the quality of the database. It will tell something about the soundness of the method (De Leeuw, 2003), and shows whether the method really does what is says it will do. Why would someone want to use a method when it does not give the result one wants to have?
Validity of Group Facilitation Methods
Many years scientist have been aware of the fact that problems arise when making decisions in groups. Though, the assumption is that groups have a greater ability to make the right decision than individuals do. The idea is that teams are able to effectively integrate various viewpoints and therefore improve the quality of the decisions made.
Though, process losses cause substantial productivity losses along the way. These losses can be prevented from happening when a theoretical baseline-model is developed. This provides a basis for the development of techniques, a framework for the integration of data and is an instrument to reveal inconsistencies of the results (Witte, 2007).
3. The method restrains from social loafing 4. The output is identifiable per person
5. Each participant has an equal opportunity to speak
There are hardly any guidelines written for the empirical procedures of developing a good method. Though, research has to be done before launching a group facilitation method.
When this is not done it can lead to some validity related consequences. A good example of how this affects a method is Brainstorming.
Brainstorming is mostly used to generate as many ideas as possible about a certain subject. Though, this goal is often not reached because of several factors. Social loafing is one factor, which gives the individuals the opportunity to make less effort because they work in a group. When there are reduced levels of motivation one can speak of group motivation losses (Steiner, 1972) and social loafing (Latané et. al., 1979). Social loafing occurs because people feel the task is boring and meaningless, and therefore try to avoid doing the task and try to freeride. They will avoid the task unless social constraints are implemented. To reduce social loafing, the method should prevent opportunities to free ride on other members’ efforts. Kerr and Bruun (1991) indicated this as the hide-in-the- crowd explanation, which means that the group members provide a cover of anonymity for the loafing participant. This can be prevented by making each participant’s output identifiable (Munro, 1997). Second, the individuals may have a fear of suggesting ideas which may make them look foolish (evaluation apprehension), and only one individual can suggest an idea at any moment. Therefore other ideas may be blocked (production blocking). These three factors count for brainstorming, but are applicable to other methods as well.
Trust is an important value in group facilitation. It can be divided into three parts;
between the facilitator and the client, the facilitator and the group(s) being facilitated and trust between the participants of the facilitated group. The last two are the most important when talking about group facilitation methods. Though, trust is not a skill that can be acquired, it can not be stated that “today we are going to learn to trust each other”. It is something that develops overtime but there are factors that can increase the development of trust after a while (Begoña Rodas-Meeker & Meeker, 2005).
7. Is individual input stimulated 6. The method creates trust
What is very important for group techniques is that they have the ability to enforce facilitative elements of group processes and to reduce inhibitory processes (consciously or unconsciously restraining from something) (Witte, 2007). The analysis of facilitative and inhibitory circumstances for group performances is essential. It can be analyzed on two conditions; the effects on individual and group level and on the cognitive, emotional and behavioral processes. An important aspect when improving group performance is that the individual positions have to be strengthened and individual motivation stimulated. This is not reached by just structuring the group process since group dynamic phenomena then still have an influence on the group productivity. Therefore the group interaction has to be limited as much as possible. Individual input is very important in Witte’s point of view; he has five postulates about it that will improve the eventual group decision.
The individual input has to be stimulated in the beginning of the process and the input has to be of high quality, more individual input is better by either one or more persons and the input has to be comprehensible. As last the high quality individual input has to influence the final group decision. Normative influences should be reduced as much as possible as they distort the individual input. Ideas will start to look alike and are not of high individual quality anymore. Emotional group cohesion does not per definition lead to better results as often is assumed (Mullen and Copper, 1994).
Witte (2007) tries to find out whether a theoretical framework builds a better and more efficient social facilitation technique for project teams. It is tested with the PROMOD method and is positively answered. Therefore we can state that methods with a theoretical basis will have a higher validity and will be of higher quality. In addition, a method found by a credible author or creator raises validity as well, since an assumption can be made about the theoretical base of the method. Additionally the creator can be contacted when there are any unclear situations in the method.
8. Is the creator of the method (well) known
9. The method is not created for large groups
11. The method does what it says it will do
10. Is there evaluation time calculated into the method
The bigger a group is the easier it is for an individual to perform social loafing. This means that a smaller group can be more effective (Kravitz and Martin, 1986). A group performs less than a person working alone. The bigger the group gets, the smaller the performance. Originally one might expect working in a group motivates because it fosters competition, though, people expect other people to perform social loafing, and thus perform less themselves which then divides the work evenly (Jackson and Harkins, 1985).
A large group is a group of more than 30 people up to several hundred people (Hunter et.
al.)
Brainstorming has been mentioned as an example of a low valid method before in this chapter. Furnham, (2000) finds another pitfall of brainstorming. Ideas are not evaluated and therefore there is only a list of ideas (Furnham, 2000).There is no closure which can lead to dissatisfaction and a lack of results. Thus, evaluation of the generated ideas contributes to the quality of the method.
To conclude we can say that the method has to do what it says will do to be valid. All indicators are an extraction of this.
2.2 Reliability
In statistics reliability is highly related to the measurement error. How strict is the instrument in giving the same result after it is repeated. No instrument is perfect therefore there will always be an error. This can easily be translated to the methods, which is described in this paragraph.
Reliability in general
Reliability is a statistical measure of how reproducible the instruments data are (Litwin, 1995). When there is less error in an instrument it will be easier to reproduce the same result as before. One should look at the reliability characteristics of the instrument when evaluating the value of a data set. De Leeuw (2003) mentions that something is reliable when a repeated measurement has the same outcome every time it is performed while nothing else has changed. To find out that nothing else has changed is very difficult, because you do not know this. Swanborn (1981), states that a method is reliable when there is no coincidence. When the coincidence is very small, a stable result will evolve after every measurement. You can evaluate reliability by doing the same test twice, with the same participants at two different time sets (test-retest). Reliability can also be tested by using different items to measure the same attribute. The difference should then only be a change in wording so that the exact same thing is measured (alternate form).
Why is it important
The reliability of a method is important for the stability of the method. It has to have the same result every time the method is used. When a method is not reliable it will not be satisfactory to the user since it does not get the result needed.
Reliability of Group Facilitation Methods
Reliability is used everyday when we speak about people or objects. We do this in such a way that when a person is reliable one can count on him, but when an employee is reliable he turns up for work on time and performs his duties as best of his abilities. A
1. Does the method give clear instructions for the preparation, 2. Does the method give clear instructions for the start
3. Does the method give clear instructions for the facilitator 4. Does the method give clear instructions for the participants 5. Does the method give clear instructions for the evaluation 6. Are these instructions detailed
7. Is the method translated in different languages
machine is reliable when it performs according to the expectation in the given time, starts every time it is turned on and never breaks down. A group facilitation method is reliable when it produces the same outcome or effect every time it is used.
When the group facilitation method is repeated it should have the same result as the previous time(s). To have the same result as the previous time the method has to be performed the exact same way is it was performed the previous time. Therefore, the method should have clear instructions how to use the method. One can think of instructions for the start, the facilitator, the participants, the relation between the facilitator and the participants etc. The more clear instructions are given, the higher the reliability of the group facilitation method will be.
To find the same outcome of a method every time will be difficult to measure in group facilitation methods, hence, how can you have the same outcome of a brainstorming session? When we think of an outcome it is therefore not an abstract outcome, but a more relative outcome. To take the example of the brainstorming method, the outcome of the method should be a good idea about for example decision making.
To implement a method well, it is important that the instructions are clear. To make the instructions more clear, it could be helpful if the methods are written in the language of the user. This will enlarge the reliability as it makes the instructions more understandable and they will be implemented the way they are meant to be implemented.
Kolfschoten (2007) wrote in her PhD research about instructions for collaboration engineering (CE) which can be compared to or be part of group facilitation methods. CE is an approach to designing, modeling and organizing repeatable collaboration processes
8. Are there step by step actions to undertake
for frequent high-value collaborative tasks that are carried out by practitioners using facilitation techniques and technology. Collaboration engineering stimulates methods using software products. Processes created with CE are processes that support the group to reach its required goal, like a group facilitation method. CE makes the processes reliable because the key facilitation interventions are codified and packaged so that it can be reused successfully. It is a fundamental assumption of the design of a repeatable (and therefore reliable) collaboration process each process has a specific sequence of thinkLets that create various patterns of collaboration among the team members. Each activity and design consists of one or more thinkLets. A thinkLet is “a named, packaged, scripted collaboration activity that produces a predictable, repeatable pattern of collaboration among people working towards a goal” (Kolfschoten, 2006). This indicates that every step of a process has to be described so that it can be repeated. This is not only the case in collaboration processes but should also be the case in group facilitation methods. When every step to be taken is described, every process will develop the same way and should have the same outcome.
2.3 Usability
Much has been written about the usability of products. A method can be compared to a product that has to be used, and therefore has to be “user-friendly”.
Usability in general
Quality is usability, as is written by De Leeuw (2003). He gives usability of a research as a very high indicator of quality. Something is usable when it is connected to the concrete problem of the customer and can be trusted. Something is usable when it “works”, which can be explained as that it does the thing that it is made for in the circumstances what it is designed for against reasonable costs. Therefore it should be solid and relevant.
Something that is relevant but not solid is not usable, just as the other way around.
Therefore a method should contain both factors to be of good quality.
Relevance contains timely availability, understandability and connection to the problem. Solid contains correctness, preciseness and consistency. De Leeuw (2003) also mentions efficiency and effort as an important aspect of usability. With the efficiency of a method one can think of the cost and benefits. When a method is used, the outcome is never guaranteed. One might assume that the result of the method will be positively on the group, but you can not know for sure. Therefore it is important how much effort you put in the method when you use it.
Many aspects are connected with usability. Validity is for example a part of solidity of a method, but in this research it is looked at independently because it is such an important matter. Reliability or consistency is also mentioned in usability, but is also separately examined in this research.
Other aspects, not mentioned by Swanborn or De Leeuw can be defined for usability of group facilitation methods. These are for example effectiveness and productivity of the methods.
Why is it important
Usability is important because you would not be able to use a method if it is not made for practical use. A method can be good in theory, but has to be used in real life. If the
1. The method has a clear goal 2. The method states its constraints
3. Does it allow people to express their feelings without judgment
method would be totally unusable the question should be whether it is then a qualitative method at all.
Usability of Group Facilitation Methods
For a group facilitation method to be usable, many factors come along. These are mostly practical factors, but also the factors that “deepen” the method.
The failure of group processes often lays in poor planning, lack of support by their creators or a breakdown in internal team process, for example communication (Salas, Sims & Burke, 2005). Just having skilled team members does not ensure success and therefore there are methods to be used. The methods therefore have to be usable and implemental so they can be used on the teams’ effectiveness.
“A team is two or more individuals with specified roles interacting adaptively, interdependently and dynamically towards a common and valued goal” (Salas et.al. 2005), but effective teamwork needs more than just a task, it needs a goal and objectives and constraints under which the team works (Salas et.al., 2005). The goal of the method always has to be kept in mind, it will keep the group on track and is a ground to stand on when the group is distracted (Hunter et. al, 1995)
Belfer (2001) agrees says that the individuals need a specified role, after having researched the usability of the De Bono’s Six Thinking Hats method. The method is useful because it allows legitimate expressions of feelings without apology or justification. It also allows unbinding of thinking and avoids confusion as it does not try to do everything at once and it provides a simple way of switching of thinking. All team members are stimulated to think in specified ways and it separates the ego from performance in thinking.
Druckenmiller (2007) emphasizes the importance of effectiveness and efficiency of a method. For a method to be effective it needs to be consistent (reliable), and the people
5. The method identifies what has been learned
6. Little effort is needed
4. The method is understandable
7. The method description provides a clear structure
who are using the method have to understand the method. The time needed to understand the method is also part of efficiency, therefore for a method to be effective and efficient;
it needs to be easy to understand so that everyone knows what he or she should be doing.
Another essential part of group facilitation is learning. Before starting with the method it has to be clearly identified what is going to be learned and afterwards what has been learned. This has to be acknowledged and afterwards celebrated. This can be done trough evaluation of the method afterwards the facilitation project.
Additionally the effort that has to be put in to the method is an important indicator of effectiveness. Effort can be measured trough the time that has to be spent with the group, the preparation time needed and the money that has to be available. The less effort needed the higher the possibility that the method will be effective, as the method has a smaller chance of failing.
In the paper of Laughlin et. al. (2002) transparency of the right solutions and ideas is the most important factor for a good group facilitation method. This means that the arguments, suggestions and ideas have to be demonstrable. This can only be done when there is a good, which means an appropriate, structure for the discussion about a certain subject. A clear structure and a clear goal to make a method usable are also put forward in the studies of Druckenmiller (JAAR) and Hunter et. al. (1995).
8 The method is easy to use
9 The method does not require professional facilitation skills
In general we can say that the usability is defined by the fact that the method is easy to use. When a method is easy to use it does not require any special facilitation skills, other than the regular facilitator possesses. This leads us to the final two indicators for usability.
2.4 Applicability
For the applicability of the methods it is important that the methods can be used in real life and not only look good on paper and can be applied in a variety of situations.
Applicability in general
Applicability is something different for every research, but as it is defined in the dictionary it is “applying or capable of being applied; relevant; suitable; appropriate: an applicable rule; a solution that is applicable to the problem” (dictionary.com).
It is therefore a very practical use. Can the method actually be used in real life? The applicability of the method depends on whether the method really works or not. It will be measured whether the method can work and under which conditions. The method should be able to be applied and not only be good in theory, and be applicable in multiple situations. Swanborn (1981) mentions that the method has to help solving the problem, it has to be applicable and have the means to handle the problem. Usability and applicability are two measures very close to each other. One can say that applicability is a part of usability (Swanborn, 1981). Even though, applicability is looked at separately because it is such an important matter in group facilitation.
Why is it important
Group facilitation methods should not only be good in paper, but they should also work in real life, this is what they are designed for. There is no use if they are not applicable in practice. Group facilitation is different every time. A group is different, a location etc.
For a group facilitation method to work well, it should be easily adapted to all different situations.
Applicability of Group Facilitation Methods
Applicability has a high resemblance with usability, both dashboard elements deal with the practical side of the group facilitation methods. Most important is that the method is easily transferrable from theory to practice in multiple situations. To be adaptive to
1. The method is flexible
2. The method is easiliy modified
3. The method is not sensitive to changes in circumstances 4. Instructions do not have to be followed strictly
5. The method is not complex
6. The method is not difficult to facilitate
7. The method does not require certain conditions
multiple situations flexibility is required (Snow & Snell, 1993). Flexibility is the ability to respond to various demands from the environment (Sanchez, 1995). To respond to these demands the method should be able to be modified. With different modifications for every situation the method will still get the required result and work well in many situations. Modifications will not be able when the instructions for the method have to be followed strictly to make the method a success.
A method can be less applicable when it is easily interrupted by external influences.
Especially a more complex or intense method will suffer from external influences. When something is complex it is more difficult to process and more time is spent on solving the complexity of the method than actually using it (Dorner, 1996). Additionally, when a method is complex the chance of not reaching the wanted outcome is larger because some facets of the method may be wrongly interpreted. When a method is more complex, it is also more difficult to facilitate. If a method is difficult to facilitate it is not very applicable in many situations.
When certain conditions are required for a method, it is less applicable. Under certain conditions we understand a required group size, a special location, special skills and many more. The more requirements a method has the less applicable in different situations it becomes.
One of the certain conditions can also be high participation from the participants. It is of course very good when the participants perform high participation, but when it is required it is less ideal. This is because the chance of failure becomes bigger. If there is a participant that does not participate at the high level that is needed, the outcome of the
9. The method can be applied in a wide spectrum of goals 10. The method works everywhere
8. The method does not require high participation from the participants
method will be different. It also makes the method less applicable in multiple situations because participants will not always have the ability, energy, to participate on a high level.
For example, participation in a group process can be voluntary or mandatory. The voluntary participants will more likely to be motivated to give high participation than the mandatory participants (Hunter et. al.).
Taking in mind all these found indicators we come back to two points all the time;
whether the method can be applied in a wide spectrum of goals and that it will work everywhere. Therefore this is put into an indicator as well.
2.5 Cultural Sensitivity
The IAF Methods Database is an international site for facilitators all over the world; the methods are used by people in different countries with different cultures. In group facilitation a lot of interaction takes place. As the participants on the group facilitation will be of different cultures (may or may it not be in the same group), they will have different perspectives and implicit ground rules for interaction (Hogan, 2005). In workshops, participants from different cultures will have a different view on the role a facilitator should have, the facilitation process and the desirable behavioral norms (Verghese, 2003).
Differences in cultures can be assessed in many different ways, such as multi cultural groups or a group of the same culture with a facilitator with a different culture.
We look at one culture in the whole group with the facilitator having or not having the same culture. We are looking for a method that is be suitable or accepted by a specific culture. It is easy to stereotype a culture, but with much research done (for example Trompenaars, 1993; Hofstede, 2005; Hall, 1990) assumptions can be made about cultures and their rules and perspectives.
For cultural sensitivity we use the research from Hofstede (2005). We chose to use the dimensions of Hostede because they are very well known, cover all aspects of cultural concern of group facilitation methods, are easily explained to the users of the IAFMD, and covers national culture which can also be applied to organizations. This is very applicable for group facilitation methods.
No set of indicators is select in this chapter, as this dashboard element is not measurable by indicators. We will explain about the Hofstede dimensions and their influence on group facilitation methods as the group facilitation methods will be measured on these dimensions.
Cultural Sensitivity explained
Culture has influence on many features in daily life. The world is full with people that do, feel, think and act differently in groups and nations. At the same time there are many people, groups and nations that have a lot of the same characteristics. Everyone has a so- called mental program with several patterns of thinking, feeling and acting inherited from
their childhood. Culture is learned, it is derived from someone’s social environment.
Differences in culture can often be seen by the region, religion, ethnicity, gender, generation and class. Hofstede (2005) defined 5 values to categorize a culture on. They will be described here.
Power Distance (PDI) measures the dependence of relationships in a country.
When a country has low Power Distance, people depend less on bosses and have a preference for consultation. There is a small emotional distance and subordinates will easily approach their bosses. Where PDI is high there is high dependence and subordinates are not very likely to approach their bosses.
Individualism and collectivism (IDV) explains if a country is more individual or group interested. In collectivist countries the power of the group predominates, the family is very important. Most countries in the world are mainly a collectivist country but a minority has an individualist culture where the interest of the individual is most important.
Everyone is supposed to look after themselves while in a collectivist country people are integrated in the in-group with high lifetime loyalty to each other.
Masculinity-Femininity (MAS) describes the masculine and feminine characteristics of a country. In a masculine country emotional gender roles are clearly dissimilar, men are assertive, tough and focused on material success and women are modest, tender and concerned with the quality of life.
Uncertainty Avoidance Index (UAI) rates the extend to which the members of a culture feel threatened by unclear or unfamiliar situations. This can be rated by nervousness or stress. In a country where the UAI is strong there will be a lot of stress and people think that what is different is dangerous. In weak UAI countries there is little stress and are curious about the unknown and differences. Also results are attributed to a persons own ability while in a strong UAI country results are attributed to the circumstances or luck.
Long- and Short-Term Oriented (LTO) stand for the goals set in the life of the members of the culture. Long Term orientates towards future rewards such as perseverance and thrift while Short Term focuses on the past and the present especially respect for tradition and “face” and fulfilling social obligations.
We chose to use Hofstede on this measure as he is the most well know researcher of culture, and has values for almost every country in the world, which is very useful for this research.
No indicators are made for this dashboard element as it is very generalizing. During the research the dimensions will be explained to the respondents so that they can evaluate the method on its cultural sensitivity themselves.
Why is it important?
As the methods are used all over the world, it is interesting to find out whether some methods are more suitable for some cultures than other, or that they have the capacity to be modified to the requirements of a specific culture. A method could be more effective in a country where the LTO is high then where it is low. When keeping this into mind a facilitator can make a more thorough selection of which method to use in a group.
Cultural Sensitivity in Group Facilitation Methods
The influence that the five dimensions of Hofstede have on group facilitation will be explained here.
Power Distance: Asian and Latin American countries have high power distance. This means that they are more reluctant to ask something from a superior and they do not appreciate consulting subordinates before making decisions. In group facilitation this will mean that when a method requires several working levels of the company, the method is likely to be less effective in Asian and Latin American countries. This is because subordinates will not easily say what they think, because the superior is always right. The people from high power distance cultures are not used to discussion. Scandinavian countries and Anglo-Saxon countries have very low power distance, implying that they will not hesitate to ask a superior a question, the superiors are not autocratic or paternalistic and it is appreciated when a superior consults his subordinates before taking a decision.
Individualism / Collectivism: Anglo Saxon and European countries score high on individualism. Countries with a high welfare level score higher on individualism than low welfare countries. Hofstede and Hofstede (2005) state that collectivism is more present than individualism which means that most countries putting the benefit of the group first.
Someone living in a individualistic culture rates high importance on freedom on work level. Work also has to be challenging and give personal satisfaction. A person from a collectivist culture rates training, good work circumstances and using talents very high. Though, in rich countries these three values are taken for granted in working life.
We can therefore assume that individualistic countries will make more use of the group facilitation methods, but that collectivist countries appreciate them more. In the Chinese Value Survey (Bond, 1991), the integration dimension was created, which corresponds to individualism. Students from individualistic countries rated the values tolerance, harmony with others, trust and satisfaction with own situation higher than their collectivist colleagues.
In a collectivist country an own opinion does not exist. One thinks what the group thinks and goes along with this. In individualistic countries it is stimulated to form your own opinion. In collectivist cultures the peace and harmony has always to be kept in a group. In individualistic countries it is more important to stay true and say what you think.
In collectivist countries verbal communication is not as important as in individualistic countries. This is also researched by Hall (1977) in high and low context cultures. In high context cultures little has to be said or written because everything is already decided by the circumstances or the personal knowledge of the person. This is mostly present in collectivist countries. In low context countries experiences and expectations are to a smaller extent used to communicate. Information is explained through words, instead of the context. Collectivist people are more introvert, individualistic people more extravert.
Most management techniques and training programs are mainly designed for individualistic cultures and do not hold in a collectivist culture. Methods that stimulate to be open and honest and share feelings with others will be less effective in collectivist countries.
Masculinity / Femininity. A country is more masculine when the employees rate high income, acknowledgement, promotion possibilities and challenging work as important on a job. The country is more feminine when good working relation with superior, good cooperation, living area is highly valued. Scandinavian countries score very low on masculinity which implies they are very feminine. Masculinity is not very easily to divide into regions. Some Asian countries are very feminine except for China and Japan. Latin countries score high on femininity, while Mexico scores very high on masculinity. All former British countries score high on masculinity. Welfare does not have an influence on masculinity.
In masculine countries responsibility and decision making are characteristics of men, women are careful and gentle. In feminine countries characteristics of men and women are more close to each other. Winning is very important in masculine countries.
Masculine cultures are more interested in facts, while feminine cultures are more interested in the idea behind the fact. In feminine countries problems are solved by negotiation and compromising, while in masculine countries the strongest player wins.
Management happens on the base of intuition and consensus in feminine countries and on decision making and assertiveness in masculine countries. The decisions are made individually.
Uncertainty avoidance means that when a culture scores high on this dimension, people are insecure in unknown situations. Latin countries score high on UAI, just as Eastern Europe and Japan. African, Anglo-Saxon and Scandinavian countries have medium to low uncertainty avoidance. Cultures with high UAI are more expressive, it is more accepted that one raises his voice and show emotions. People are not happy when they do not know what to expect. In a group facilitation method it will therefore be important to explain what will happen during the method. High UAI cultures like structure, while low UAI countries like open ended goals. In low UAI countries rules only exist where the purpose of the rules are clear. There are only necessary rules, and people live up to these rules. In High UAI cultures people like to work hard and do a lot of things at the same time. Time is money. Conflicts have to be avoided and everything must be as clear as possible. Low uncertainty avoidance counties are more tolerant about what is different.
High UAI cultures think that what is different is weird. Low UAI have more attention for the process of decision making while high UAI care more about the outcome.
Long- and Short-term Oriented. Long term orientation is valued the highest in Asia, and in Brazil. Anglo Saxon countries score low and Scandinavian countries in the middle.
Short term oriented cultures like quick efforts with quick results while long term oriented cultures will invest in the future and with perseverance leading to gradual results.
Methods that give results in the future will therefore be more effective in a long term oriented culture. Success and failure is a coincidence in short term oriented cultures whereas the amount of hard work determines the success in long term oriented cultures.
Strong values at work in short term oriented cultures are freedom, rights success and individual thinking. People want cognitive consistence. In long term important in work are learning, honesty, flexibility, liability and self discipline. Differences in opinion are not a crime.
Calori (1994) makes a division of the world, especially Europe, based on Hofstedes dimensions. He says that in a management perspective, Japan and the United states are worlds apart, and stand opposite of each other in management style. Germany is more like Japan and the United Kingdom more like the United States. Europe is divided into 4 groups:
1. Germanic (Germany, Austria, Switzerland)
2. Scandinavian (Sweden, Finland, Norway, Denmark) plus The Netherlands 3. Anglo-Saxon (Britain and Ireland)
4. Latin (France, Spain, Italy, Portugal, Greece)
From the work of Hofstede himself we can state that Latin America is part of the Latin countries group as well, and that the United States and Australia and New Zealand can be part of the Anglo Saxon group. Asia (except Japan) does not correlate with any culture in Europe and is therefore a group on itself, as well as the African countries. There are many other ways to which you can classify the countries into groups (Simonet,1992; Albert, 1991), but as we use the dimensions of Hofstede it is the most appropriate to make a division based on the scores of his research.
2.6 Summary
In this chapter we found several indicators for every dashboard element to measure the group facilitation methods on. All dashboard elements have between 7 – 11 indicators that indicate the score of the dashboard element. Only the dashboard element cultural sensitivity will be measured in a different way. An explanation is given what influence culture can have on group facilitation methods.
In the next chapter we will look at how these indicators can be measured. This is done with a questionnaire and a special research team.
3. Research Design
In the previous part the theoretical background was explained. Several indicators to measure the group facilitation methods were formed. With these indicators it is possible to evaluate the quality of the group facilitation methods in the IAF Methods Database.
Several choices have been made on how to evaluate the group facilitation methods which will be explained in this chapter.
Indicators
The goal of the research is to create a tool by which the group facilitation methods of the IAFMD can be measured. To make this operational the dashboard elements had to be split up in several indicators that could be measured. These indicators were thoroughly constructed through theoretical research which is described in the previous chapter. Every dashboard element has several indicators. For the questionnaire these indicators are narrowed down to 7-11 questions. For the evaluation they are narrowed down to 2-4 indicators. These questions and indicators together form the score of the dashboard element. The higher the score, the better the dashboard element is represented in the group facilitation method. Cultural sensitivity will not be evaluated in the first part in the research, as no indicators are set for the dashboard element. A separate chapter is devoted to this element. First we will explain the procedure of the questionnaire.
3.1 Questionnaire
With a questionnaire you can easily approach large numbers of people, and measure a large number of variables. It can be used to measure opinions, motives and beliefs (Swanborn, 1987). Because of these reasons and because it is cheap and quick, the choice was made to send out an online questionnaire to all registered users of the IAFMD.
The goal of the questionnaire was to find out whether the indicators really measure the validity, reliability, usability and applicability of the methods. Additionally the opinion of the users about the methods they use was generated. Also some questions about cultural sensitivity were in the questionnaire. This questionnaire was sent to 3610 users of the IAFMD which were registered at the time of sending (02/04/08).
The questionnaire consisted of 63 questions based on the theoretical background. Some questions are split up in sub-questions, resulting in a total number of 97 questions. The users of the IAFMD receive an email explaining them the purpose of the questionnaire and the reason why they should participate. When they click on the link they open the program of the questionnaire (www.thesistools.com). The questionnaire starts with a short explanation and an expression of thankfulness of the respondent for his or her willingness to fill in the questionnaire. At the end of the questionnaire they have the possibility to fill in their email address to receive a copy of the results of the research.
The first questions are basic demographic questions and questions about group facilitation. On the second page the respondent is asked to choose a method that he or she is willing to answer some questions about. The rest of the questionnaire is about this method. The questionnaire is made up of several open and closed questions and a view checkboxes. Most questions are closed questions representing a statement. These statements involve the indicators found in the theoretical background. The statements are measured by a 6 point-Likertscale. This method makes it possible to carry out several statistical tests. The choice to use a scale without middle class was made to force the respondents into a direction of either agreeing or disagreeing with the statement. Some questions are formulated in a positive way and some questions in a negative way, which was done to avoid a standard answer rhythm. The usability and clarity of the test was guaranteed by having tested this on a view test respondents.
To acquire as many as possible respondents the questionnaire was announced in the monthly IAFMD newsletter. This newsletter is sent every month and is only received by the users that are subscribed to the newsletter. Two weeks after the questionnaire was first sent out, a reminder was sent to remind people that they still had to fill in the questionnaire. Two weeks after this reminder the questionnaire closed down. It has been open for four weeks in total. The complete questionnaire can be found the in appendix I.
As all the users of the database are registered with an email address, it was easy to reach them through email.
Population
The questionnaire was sent to 3610 registered users of the IAFMD. All these users have a free registration at www.iaf-methods.org. You need to register if they want to see the full contents of the group facilitation methods. The only other email the users receive from the IAFMD is a monthly newsletter which you have to subscribe to. The questionnaire was announced in this newsletter too.
Results
Sample
In the end 259 people answered the questionnaire. After deletion of the incomplete questionnaires 178 questionnaires remain1. Because the questionnaire was reasonably long (63 questions) not all participants completed the whole questionnaire. To maximize the results, the questionnaires that are filled in for more than 80% are kept into the database. The missing values are replaced by SPSS to obtain a more thorough database.
To get a good impression about the people who filled in the questionnaires it is wise to look at some frequency tables.
Occupation
The IAFMD is mostly used by facilitators, for which it is in the first place designed. But it is also used by managers, consultants, teachers and students. The questionnaire gives a good representation of this population. Almost all respondents are facilitators; there is only a small number of other respondents. This is good, as the questionnaire assumes that the respondent is a facilitator. Additionally it is interesting to see the opinion of participants and managers too.
1 A factor analysis is carried out for the research. This requires 5 respondents per variable. There are
When I use a group meeting method I am in the first place a (1 answer possible)
156 89,7 89,7 89,7
9 5,2 5,2 94,8
6 3,4 3,4 98,3
3 1,7 1,7 100,0
174 100,0 100,0
Facilitator Participant Manager Other Total Valid
Frequency Percent Valid Percent
Cumulative Percent Table 1 – Occupation of respondents
Experience
The respondents have several differences in experience. This is regarding the time they spend on facilitation and the years of experience they have. 34% of the respondents have more than 15 years of experience in facilitation; only 5 % has less than 1 year of experience. Almost all respondents are part time facilitators. This is representative for the population, since most facilitators are freelancers with other occupations as well. Only 8% spends more than 80% of its time on facilitation, almost 60% spends less than 40% of its time on facilitation.
Demographics
More females than males filled in the questionnaire (54%/46%). Though, the difference is not that large that there will be a tendency to more female or male answers. Other researches have been done among facilitators as well, which also show an almost equal deviation between men and women.
Of the respondents 52% is above the age of 45. This is a high age, but normal in facilitation. As we have seen before the facilitators have a lot of experience, which can only be reached at a higher age. Only 3% of the respondents is younger than 25.
Nationality
Many nationalities use the IAFMD, which is also represented in the questionnaire. In total 35 different nationalities participated in the research. Most of them are from the United States (33%), Canada (15%) United Kingdom (10%) and The Netherlands (9%).
All continents are represented. To make the wide range of nationalities manageable they have been classified according to Hofstede’s dimensions. We chose Hofstede for this
classification because then the countries with the same culture, which assumingly have the same preference for group facilitation methods, are together. The nationalities are divided into 6 areas; Africa, Anglo-Saxon countries, Asia, Germanic countries, Latin countries and Scandinavian countries. Not all areas are equally represented. The Anglo- Saxon group is most largely present. This reflects the usage of the database, but is difficult when analyzing general opinions. This has to be taken into consideration when generalizing the results.
Table 2 – Hofstede culture of respondents
3.2 Evaluation of methods by research team
Not all methods are evaluated in the questionnaire. The respondents could choose themselves about which method they wanted to answer the questions. Only 12 methods are evaluated more than once. Some of the methods mentioned are not even in the IAFMD. Because of this, a research team was set up to evaluate all methods in the database on the most important indicators able to find in the method description. In this paragraph we will explain about the research team and the evaluation performed on the methods.
Research team
To evaluate all 371 methods in the IAF Methods Database a research team was set up.
The research team consisted of 6 people, including the researcher. The team was formed
Hofstede
9 5,1 5,2 5,2
116 65,2 66,7 71,8
9 5,1 5,2 77,0
8 4,5 4,6 81,6
13 7,3 7,5 89,1
19 10,7 10,9 100,0
174 97,8 100,0
4 2,2
178 100,0
African Anglo Saxon Asia
Germanic Latin
Scandinavian Total
Valid
missing Missing
Total
Frequency Percent Valid Percent
Cumulative Percent
by the researcher and formed especially for the cause of evaluating the group facilitation methods of the IAFMD for this research. All team members are female students of the Hanze University of Applied Sciences or the University of Groningen. They have studies of Communication, Law, Social Sciences and Education. None of the team members was familiar with the subject; therefore a short briefing took place before the evaluation of the methods. The methods were graded by common sense and insight which are therefore not completely scientifically reliable. The results are used as a control factor.
Method descriptions
The evaluation of the methods was performed on the method descriptions. These descriptions can be found on www.iaf-methods.org2. The method description is made by the supplier of the method. This supplier can be the developer of the method, or someone who has seen the method somewhere else and wants to share it with other people. When you want to supply a method, you have to register as a supplier. When supplying a method, 35 fields of information about the method have to be filled in. This contains for example the instructions for the method, how much time the method takes to complete, if the method is flexible, etc. None of the information to be filled in is complementary.
Process
The methods were evaluated on 14 indicators. These indicators correspond with the indicators found in chapter 2. These indicators are narrowed down indicators from the indicators found in chapter 2, to be able to work with from the method description. The indicators are classified according to the dashboard element they belong. The indicators are shown in table
Table 3 – Indicators per dashboard element for evaluation
Validity Reliability Usability Applicability
Group size Recognizable
components Clear intent / Purpose Level of participation Follow up required Instructions / Procedures Optimal time Ideal conditions Developer known Potential pitfalls Difficulty to facilitate References / Book Pre-work required Flexible
Not all the dashboard elements have the same amount of indicators. This is because the information of the method descriptions is not more extensive. All indicators were given a score of 0, 1, 5 or 10 with 10 being the most positive, 5 somewhat positive, 1 the least and 0 not available. Some indicators could only be graded positive or negative. For the complete evaluation form see appendix III. All methods (371) present in the database at that moment (April 16, 2008) were evaluated according to this evaluation form.
3.3 Conclusion
In this chapter we have described how to analyze the indicators that will measure the group facilitation methods. Through a questionnaire among the users of the database and an evaluation done be a specially designed research team the indicators will be tested.
Not all indicators found in the theory will actually measure the dashboard elements. The actions undertaken to obtain the final indicators are described in the next chapter.
