“The Evidence Map as a Tool for Risk Communication”
MASTER THESIS OF:
Anke M. van Wingerden
GRADUATION COMMISSION:
Dr. M. Kuttschreuter Dr. M. de Jong
The Effect of Information Lay‐out on Risk Comprehension, Risk Perception,
Perceived Reader Friendliness and Preference
‐ 2 ‐
COMMISSIONED BY:
University of Twente
IN COOPERATION WITH:
Forschungszentrum Jülich
Faculty:
Behavioural Sciences
Study:
Communication Science
Postbus 217 7500 AE Enschede
www.gw.utwente.nl
Forschungszentrum Jülich
info@fz‐juelich.de
‐ 3 ‐
Dutch summary
Mensen hebben moeite met het begrijpen van wetenschappelijke risico’s, onder andere vanwege het gebrek aan eenduidigheid van deze informatie. Toch is kennis van deze risico’s van zeer groot belang om een veilig leven te kunnen leiden. Om het begrip met betrekking tot deze risico’s te vergroten is de zogenaamde evidence map ontworpen. Dit onderzoek onderzocht het effect van deze presentatievorm voor het begrip van risico‐informatie, maar ook voor risicoperceptie, leesbaarheid en voorkeur voor een specifieke presentatievorm.
Het onderzoek bestond uit een experiment waarbij twee risico’s (blootstelling aan elektromagnetische straling (EMF) en ervaren gezondheidsklachten bij kinderen en blootstelling aan EMF en het ontstaan van kanker) golden als within‐subjects factor en drie presentatievormen (beschrijvende tekst, evidence map en een combinatie van tekst en map) als between‐subjects factor golden. In totaal werden 118 proefpersonen aselect aan een van de zes experimentele condities werden toegewezen. Chikwadraattoetsen en ANOVA’s toonden aan dat de condities vergelijkbaar waren met betrekking tot eventuele storende variabelen (demografische gegevens, leessnelheid, mate van voorkennis en risicoperceptie) met uitzondering van de variabele leeftijd. ANOVA’s toonden echter aan dat dit geen invloed had op de resultaten van de meeste afhankelijke variabelen, behalve voor verwerkingssnelheid en mate van wetenschappelijke kennis over de risico’s.
De resultaten toonden aan dat het effect van de evidence map met betrekking tot het risicobegrip beperkt was voor beide risico’s. In tegenstelling tot wat de Mental Model Theory en de Assimilation Theory voorspelden, werd de informatie niet beter en sneller verwerkt door proefpersonen die de evidence map gebruikten. De effect size toonde echter wel aan dat kleine verschillen in verwerkingssnelheid significant zijn wanneer meer proefpersonen worden gebruikt. Aan de andere kant snapten proefpersonen die de tekst gebruikten de informatie niet beter of sneller, waardoor ook de voorspellingen van het Active Processing Model niet werden bevestigd. Opvallend was de sterke voorkeur voor het gebruik van de evidence map boven een beschrijvende tekst en een combinatie van tekst en map. Dit werd echter niet ondersteund door de resultaten met betrekking tot de ervaren leesbaarheid.
De evidence map bestond weliswaar uit makkelijkere zinnen, was korter en beter in het samenvatten van essentiële informatie, maar de beschrijvende tekst werd als duidelijker en vloeiender ervaren. Het onderzoek van Börner, Schütz en Wiedemann (2009) vond nagenoeg dezelfde resultaten voor vloeiendheid en de eenvoud van zinnen. Dit impliceert dat deze kenmerken als inherent aan de presentatievorm kunnen worden gezien. Ondanks de verschillen in de ervaren leesbaarheid van de presentatievormen werden geen significante verschillen in risicoperceptie gevonden.
Hoewel het doel van de evidence map niet werd bevestigd, toonde het onderzoek wel een sterke significante voorkeur voor de evidence map aan. Op basis hiervan kan worden geadviseerd om risico‐ informatie via een evidence map over te brengen. Het ontwerp van de evidence map zou kunnen worden verbeterd door aanpassingen aan te brengen in de vloeiendheid en duidelijkheid van de map. Daarnaast zou het concept van hiërarchische informatieverwerking kunnen worden toegevoegd om tot grafisch ontwerp te komen dat het begrip van complexe risico‐informatie mogelijk kan verhogen en versnellen.
‐ 4 ‐
Summary
People have difficulties comprehending complex scientific risks. However, knowledge about these risks is very important to live safely in today’s society. The evidence map was designed as a tool to comprehend the information about (potential) risks better and faster. This study tested the value of an evidence map next to a narrative text and a combination of text and evidence map for risk comprehension, risk perception, perceived reader friendliness and preference for a specific lay‐out.
The study conducted an experiment by including two types of risks (exposure of electromagnetic fields (EMF) and children’s health disturbances and exposure of EMF and carcinogenicity) as a within subjects factor and three types of information lay‐out (narrative text, evidence map and a combination of text and map) as a between subjects factor. An amount of 118 participants were randomly assigned to one of the six experimental conditions and filled out a questionnaire. Chi‐squares and ANOVA’s indicated that participant’s characteristics (demographic characteristics, level of reading speed, level of prior knowledge and level of prior risk perception) were equally distributed among the conditions and therefore were comparable except for age. ANOVA’s showed that this did not influence the results, except for processing time and the amount of scientific knowledge.
The results indicated that the beneficial effect of the evidence map was limited for risk comprehension for both risks. Participants using the evidence map did not process the information better or faster than participants using the narrative text which was in contradiction with propositions of the Mental Model Theory and the Assimilation Theory. However, a small effect size of processing time indicated that participants using the evidence map could significantly process the information faster when more participants were included. Also the principles of the Active Processing Model could not be confirmed, because participant using a text did not perform better than participants using the map. However, a large majority of the participants significantly preferred evidence maps as tools for communicating controversial risk information. Surprisingly, this was not completely underpinned by the findings regarding perceived reader friendliness. Although participants judged the evidence map as shorter, better in limiting the essential information and containing simpler sentences, the text was judged as clearer and more fluently. Also Börner, Schütz and Wiedemann (2009) found that the evidence map was less fluent and contained simpler sentences. Therefore, these characteristics can be considered as stable for each type of information lay‐
out, because they were found for three types of risks. Although participants judged the information lay‐outs differently, this did not result in significant differences in risk perception between the lay‐outs. This was found for all six aspects of risk perception for both risks.
Although the main objective of the evidence map (enhancing comprehension) was not achieved, people strongly favour the supply of information presented by the evidence map. Risk communicators could be recommended to use evidence maps as tools for communicating about risks. Improvements of the design of the evidence map regarding fluentness and clarity could result in enhancing the level of perceived reader friendliness. Furthermore, adding the concept of hierarchical tree diagrams in combination with the evidence map could be used to create a graphical overview that facilitates the comprehension of controversial scientific information.
‐ 5 ‐
Table of contents
Dutch summary p. 3
Summary p. 4
1. Introduction p. 7
2. Theoretical background p. 11
2.1 Information lay‐out p. 12
2.1.1 Information lay‐out and risk comprehension p. 12
2.1.2 Information lay‐out and risk perception p. 16
2.1.3 Information lay‐out and perceived reader friendliness p. 17
2.1.4 Information lay‐out and preference p. 18
2.2 Defining risk types p. 18
2.2.1 Defining “risk” p. 18
2.2.2 Psychometric paradigm p. 18
2.2.3 Fright Factors p. 19
2.2.4 Evaluating risk differences p. 19
2.3 Confounding variables p. 22
2.4 Research questions p. 24
3. Method p. 26
3.1 Research design p. 26
3.2 Manipulation p. 26
3.3 Participants p. 27
3.3.1 Selection of participants p. 27
3.3.2 Research sample and participants characteristics p. 28
3.4 Instruments p. 33
3.4.1 Developing the questionnaire p. 33
3.4.2 Testing the questionnaire p. 34
3.4.3 Construction of dependent variables p. 35
3.4.4 Construction of confounding variables p. 38
3.5 Procedure p. 40
‐ 6 ‐
4. Results p. 41
4.1 General description of the research sample p. 41
4.2 Risk comprehension p. 44
4.2.1 Objective risk comprehension of the summary p. 44
4.2.2 Subjective risk comprehension of the summary p. 46
4.2.3 Processing time of the summary p. 47
4.3 Risk perception p. 48
4.4 Perceived reader friendliness of information lay‐outs p. 50
4.5 Preference for information lay‐outs p. 55
5. Discussion p. 57
5.1 Restating the research question p. 57
5.2 Summarizing the research results p. 57
5.2.1 Risk comprehension p. 57
5.2.2 Risk perception p. 58
5.2.3 Perceived reader friendliness p. 58
5.2.4 Preference p. 59
5.2.5 Risk differences p. 59
5.3 Interpreting the research results p. 59
5.3.1 Methodological deficits p. 59
5.3.2 Implementation of the evidence map p. 60
5.3.3 Reconsideration of underlying theories p. 62
5.4 Future research p. 64
5.5 Implications p. 64
References p. 67
Appendixes p. 71
Appendix A1 p. 72
Appendix A2 p. 74
Appendix B p. 76
Appendix C p. 101
Appendix D p. 102
Appendix E p. 105
Appendix F p. 107
‐ 7 ‐
1. Introduction
“The public’s right to know”. Almost everybody in today’s world will agree with this statement. People have the right to obtain proper information about (potential) risks and hazards. Knowledge about developments in science and technology are fundamental in today’s world, because they have a great influence on society. So mutual understanding of these developments is important, both for society as for scientists.
Especially for policy makers it is important to obtain proper information in order to make the right decisions.
Policy makers can be described as the persons with power to influence or determine policies and practices at an international, national, regional, or local level (Policy maker, n.d.). Because they are (partially) responsible for designing well‐functional policies that supports society, their policies have to rely on reliable information. Policy makers make use of the most recent scientific information as a foundation to make the most deliberate decisions for society. Also, they can prepare themselves to the use of new innovations, like medicines and electronic devices. It is important to communicate this important information towards the audience (e.g., society). Informing society does not only lead to passive agreement; it also motivates people to participate in decisions about these matters so decisions are made with mutual approval (Donkers & Willems, 2005).
But how can you inform policy makers about scientific evidence about (potential) hazards in a proper way?
Answering this question is a fundamental task in risk assessment. And a difficult one. First of all, scientific evidence is often not clear. It contains studies that report inconsistent data or even contradictory results. So making a proper summary of controversial evidence is a real challenge. Furthermore, communicating such results to a heterogeneous audience of different stakeholders is a difficult task. What does a message have to look like in order to be understood by its audience? Because of the complexity of the information, people are not always motivated to put effort in understanding the message.
In order to overcome these difficulties, Wiedemann and Schütz (2008) designed the so called “evidence map”.
“Evidence maps are an approach to evidence characterization that aims improving the transparency and reasonableness of reporting scientific evidence regarding the existence of the hazard or a risk” (p. 151). The map shows experts’ arguments for drawing conclusions about (potential) risks and hazards by giving a graphical representation of arguments that speaks for or against the existence of a causal relationship between the risk and its effects. In this graphical overview, the remaining uncertainties about this relation are included as well, because scientific information is often controversial. The aim of the map is to summarize the most important scientific information about a (scientific) topic in such a way that it can be easily understood.
The theoretical basis of the evidence map can be described as two‐fold according to Wiedemann and Schütz (2008). First, the principles of Toulmin’s Argumentation Theory are used to design the evidence map. The theory indentifies four main components that are important in argumentation within a message, namely data, warrant, backing and claim. With regard to the evidence map, data refers to the scientific evidence provided by the
‐ 8 ‐
scientific studies on the respective topic. The claim refers to the conclusion whether a causal relationship exists between exposure to a risk and the adverse health effect. The warrant consists of “pro” and “con” arguments which link data to the claim, with their respective supporting and attenuating arguments (the backing). Second, the weight‐of‐evidence (WOE) approach was used to rate the value of the “pro” and “con” arguments used. The evaluation of the weight of evidence can only be adequately executed when all scientific evidence should be included in a risk assessment according to its methodological rigor. Then, positive and negative evidence should be weighted against each other. Figure 1.1 shows the template for an evidence map.
Figure 1.1. Template of an Evidence Map (Wiedemann & Schütz, 2008).
To illustrate the construction of the evidence map, Wiedemann and Schütz (2008) outline the development of the evidence map about cancer epidemiology. Out of 122 studies, two experts selected 13 scientific studies as acceptable for risk evaluation based on the requirements of Wiedemann and Schütz. Figure 1.2 shows the evidence map for cancer epidemiology. The selected studies found both pro and con arguments for the existence of a causal relationship between exposure to EMF and carcinogenicity. When the methodology used in the studies that found the evidence was strong (e.g., large amount of participants, proper determination exposure to radiation), the pro and con arguments are supported. However, the quality of the evidence was attenuated when the studies contained methodological deficits (e.g., inadequate determination of exposure to radiation, too short exposure durations and too small sample sizes). As shown in figure 1.2, both the pro and con argument were attenuated by methodological deficits. In the conclusion, the pro and con arguments are weighted against each other according the methodological rigor. Based on this evaluation, a conclusion about the relationship between exposure to EMF and carcinogenicity is formulated. Figure 1.2 illustrates that scientific risk information is complex and therefore no hard conclusions can be drawn. Because of the controversial nature of scientific risk information, the conclusion is supplemented with an overview of uncertainties that still remain.
Evidence Basis:
•
<Number of studies>Conclusion:
< ... >
Remaining Uncertainties
< ... >
Contra‐Argument:
< ... >
Attenuating < >
Supporting
< ...>
Pro‐Argument:
< ... >
Supporting < ...>
Attenuating
< .. >
‐ 9 ‐
Figure 1.2. Evidence Map for Cancer Epidemiology (Wiedemann & Schütz, 2008).
Although the theoretical basis of the evidence map is underpinned by the Argumentation Theory and the WOE‐
approach, only few studies have tested its effects. So it is still uncertain whether the map succeeds in enhancing people’s understanding of a certain risk. Therefore, various questions arise. Do people really prefer this type of information presentation over other information lay‐outs? Does it really enhance the amount of objective comprehension and what are the effects on risk perception? Could the effects be generalized to more types of risks? Or could these risks and people be identified by certain characteristics? These questions can be summarized into the following research question:
“What is the value of the evidence map for risk comprehension, risk perception, perceived reader friendliness and preference for communicating controversial information about electromagnetic fields and their effects on health disturbances and cancer?”
This study tries to find an adequate way of communicating about controversial risk information by investigating the effects of the evidence map for risk comprehension, risk perception, perceived reader friendliness and preference. By doing so, society not only receives important scientific risk information, but also understands this information. This might enable people to make better decisions. In order to adequately investigate the research question, the second chapter will evaluate the idea of the evidence map as a tool for risk communication based on a review of scientific literature about the effects of information lay‐out for risk comprehension, risk perception, perceived reader friendliness and preference. The third chapter will describe the method used by outlining the research design and the instruments and procedure used. The results will be described in the fourth chapter by outlining the effects of the evidence map for risk comprehension, risk perception, perceived reader friendliness and preference. An interpretation of these results by critically outlining eventual limitations of the study and by
Evidence Basis:
* 122 studies
* 13 selected
Conclusion:
* Vague initial suspicion (two experts)
* On this basis of evidence, an evaluation is not possible (third expert).
Remaining Uncertainties
Studies are mostly exploratory and single out individual results Contra‐Argument:
Eight studies find no statistically significant relationship between exposure to mobile phone communication fields and cancer.
Attenuating
Seven of these studies are only
conditionally meaningful since they posses inadequate exposure determinations, too short exposure durations, too small sample sizes or other methodological deficits.
Pro‐Argument:
In five studies, noticeable results are found that point to a relationship between exposure to mobile phone communication field and cancer.
Attenuating
* All of these five studies demonstrate methodological deficits, above all with the determination of exposure. They are therefore only conditionally meaningful.
* Theoretical considerations are missing in the selection of the tumors.
‐ 10 ‐
comparing the findings with other research results will be given in the fifth chapter, as well as the practical implications of the evidence map for the field of risk communication.
‐ 11 ‐
2. Theoretical background
Imagine the world of a policy maker. All day you receive controversial information about potential dangers and you have to make up your mind about how to deal with those risks in order to protect society from all kind of dangers. In order to decide what is best for society, you have to read a large amount of controversial information from different sources in order to make a coherent document that underpins the decisions you take. As a policy maker, how do you want the needed information to be presented? Various things seem to be important.
First of all, a policy maker has to understand the information in order to make a proper indication of a potential risk in terms of seriousness and dreadfulness. This means that information lay‐out could affect risk comprehension, but also risk perception. Secondly, policy makers do not only have to read a large amount of information, they also have to read it quickly in order to satisfy the deadlines of decisions. Therefore, the relevant information should be organized in a proper way in order to make quickly use of it. It is appreciated if one can see the main ideas of the information at first glance in order to save time. This is connected with certain characteristics an information lay‐out consists of. A lay‐out could be seen as more reader friendly because of these characteristics. Additionally, this could result in a certain preference for a specific lay‐out when processing risk information.
In this study, the effects of information will be specifically investigated for controversial scientific risk information (the effects of EMF for cancer and general health disturbances). This information is considered to be controversial, because it consists of a multitude of inconsistent or even contradictory results of scientific studies (Wiedemann &
Schütz, 2008). The information consists of different views (the “pro” and “con” arguments) on the same issue and the reader has to judge this information to formulate a final conclusion (Britt et al. 1999, Rouet et al. 1996; cited in Kobayashi, 2009). Because the controversial content of the information, the study also focuses on the reader’s judgments about the perceived reader friendliness of the information lay‐outs used. Which lay‐out is considered to be more reader friendly and which characteristics support this decision? Furthermore, the readers’ preference for a specific type of information lay‐out while processing complex information will be investigated.
In order to evaluate the effects of information lay‐out on risk comprehension, risk perception, perceived reader friendliness and preference, the possible value of the evidence map with regard to these variables has to be scientifically founded first. In paragraph 2.1, relevant scientific literature is evaluated in order to see whether an evidence map is beneficial in understanding complex texts. Results from scientific research will be summarized and discussed. Then, the generalizability of these results towards different types of risks will be evaluated. An overview of risk characteristics will be given in order to decide whether the effects of lay‐out could be generalized towards various types of risks (paragraph 2.2). To exclude the influence of chance when interpreting the results, variables that could possibly influence the results need to be detected (paragraph 2.3). Based on this theoretical background, sub questions towards the results of this research can be formulated. These will be outlined in paragraph 2.4.
‐ 12 ‐
2.1 Information lay‐out
Wiedemann and Schütz (2008) thought that a schematic presentation of information – the evidence map – would result in better understanding information that contains controversial information. The question of how to present information in a proper way is asked many times. In this paragraph, an overview of relevant literature is presented in order to scientifically found the idea of using an evidence map in order to enhance comprehensibility (paragraph 2.1.1). Furthermore, the effects of information lay‐out on risk perception will be evaluated. Does research indicate that the perception of a risk could be influenced by means of how information is presented (paragraph 2.1.2)? Thirdly, the idea that people hold certain stable characteristics against specific lay‐outs will be found in paragraph 2.1.3. Finally, scientific literature will be used to evaluate the effects of information lay‐out on the preferences people have for a specific lay‐out in order to found the idea that people have certain preferences for the way they receive information (paragraph 2.1.4).
2.1.1 Information lay‐out and risk comprehension
The idea of the evidence map as a tool of enhancing people’s understanding of a hazard or risk is based on scientific insights in the cognitive psychology. To comprehend a text, the text information should become part of the reader’s personal knowledge base (Kintsch, 1988, 1994; cited in Kools, van der Wiel & Ruiter 2005). While reading a text, readers need to cognitively “encode” the incoming information in order to create a coherent representation of the texts’ content that stimulates understanding. So new incoming information is constantly connected to information that was already been taught (Kools et al., 2005).
Research shows that graphical overviews can have an additive value in enhancing text comprehension. (Chang, Sung & Chen, 2002; Kools, van de Wiel, Ruiter, Crüts & Kok., 2006; Oliver, 2009; Salmerón, Baccino, Canas, Madrid
& Fajardo, 2009). The effects are especially strong on the macro level of coherence of a text (Kools et al., 2006), when information is more complex and controversial (Kobayashi, 2007) and when the prior knowledge base on the text topic is low (Lorch & Lorch, 1996; Mayer, 1979; cited in Salmerón et al., 2009).
In this study, the focus is mainly on the global structure of information presentation. Policy makers are often required to read controversial texts, because scientific information consists of inconsistent and contradictory scientific results (Wiedemann & Schütz, 2008). It is important for them to identify the quality of the pro‐ and contra‐arguments. Therefore, understanding of intertextual relations is crucial in order to make a judgment about the controversial issue according to Kobayashi (2009). This is what Kools et al. (2006) called the “macro” (e.g., the
“global”) level of coherence of information presentation. Coherence at the macro level implies that the order of topics is logical as opposed to random and that the smaller and larger sections of a text are clearly related to each other and the overall topic. It enables the reader to maintain a coherent picture of the text and therefore stimulating the understanding of a text. Based on this information, graphical overviews seemed to be beneficial in comprehending information.
‐ 13 ‐
Both the Mental Model Theory (Johnson‐Laird, 1980) and the Assimilation Theory (Mayer, 1979) underpin the idea of enhancing comprehension by graphical overviews. Graphical organizers increase the salience of the (text)structure, thereby increasing the comprehensibility of the information presented. In addition, the Active Processing Model (Shapiro, 2005) predicts the opposite effect. Because that graphic organizers map the intertextual relations between the sections of the information presented, it lessens the comprehensibility of information. All theories will be explained below.
Mental Model Theory
The Mental Model Theory is one of the theories that stated that specific information lay‐outs could influence the levels of text comprehension while reading a text (Connelly & Knuth, 1998). The theory stated that comprehension of a text appears to result in multiple representations (Carpenter & Just, 1987; Van Dijk & Kintsch, 1983; cited in Glenberg & Langston, 1992). For example, one may be a representation of the text itself (a representation of the words and sentences); another may be a representation of what the text is about. According to Glenberg and Langston, mental models are representations of situations described by a text, rather than representations of the text itself. To illustrate this definition of a mental model, read the following sentences about the spatial relations between four common objects (Johnson‐Laird, 1980):
The knife is in front of the spoon The spoon is on the left of the glass The glass is behind the dish.
A drawing of the corresponding lay‐out using the names of the objects results from reading these sentences.
People make an imagination of the outlined situation, which can be seen as the process of creating a mental model of the situation. The model forms the basis for their further thinking and reasoning. It influences the way we see the objects around us and how we deal with those objects.
Mental models are considered as propositional representations. A proposition (“a description”) can be based on a text. This proposition could be true or false, ultimately with respect to the worldview. However, this worldview is only an internal representation of the world; people cannot apprehend the world directly. So a propositional representation is true or false with respect to a mental model of the world (Johnson‐Laird, 1980, p. 98).
With regard to the Mental Model Theory, pictures help to build mental models of what the text is about (Glenberg
& Langston, 1992). Because mental models could be both verbal (propositional) and visual, people have to combine both in their working memory to foster comprehension (Kools et al., 2006). This means that people need more cognitive resources when they have to read a text alone than when they see a picture. The rationale behind this statement is that a text has to be mentally transformed into a propositional representation and then into a mental model of the situation. In addition, pictures are considered as external models. They directly allow the construction of a mental model of the situation.
‐ 14 ‐
There is much support for this version of the Mental Model Theory. Studies of Zikmund‐Fisher et al. (2008) and Kools et al.(2005) showed that illustrations have an additive value in responding towards and understanding a message. Also Glenberg and Langston (1992) investigated whether pictures helped people to comprehend and remember texts. In their experiment, 48 students had to read texts describing four‐step procedures in which the middle steps were described as occurring at the same time, but the verbal description of the text was sequential.
Students were assigned to a no‐picture or a with‐picture condition. Subjects in the no‐picture condition had to respond primarily on the basis of a representation of the text. The with‐picture condition represented the procedure using the same text, but with a respective picture that represented a structure of a mental model for this text. Their findings indicate that pictures facilitate comprehension and memory for texts. Pictures can also affect the representation of the information. Without pictures, subjects organize the information much like a text;
subjects did not observe that the middle steps happened at the same time, but thought that they were taking place next to each other. In addition, subjects in the with‐picture condition recognized the middle steps as occurring at the same moment, because the representational elements were close in the mental model. So when texts were presented alone, subjects tended to mentally represent the text. When texts were accompanied with appropriate pictures, subjects tended to mentally represent the procedure.
Assimilation Theory
Whereas the Mental Model Theory (Johnson‐Laird, 1980) solely focuses on the beneficial effects of pictures in comprehending information, the Assimilation Theory (Mayer, 1979) focuses more on the concept of graphic organizers, because research indicates that other forms than pictures could enhance comprehension levels as well. In some cases these displays work even better than pictures.
According to Chang, Chen & Sung (2002), there are three types of graphical overviews that facilitates learning:
graphic organizers (Barron & Schwarz, 1984; Griffin et al, 1995; Katayama & Robinson, 2000; cited in Chang et al., 2002), knowledge maps (Chmielewski & Dansereau, 1998; Dansereau & Newbern, 1997; cited in Chang et al., 2002) and concept maps (Novak, 1990; Novak & Gowin, 1984; cited in Chang et al., 2002). Despite the differences in visual presentation, the three strategies are very similar, because they rely on the same underlying concepts. In this research, we focus solely on the effect of graphic organizers, because the evidence map is most similar to this type of graphical overview.
The history of graphic organizers is rooted in Ausabel’s advanced organizers (Ausabel, 1968; cited in Culbert, Windler & Work, 1998). He describes the main purpose of the organizers as bridging “the gap between what learners already know and what they have to learn at any given moment in their educational careers” (p. 336).
This purpose forms the input of the main idea of a graphic organizer nowadays, namely that it has to facilitate the understanding of a text by explicitly highlighting the main ideas and by showing the relationships between the main ideas and the supporting details. Therefore, its purpose is to make texts more accessible and comprehensible for the reader (Kools et al., 2006).
‐ 15 ‐
In this respect, the evidence map could be seen as a sort of graphic organizer. An evidence map could foster comprehension, because it represents the information in a logical way so that subjects are able to better understand the relationships of concepts in a text. Especially because the evidence map contains controversial information, the effects for enhancing comprehensibility could be strong (Kobayashi, 2009). Furthermore, the idea of the evidence map as a tool to enhance comprehension is scientifically founded by the ideas of the Assimilation Theory.
The Assimilation Theory of Mayer (1979) states that graphical overviews enables readers to construct an accurate mental model of the text, as reflected in the overviews, and thus provide an organizational framework, prior to reading, that affects the reading process. Thereby the salience of the text structure increases, which fosters the level of text comprehension (Salmeron et al., 2009). The Assimilation Theory predicts that graphic organizers will have a positive effect on learning when the two functions of organizers (to make available and activate meaningful learning sets) are not present. This indicates that the material must be unfamiliar to the learner and it has to be cohesive and structured (e.g., not a text listing the characteristics of imaginary countries). The graphical organizer has to create a meaningful context and encourage the learner to integrate the information. When these conditions are fulfilled, the theory predict that graphical organizers have a stronger positive effect for poorly organized text than well organized text, that graphical organizers have a stronger effect for learners lacking prerequisite knowledge, that graphical organizers have a stronger effect for learners lacking prerequisite abilities and that graphical organizers have rather an especially strong effect on measures of transfer than on retention (Mayer, 1979). However, there are alternatives of the Assimilation Theory that claim the opposite effects of graphical organizers, like the Active Processing Model.
Active Processing Model
Also the Active Processing Model (Hofman & van Oostendorp, 1999; cited in Shapiro, 1998) focuses on the effects of graphical organizers, like the Assimilation Theory. Only this theory predicts the exact opposite effect. Instead of enhancing comprehension, structured overviews may inhibit the use of comprehension strategies by readers (Salmeron et al., 2009). Despite the demonstrated benefits of text cohesion, comprehension does not reside in the text; it emerges in the mind of the reader (McNamara & Shapiro, 2005). The reader uses knowledge of words, syntax, context and the topic to interpret and integrate the text. Because the reader constructs the mental representations of the text by using their own ability and intentions, it requires a deeper level of processing the information. The reader has to put more thoughts to discover the relations between sections.
McNamara et al. (1996; cited in McNamara & Shapiro, 2005) reasoned that the level of prior knowledge of readers would interact with the effect of cohesion. Readers with more knowledge about the topic are able to form a more coherent situation (mental) model of the text, whereas readers with a low amount of prior knowledge cannot create a coherent model of the text. Research among middle‐school students showed that low‐knowledge readers cannot easily fill in gaps in low‐cohesion text, because they do no have the knowledge to generate the necessary interferences. In addition, high knowledge readers benefited from low‐cohesion text, because they produce a
‐ 16 ‐
deeper level of understanding a text. This leads to the proposition of the active processing model: Low‐knowledge readers benefit more from an easier, cohesive text, whereas high‐knowledge readers should be allowed to make their own inferences with more challenging, less cohesive text.
Based on this literature review, it becomes clear that the idea of adding graphics to a text in order to foster comprehension is not new. Evidence maps could be seen as a type of graphical organizers. Based on the Mental Model Theory and the Assimilation Theory, by summarizing the complex scientific evidence and schematizing this information into a logical structure, it is expected that it will enhance people’s comprehensibility of a risk compared to the level of comprehensibility when the information was presented by a text alone. In contrast, the Active Processing Model takes the opposite position and states that graphical overviews inhibit a deeper understanding of risk information. It is unclear whether hierarchical overviews enable the reader to construct a more accurate mental representation of the text information or that it inhibits a deeper processing of the information. Therefore, the following question can be formulated in order to answer a part of the research question:
Q1: “What are the effects of information lay‐out (narrative text, evidence map or combination of text and evidence map) on the comprehensibility of a risk?”
2.1.2 Information lay‐out and risk perception
This research solely focuses on the presentation of complex risk information towards its target groups. Therefore, it is important to check whether presentation lay‐out of risk information influence the perception of a risk. Various studies into the field of risk communication showed that graphics of risk information could influence the perception of a risk. Ter Huurne (2008) states that there are various ways of influencing people’s risk perception by choosing a presentation format, but the results are not clear. She identifies various strategies to influence people’s risk perception. For example, making comparisons towards other risks, using numbers instead of a narrative description to indicate the magnitude of a risk or using graphical presentations of risks, e.g., risk ladders (Sandman, Weinstein & Miller, 1994).
This study focuses on the last strategy of influencing people’s risk perception, but specifically of the effect of graphical overviews (Ausabel, 1968; cited in Culbert et al., 1998). Since there is limited research towards the effects of graphical overview for risk perception, researches of the effects for risk ladders and pictograms were used to illustrate that graphics could possibly influence risk perception. Both direct and indirect effects were found.
Experimental research of Sandman et al. (1994) shows that the level of perceived threat varies with the location of the subject’s assigned level on the risk ladder. Variations in test magnitude had no effect on risk perception, instead, the location on the ladder matters. Participants perceived a greater danger when the risk was assigned three‐quarters of the way up the ladder than when the risk was assigned one quarter of the way up the ladder. In
‐ 17 ‐
this research, the focus is on the effects of the graphical presentation of risks. There is not much research done to the effects of graphic organizers on risk perception.
Zikmund‐Fisher et al. (2008) experimentally tested whether pictograms would influence perceptions and comprehension of side effect risks in an online decision aid with regard to breast cancer. They found a low level of knowledge of side effects when they were displayed by numeric text only. Adding pictograms, however, produced significantly higher comprehension levels. They concluded that pictographs make risk statistics easier to interpret, moreover because they reduce biases associated with incremental risk presentations. However, they did not find a direct relationship between information lay‐out and risk perception.
Schapira, Nattinger and McAuliffe (2006) stated that risk perceptions varied with the graphic format used.
Identical numeric risks were perceived to be of greater danger when presented with a pictorial display compared with a bar graph. The researchers confirmed the additive value of graphic displays by stating that the specificity of the provided information may increase by using numeric terms and graphical displays. However, they also introduce lay‐out effects on risk perceptions. They concluded with the statement that the chosen lay‐out of the presented information needs to be controlled.
Although the effects of graphic organizer for risk perception are not yet investigated, the idea that presentation format influences risk perception seems valid. Based on studies towards risk ladders (Sandman et al., 1994) and pictograms (Schapira et al., 2006 and Zikmund‐Fischer et al., 2008) different effects for risk perception were found. Presentation format could directly and indirectly influence the perception of a risk. To investigate the effect of a graphic organizer on risk perception, the following question will be investigated:
Q2: “What are the effects of information lay‐out (narrative text, evidence map or combination of text and evidence map) on the perception of a risk?”
2.1.3 Information lay‐out and perceived reader friendliness
A one‐factor experimental study of Börner, Schütz and Wiedemann (2009) indicated that students judged the level of perceived reader friendliness differently for a narrative text and an evidence map about the explanation of toxicogenomics. The results showed that the evidence map was evaluated in all (but one) cases more favourably than the narrative text, however, the differences in the evaluation were not particularly strong. This indicates there is a relationship between information lay‐out and the level of perceived reader friendliness of this lay‐out.
This study will investigate whether this effect of information lay‐out is present for a narrative text, an evidence map and a combination of the text and the map. This can be summarized in the following question:
Q3: “What are the effects of information lay‐out (narrative text, evidence map or combination of text and evidence map) on the level of perceived reader friendliness of the information?”