Be your best self : the effect of positive feedback on cognitive performance

(1)

Be Your Best Self: The Effect Of Positive

Feedback On Cognitive Performance

Madeline Mol

Thesis

(2)

1

Abstract

The literature concerning feedback is highly mixed. In the present paper, two studies tried to replicate and build on a study by Kazén and Kuhl (2005), which suggests that the experience of positive affect, brought about by the reception of positive feedback, can enable one to complete the Stroop task without showing the classic Stroop effect. The first study was a slightly changed replication of this original study. Due to a programming error, it was impossible to draw any intelligible conclusions based on the data. The second study was another replication attempt without this programming error. This attempt failed as well. Possible explanations for this failure to replicate are discussed.

(3)

2

Introduction

The literature concerning feedback is highly mixed. While some studies claim that positive feedback can have a motivating effect, (Ilies & Judge, 2005), other studies suggest that positive feedback does not always enhance performance (Webster & Martocchio, 1992; Deci, Koestner & Ryan, 1999; Podsakoff & Farh, 1989; Kohn, 1993). Furthermore, the motivational effects of feedback can differ temporally (long- versus short-term effects; Burgers, Eden, Van Engelenburg, & Bunungh, 2015), and can be dependent on the quality of the feedback (Carpentier & Margeau, 2013).

Several theoretical approaches have tried to clarify when and why the experience of positive affect, caused by receiving positive feedback (Ilies & Judge, 2005), enhances performance. Two of these, the approach considering motivational intensity of positive affective states, and Personality Systems Interaction (PSI) theory, will be explicated below.

The first account, which considers the motivational intensity of positive affective states (Gable & Harmon-Jones, 2011), states that positive affect can be low (i.e. feeling serene) or high (i.e. feeling enthusiastic) in approach motivation. Low approach motivation often occurs after a goal has been achieved (post-goal), while high approach motivation is experienced in pursuit of a goal (pre-goal). Pre-goal positive affect has been found to be particularly motivating: participants performed better on a task after a pre-goal gain cue compared to a pre-goal neutral cue (Gable & Harmon-Jones, 2011). In line with this, an applied study showed the particularly non-enhancing effect of post-goal positive affect (which is what positive feedback ultimately establishes) on performance: basketball players who had won a game (and thus received positive feedback) were more likely to lose the next game (Mizruchi, 1991).

(4)

3

But why does positive affect enhance performance pre-goal, in particular? According to Carver (2003), when a person is experiencing positive affect while striving towards an attainable goal, attention is narrowed in order to shut out all irrelevant cognitions and perceptions. This improves goal attainment and thus performance. In contrast, experiencing a positive affective state post-goal suggests that things are going better than necessary and one can reduce effort and broaden attention, and in this way positive feedback has a dampening effect on performance. Indeed, the experience of pre-goal positive affect does lead people to attend more to local targets, indicating more narrow attention, while experiencing post-goal positive affect leads to more attention to global targets, indicating broader attention (Gable & Harmon-Jones, 2011).

However, a series of studies by Kazén and Kuhl (1999, 2005) contradict this idea that positive feedback dampens performance. When receiving positive feedback, participants showed less of a Stroop effect in a four-colour Stroop task after receiving positive feedback compared to negative or neutral feedback. In the Stroop task (Stroop, 1935) participants have to respond to the colour in which a colour word or a row of XXXX’s is displayed. The Stroop effect is the effect that occurs when the colour and content of the word are incongruent: people make more errors and are slower when making a correct response compared to when categorizing the colour of a row of XXXX’s. Reaction times on the control trials are subtracted from the reaction times on the incongruent trials. The higher the resulting number, the bigger the Stroop interference effect.

In their 1999 and 2005 papers, Kazén and Kuhl had participants complete two tasks and found that they would perform better on the first of these tasks in response to positive feedback. In their Study 4 (Kazén & Kuhl, 2005) however, they employed a

(5)

4

single task paradigm. Participants completed 96 experimental trials of the Stroop task. Each trial started with a fixation cross, followed by a prime which counted as positive, neutral, or negative. Participants were told that this prime was not only feedback on their performance on the previous trial, but also a preparation signal that the next trial was coming up. The Stroop effect virtually disappeared on trials where participants received positive feedback compared to when the feedback was negative or neutral. However, it should be noted that the study was a little underpowered: N= 25, F(2, 46) = 3.25, and partial η2_{= .12.}

The studies discussed previously (Mizruchi, 1991) have shown that positive feedback can dampen performance, because it is by default a post-goal phenomenon (one cannot receive feedback on one’s performance on a certain task before one has performed that task) and post-goal positive feedback dampens performance. However, performance in the studies by Kazén and Kuhl (Kazén & Kuhl, 2005; Kuhl & Kazén, 1999) was actually enhanced by giving positive feedback.

Their theory is that people form intentions in anticipation of performing a particular task, which helps them complete this task. This is at the core of their Personality Systems Interaction (PSI) theory (Kuhl & Kazén, 1999). People’s formed intentions are defined as action plans (plans about which actions to take to obtain a certain outcome), which are held in an active state and are stored in intention memory. Intention memory is somewhat similar to working memory. The intentions (or action plans) stored in intention memory are inhibited to prevent their premature enactment. The experience of positive affect signals that the time has come to act upon one’s intentions. This causes the inhibitions to be released and allows a person to enact the previously inhibited intentions.

(6)

5

One of the possible conditions under which an intention is stored in intention memory is that an intention needs to consist of multiple consecutive action steps (or tasks). When an intention consisting of multiple consecutive action steps is stored in intention memory and positive affect is experienced, one starts sequencing through these multiple action steps. This makes maintenance of an abstract representation of the intention in intention memory useful, since it is necessary to control the correct output sequence of each step. This is what participants did in several of the Kazén and Kuhl studies (1999; 2005, Study 1 – 3). However, since monitoring of sequencing through just one step is not necessary, this is not the case with intentions that consist of only one step (Kuhl & Kazén, 1999).

A second way to ensure that people load their intention memory is through activating achievement motivation. Since maintaining difficult intentions active in memory is a crucial aspect of achievement motivation, constantly reminding them of the achievement character of the task makes activation of their intention memory (‘loading intention memory’, or maintaining difficult intentions in memory, easier). This is how participants were made to load their intention memory in Study 4 (Kazén & Kuhl, 2005).

The results of Kazén and Kuhl’s Study 4 (Kazén & Kuhl, 2005) do not fit with the findings by Mizruchi (1991) in which positive affect that is experienced post-goal actually debilitates performance. Kazén and Kuhl argue that showing feedback about the participant’s current performance on the task at the start of every trial should create an achievement-related context for the participant by reminding them of how well they are performing, hence activating intention memory. Seeing the positive feedback signs should therefore not only create an achievement-related context and thus cause participants to load their intention memory, but it should also function as a positive

(7)

6

affective prime, hence causing participants to act upon the difficult intentions stored in their intention memory. In this case, the difficult intentions that had to be kept active in memory were the intention to respond to the colour of the word, instead of to its meaning. Positive feedback on the previous trial should bring about positive affect and therefore signal that the time has come to act upon these intentions. This should lead to enhanced performance on the present trial and thus function as enhancing pre-goal-, rather than debilitating post-goal positive affect.

The aim of the present research was to disentangle positive affect and intention memory activation. Since in Kazén and Kuhl’s Study 4 (2005) the prime that was supposed to cause participants to load their intention memory was the same as the prime that was supposed to cause them to act upon it, it is impossible to say whether it is true that participants will perform better on a Stroop task after receiving positive feedback only if they have loaded their intention memory. It could as well be the case that just seeing positive feedback facilitates performance, since there is no way to know that participants actually loaded their intention memory in response to the achievement-motivation/feedback prime. In the current study, we are aiming to see if positive feedback, and thus positive affect, can indeed cause a decrease in the Stroop interference effect only if intention memory is loaded.

In order to disentangle positive affect from intention memory and show that it only works if the order is right, we intend to replicate the findings by Kazén and Kuhl (Study 4, 2005) in the first study (Study 1), using a slightly modified design which will be elaborated upon later, so that we can build upon the expected effect in the second study (Study 2). We expect that positive feedback will enhance performance compared to neutral or negative feedback. If we will not find the expected effect that positive

(8)

7

feedback will enhance performance compared to neutral or negative feedback, it does not make sense to disentangle this unfound effect any further in Study 2. The design of Study 2 will therefore depend on the results of the first. If our replication attempt in Study 1 is successful, we will aim to examine whether positive feedback does indeed enhance performance only if it is experienced after intention memory has been loaded in Study 2. We expect performance to be enhanced when receiving positive feedback follows the loading of intention memory compared to when positive feedback precedes it. However, if our replication attempt turns out to be unsuccessful, we will try to replicate the findings by Kazén and Kuhl (2005) again, this time sticking to their original method. Both studies were preregistered.

(9)

8

Study 1 Method

Participants:

Forty-eight Dutch-speaking participants, who were, or had recently been, students, and who had a mean age of 21.5 (SD = 2.7) participated in the present study. They either signed up themselves through the University of Amsterdam’s online system (DPMS), or they were approached on the university premises. Two of them indicated that they were ambidextrous, while the remaining 46 participants were all right handed. Forty-seven of them had normal or corrected-to-normal (colour) vision, while one of them indicated that they normally wore glasses when working on the computer but did not have these with them. Including them did not influence the results. Participants who were left-handed, did not have normal colour vision, were not students at an institution of higher education, and whose mother tongue was not Dutch were excluded from participation. All participants received either money or research credit.

Materials

Except for the informed consent, all materials were presented to the participant through a computer. For programming of the task and all the questionnaires, InQuisit 4 was used.

Stroop task

A four-color version of the Stroop task was used. The colours included were green, yellow, blue and red. The InQuisit colour codes were (0, 149, 0), (255, 255, 064), (0, 0,

(10)

9

255), and (255, 0, 0), respectively. The background colour was black, which corresponded to InQuisit colour code (0, 0, 0). Participants responded to these colours with the S, F, H, and K keys on a QWERTY-keyboard. A sticker in a corresponding colour was stuck on the key as a reminder. Four different colour-key combinations were used.

Demographic questionnaire

A demographic questionnaire asking about participants’ age, gender and education was administered. This and all other questionnaires are included in the Appendix.

Post-experimental Questionnaire

A post-experimental questionnaire was administered to check whether people were suspicious about the feedback, were paying attention, and understood the instructions correctly. They were also asked about their motivation to perform well, how hard they found the task, what effect they thought the feedback had on their performance and how accurate they thought the feedback was.

Procedure

Study 1 was a replication of Kazén & Kuhl (2005, Study 4) with some modifications that will be explicated where it is applicable. After the participant entered the laboratory, the experimenter explained that the task should be self-explanatory but to come find them nonetheless if anything was unclear, and asked the participant to fill out the informed consent. The experimenter then left the room. After giving informed

(11)

10

consent, the participants completed a demographic questionnaire and received instructions, in which they were told that the goal of the study was to see whether positive or negative feedback would improve their performance (see Appendix). Then the task started. Trials started with a fixation cross, followed by a preparation signal which supposedly was also feedback on their performance on the previous trial. This signal could be positively (+++), neutrally ( ), or negatively (---) valenced. Then they performed a four-colour Stroop trial. Afterwards they completed a questionnaire to check for suspicion of the study’s purpose, credibility of the feedback, and attentiveness. They were then debriefed, thanked and rewarded.

Participants were supposed to complete a total of 134 trials, consisting of 2 warming-up trials, 12 practice trials in which they only received neutral feedback and 120 experimental trials (instead of the original 96). After the practice trials the instructions were shown again and the experimental phase was announced. Stroop trials were either incongruent (the content of the word did not match its colour) or control (a row of XXXX’s in either colour). The fixation cross appeared for 500 milliseconds, the feedback signal for 1500 milliseconds and the Stroop trial for 2000 milliseconds or until the participant made a response (see Figure 1).

(12)

11

Figure 1. Schematic representation of the procedure of Study 1.

The following alterations were introduced into Kazén and Kuhl’s (2005) original design. While the authors of the original study presented participants with

pre-programmed, random feedback which was not contingent on their performance at all, we decided to have too slow or incorrect responses always being followed by negative feedback in order to make sure the rest of the pre-programmed, randomly occurring feedback was as credible as possible. Because we want to see the effect of positive feedback on the latencies of participants’ correct responses, and thus will remove the latencies of incorrect responses from the data prior to analysis, more trials were run

(13)

12

than in the original study to maintain power. Secondly, while in the original study participants had all the time in the world to respond, we introduced a response window of 2 seconds to remind participants that both speed and accuracy were key. Thirdly, while the original authors required participants to fill out several questionnaires before commencing the study, we did not. Fourthly, we did not assign response keys to each participant individually, while the original authors did. Instead we had four different combinations of colours and response keys, used coloured stickers on the keys, and assigned altogether different response keys than in the original study. These changes were all discussed with Dr. Miguel Kazén. He did not foresee any problems in replicating the effect despite these changes.

Results

Trial distribution

The current study employed a 3 (feedbacktype: positive, neutral, negative) x 2 (Stroop stimulus: control, incongruent) within-subjects repeated measures design. This created six conditions. Each participant was therefore supposed to see 20 trials of each feedbacktype and Stroop stimulus combination during the experiment. Furthermore, participants were supposed to see 15 experimental and 15 control trials of each colour.

(14)

13

Participants and Data Preparation

The three participants who gave incorrect answers to questions about the meaning of the feedback were assumed to have had misunderstood the instructions and thus were removed from the data. This left us with data of 45 participants. Since there were so many participants who satisfied at least one of the other two exclusion criteria, namely non-attentiveness (giving a wrong answer to the attentiveness question, which 21 participants did) and suspicion of the feedback (saying that the feedback was fake or random, which 11 participants did), these were taken into account as exploratory exclusion criteria. We feared that the power of our study would suffer greatly (and possibly unnecessarily so) from removing them from all analyses.

Similar to Kazén and Kuhl’s (2005) data preparation method, latencies of all erroneous responses (8.5% of all responses), too fast (<200 ms), or too slow (>1500 ms) responses were removed from the data prior to analysis (0.3% of all responses). The response times were log transformed. In this paper, only untransformed means (in milliseconds) are reported for clarity reasons. Then, mean response times for each feedbacktype and Stroop stimulus combination were calculated. Alpha was set at .05 for all analyses. Means and standard deviations of response times and error rates are summarized in Table 1. All analyses on response times include log-transformed response times.

(15)

14

Table 1. Means (and Standard Deviations) of the Reaction Times (in Milliseconds),

Error Rates, Mean Number of Trials Seen per Participant and Range of Trials Seen per Participant for the Six Different Feedbacktype and Stroop Stimulus Combinations

Control Stroop trials Incongruent Stroop trials positive neutral negative positive neutral negative RT 606 (105) 596 (97) 607 (102) 611 (102) 617 (96) 621 (93)

error rates .90 (.10) .94 (.10) .91 (.08) .93 (.06) .91 (.07) .92 (.06)

mean n trials 10.11 10.07 12.98 29.80 30.89 38

range trials 7 – 14 5 – 15 8 – 21 26 – 33 26 – 36 27 - 59

Note: N = 45 for every cell.

In Table 1, one can immediately see that the trials were not equally distributed over the six different conditions. Also, the mean amount of experimental trials completed (131.8) was more than the total amount of trials each participant should complete. After inspection of the data and InQuisit scripts a programming error was discovered that had caused this imbalance in the trial distribution. Firstly, instead of showing participants an equal amount of control and experimental trials, participants saw approximately three times as many experimental trials as control trials. Also, participants were required to answer 120 trials correctly before moving on to the post-experimental questionnaire, instead of just completing a total of 120 trials, regardless of error rates. Hence, not all participants saw an equal number of trials, but the amount of trials they saw was performance-contingent (participants who made 6 mistakes saw 126 trials, while those who made no mistakes saw 120). The amount of total completed

(16)

15

trials ranged from 122 to 155. Nonetheless, we decided to analyse the data in light of our hypotheses anyway.

Confirmatory Analyses

A repeated measures factorial ANOVA was run with feedbacktype and Stroop stimulus as within-subjects variables. Mauchly’s Test of Sphericity indicated that the assumption of sphericity had not been violated for the expected interaction between feedbacktype and Stroop stimulus, χ2_{(2)= .296, p = .862.}

A main effect was only found for Stroop stimulus, F(1) = 8.760, p < .05, partial η2₌

.157, indicating that participants were significantly faster when responding to control (M = 1808.7, SD = 275.8) compared to incongruent (M =1848.7, SD = 278.6) trials, and thus exhibited a classic Stroop effect. The expected interaction effect of feedbacktype by Stroop stimulus was not significant, F(2) = .616, p = .542. Thus, participants’ response latencies to incorrect trials did not decline significantly after receiving positive feedback. There were no significant effects on error rates.

Exploratory Analyses

Since it was decided to treat non-attentiveness and suspicion of the feedback as exploratory variables, repeated measures factorial ANOVA’s were run. In the first, only participants who had expressed suspicion about the feedback were removed (remaining N = 34), in the second only those who seemed to not pay attention (remaining N = 44), and in the last both groups were excluded (remaining N = 15). A main effect of Stroop

(17)

16

stimulus on response times, with participants responding faster to control (M = 615.5,

SD = 95.1) than to incongruent (M = 631.4, SD = 97.9) trials, F(1) = 5.561, p < .05, partial η2_{= .144, and thus a classic Stroop effect, was found for participants who did not express}

suspicion of the feedback. No other effects from any of the analyses were significant.

Discussion

In Study 1, we did not replicate the study by Kazén & Kuhl (2005). A programming error caused participants to see much more incongruent than control trials. This caused them to have three times as much practice at performing incongruent trials compared to performing control trials, which could have resulted in participants becoming better at completing incongruent Stroop trials at a faster rate than them becoming better at completing control Stroop trials. Stroop himself already showed that practice effects are possible with repeated completion of incongruent trials (Stroop, 1935). This could have caused the small Stroop effect in the present studies: in the original study the Stroop effect was much bigger (partial η2_{= .44; Kazén & Kuhl, 2005).}

Because this programming error prevented a fair replication of Kazén & Kuhl (2005), it was decided that Study 2 would use roughly the same setup as Study 1, the most important difference being that now participants would see as many control as incongruent trials and that participants would see 20 trials of each feedbacktype and Stroop stimulus condition.

(18)

17

Study 2 Method

Participants

Forty-three right-handed, Dutch speaking participants who were or had recently been students, with a mean age of 22.3 (SD = 3.9) participated in this study. They either signed up themselves through the University of Amsterdam’s online system (DPMS), or they were approached on the university premises. Forty-one of them had normal or corrected-to-normal (colour) vision, while two of them indicated that they usually wore glasses while working on the computer but did not have these with them. Including their data did not influence the results. Participants who were left-handed, did not have normal colour vision, were not students at an institution of higher education, and whose mother tongue was not Dutch were excluded from participation. All participants received either money or research credit.

Materials

The materials were identical to the ones in Study 1, except when mentioned otherwise. Differently from in Study 1, the average number of trials per condition was 20. The response keys were changed from S, F, H, and K to W, D, L, and P on a QWERTY-keyboard because this was thought to be a more comfortable position for the participant performing the task. Also, in the post-experimental questionnaire a question was added asking whether the participant had ever before participated in this or a similar study.

(19)

18

Procedure

The procedure was the same as in Study 1.

Results

Trial distribution

The current study also employed a 3 (feedbacktype: positive, neutral, negative) x 2 (Stroop stimulus: control, incongruent) within-subject repeated measures design. This created the same six conditions as in Study 1. Participants were supposed to see an average of 20 trials per condition. The actual trial distribution is displayed in Table 2.

Participants and Analyses

None of the participants showed any sign of misunderstanding the instructions. One participant had already taken part in Study 1, which led to removal of her or his data. Another participant answered the question whether they had participated in this or a similar study before with ‘yes’, but was not found to be on the participations lists of either study, nor did any of their answers indicate familiarity with this particular study and its research question. Therefore, it was decided to leave this participant in the dataset. None of the participants made a number of mistakes that was more than two standard deviations above the mean. Again, suspicion of the feedback and non-attentiveness were treated as exploratory exclusion criteria. Removal of one participant left the data of 42 participants. The means and standard deviations of the reaction times and error rates can be found in Table 2.

(20)

19

Just as in Study 1, the latencies for all erroneous responses were removed from the data prior to analysis (8.6% of the total number of responses). Also, too fast (<200 ms), and too slow (>1500 ms) responses were removed (0.5% of the total number of responses). The mean response times for each feedbacktype and Stroop stimulus combination were calculated and these means were log-transformed. Non-transformed means are reported for sake of clarity. Alpha was set at .05 for all analyses. Again, all analyses on response times include log-transformed response times.

Table 2. Means (and Standard Deviations) of the Reaction Times (in Milliseconds),

Error Rates, Mean Number of Trials Seen per Participant and Range of Trials Seen per Participant for the Six Different Feedbacktype and Stroop Stimulus Combinations.

Control Stroop trials Incongruent Stroop trials positive neutral negative positive neutral negative

RT 575 (86) 566 (85) 584 (83) 615 (92) 632 (90) 642 (106)

error .92 (.05) .92 (.07) .91 (.07) .91 (.08) .91 (.06) .92 (.06)

mean n trials 19.8 20.7 21 20 19 21.6

range trials 14 – 26 13 – 27 12 – 25 14 – 26 13 – 26 16 - 28

Note: N = 42 for every cell.

As can be seen from Table 2, again the trial distribution was not perfect (although much better than in Study 1). Because of the fact that we did actually give participants some performance contingent feedback, namely showing them negative feedback after each erroneous or too slow response, the programme was not completely free in

(21)

20

randomizing which trials were to be shown. As long as no erroneous or too slow responses are made, the programme can just follow its own randomization program in which it administers exactly 20 trials per condition. However, usually a participant will make an erroneous response at least once. In order to administer exactly 40 negative trials (20 control and 20 incongruent), the programme has to ‘know’ a priori how many errors a participant will make, so it can subtract this number from the total of 40 negative feedback trials that it will administer spontaneously, i.e. not in response to errors or slowness. For example, if a person makes 10 mistakes, the programme should administer no more than 30 negative feedback trials spontaneously if it wants to make sure that a participant sees no more than 40 negative feedback trials in a single experiment. Of course, the problem is that there is no way in which the programme can know a priori how many erroneous or too slow responses a participant will make, and hence it cannot know how many negative feedback trials to ‘save’ for these occasions.

To make sure that the number of negative feedback trials administered was as close to 20 as possible, the programme was told that it was not supposed to spontaneously administer negative feedback trials any more after a total of 40 negative feedback trials had been administered. As soon as this number was reached, negative feedback trials were only shown in response to erroneous or too slow responses. All participants saw a total of 120 experimental trials.

Confirmatory Analyses

A repeated measures factorial ANOVA was run with feedbacktype and Stroop stimulus as within-subjects variables and log-transformed response times as the

(22)

21

dependent variable. Mauchly’s Test of Sphericity indicated that the assumption of sphericity had not been violated for the interaction effect between feedbacktype and Stroop stimulus, χ2_{(2) = 1.393, p = .498. A main effect of Stroop stimulus was found, F(1)}

= 96.372, p < .001, partial η2 _{= .702, indicating that participants did indeed respond}

slower to incongruent trials (M = 629.9 , SD = 89.0) than to control trials (M = 575.3 , SD = 77.4). A significant main effect of feedback type, with participants having longer response times after receiving negative feedback (M = 612.9, SD = 87.8) compared to after receiving neutral (M = 599.4, SD = 80.7) or positive feedback (M = 595.6, SD = 85.5) was also found, F(2) = 3.330, p < .05, partial η2_{= .075. The Bonferroni post hoc test did}

not show any significant results. The expected interaction effect between feedbacktype and Stroop stimulus, with participants showing less of a Stroop effect after receiving positive feedback compared to after receiving neutral or negative feedback, was not significant, F(2) = 2.049, p = .135. There were no effects on error rates.

Exploratory Analyses a Priori

We decided to run some additional exploratory analyses considering our exploratory exclusion criteria. For the first we removed the participants who were suspicious about the feedback, for the second we removed the participants who failed to answer the attentiveness question, and for the third neither of these groups were included in the dataset.

Three repeated measures factorial ANOVA’s were run with feedbacktype and Stroop stimulus as within-subject variables and log-transformed response times as the dependent variable with not suspicious, attentive, and both not suspicious and attentive

(23)

22

participants, respectively. The means and standard deviations of their response times are displayed in Table 3. A significant main effect for Stroop stimulus, indicating that participants did indeed respond slower to incongruent trials than to control trials, was found in all analyses, F(1) = 53.386, p < .001, partial η2_{= .640, F(1) = 101.758, p < .001,}

partial η2_{= .761, and F(1) = 50.096, p < .001, partial η}2 _{= .705, respectively. Also, a}

significant main effect of feedbacktype was also found in every analysis, F(2) = 4.222, p < .05, partial η2_{= .123, F(2) = 4.209, p < .05, partial η}2_{= .116, and F(2) = 7.301, p < .05,}

partial η2_{= .258, respectively. Post hoc test using the Bonferroni correction were run for}

all three datasets and showed that participants’ response times were lower after receiving positive compared to negative feedback, p = .018, p = .022, and p = .001, respectively. No other effects from any of the analyses were significant. The means and standard deviations are displayed in Table 3.

Table 3. Means and Standard Deviations of Response Times of Participants Who

Were Not Suspicious About the Feedback, Participants Who Showed Attentiveness, and Participants Who Were Both Non-Suspicious and Attentive.

not suspicious attentive non-suspicious and attentive means (SDs) control incongruent positive negative 583.5 (85.8) 635.8 (97.4) 599.8 (94.2) 622.0 (94.3) 564.7 (67.9) 624.0 (87.5) 583.7 (76.2) 606.4 (89.1) 571.0 (78.1) 629.4 (99.3) 583.7 (85.8) 616.2 (99.7) N 31 33 22

(24)

23

In addition to these, three more repeated measures factorial ANOVA’s were run with feedbacktype and Stroop stimulus as within-subject variables and error rates as the dependent variable. No significant effects were found. Since non-attentiveness or suspicion that the feedback was fake could apparently not explain the absence of the effect, it was decided to check which other factors could possibly explain the absence of the expected effect in our study.

Additional Exploratory Analyses

First, it is possible that participants get habituated to seeing feedback during the task. This could diminish its rewarding (in the case of positive feedback) or punishing (in the case of negative feedback) value, leading to a weakening of its effect. Also, participants could simply get bored while performing the task, leading to them being not as serious about the feedback as in the beginning of the task, although no one indicated that they did not like the task at all: those who liked it the least were neutral about it.

Whether any of the two aforementioned possibilities could be the case was checked by running a repeated measures factorial ANOVA with feedbacktype, Stroop stimulus and block number as within-subjects variables and the log-transformed responses of all 42 participants as dependent variable. The three-way interaction between feedbacktype, Stroop stimulus and block was not significant. None of the other effects were significant either. There were no significant effects on error rates.

Another reason why the results were not as expected could be that some people indicated that they struggled with the assignment of the colours to the keys, or they mentioned that the practice block was too short for them to really get used to the

(25)

colour-24

key combinations, possibly leading to longer response times and more unnecessary errors. Although one could argue that this should be the same for both control and incongruent trials, and thus the relative difference between the two should remain the same for all feedback conditions, the possibility was explored anyway. Only a main effect of Stroop stimulus on response times was found, F(1) = 85.101, p < .001, partial η2₌

.727, with participants responding faster to control trials (M = 587.3, SD = 76.7) than to incongruent trials (M = 647.2, SD = 86.8). There were no effects on error rates.

Furthermore, some people indicated that they did not consciously pay attention to the feedback, in that they were more focused on the trials and therefore used the feedback more as a preparation signal than as a true feedback signal. If people do not regard the feedback as such, this eliminates the possibility of the feedback affecting their performance. Thus, it was decided to see whether removing these participants from the analyses would produce the expected effects. Again, only a main effect of Stroop stimulus on response times was found, F(1) = 55.822, p < .001, partial η2_{= .658, with}

people responding faster to the control trials (M = 578.2, SD = 70.8) than to incongruent trials (M = 627.2, SD = 70.5). There were no effects on error rates.

Some people indicated that they found the task easy or very easy. Finding the task very easy could result in a ceiling effect, with participants being very and equally fast and accurate on all trials, regardless of the condition of the trial. Therefore, two repeated measures factorial ANOVA’s with feedbacktype and Stroop stimulus as the independent variables were run over the data of the participants who did not find the task easy or very easy. Again, there was only a main effect of Stroop stimulus on response times, F(1) = 84.510, p < .001, partial η2_{= .701, with participants responding faster to control (M =}

(26)

25

581.2, SD = 74.8) than to incongruent trials (M = 637.3, SD = 87.8), and no effect on error rates.

Lastly, we thought that the extent to which the feedback would be congruent with participants’ own expectation could possibly affect its facilitating properties. We assumed that participants would expect to see positive feedback for all trials to which they responded faster than their personal mean, while they would expect to see negative feedback for all trials to which they responded slower than their personal mean. When they saw negative feedback, even though they made a sufficiently fast response, or when they saw positive feedback, even though they made a particularly slow response, this feedback was supposed to be unexpected. The results of a factorial ANOVA showed that the main effect of Stroop stimulus, with participants responding faster to control trials (M = 611.6, SD = 77) than to incongruent trials (M = 627.2, SD = 83) was significant, F(1) = 24.153, p < .001, partial η2_{= .998. Also, the interaction between feedbacktype and}

expectation congruence was significant, F(1) = 1267.76, p < .001, partial η2_{= .969 with}

participants responding significantly faster after seeing expected positive (positive feedback after responding faster than the personal mean, M = 484.2, SD = 57.9) compared to expected negative feedback (negative feedback after responding slower than the personal mean, M = 757.4, SD = 109.5), while after seeing unexpected negative feedback (negative feedback after responding faster than the personal mean, M = 485,

SD = 61.5) they responded faster than after seeing unexpected positive feedback

(positive feedback after responding slower than the personal mean, M = 751.1, SD = 111.9). See Figure 2 for a graphical representation of this interaction effect.

(27)

26

Figure 2. Graphical representation of the feedbacktype*expectationcongruence

interaction effect.

Discussion

Again we did not succeed in replicating Kazén and Kuhl’s (2005) Study 4. People did not show a decreased Stroop effect in response to positive feedback. However, two main effects that require further explanation were found. The first one, the main effect of Stroop stimulus on response times, indicates that participants did indeed show a classic Stroop effect: they were slower when responding to the incongruent compared to the

0 100 200 300 400 500 600 700 800 900 1.000 Expected Unexpected R es po ns e ti m es in m s Expectation Congruence Positive feedback Negative Feedback

(28)

27

control trials. The second one, the main effect of feedbacktype on response times, indicates that participants tended to respond more slowly after seeing negative feedback, compared to after seeing positive feedback, regardless of whether the Stroop trial was a control or incongruent trial. But what does this second main effect actually tell us?

In the cognitive literature, there is a well-known phenomenon called ‘post-error slowing’. Post-error slowing is the observation that response times are higher following an erroneous compared to a correct trial. This phenomenon is attributed to adaptive control mechanisms that motivate one to behave more carefully to reduce the probability of making another error (Notebaert et al., 2009). Since in the present study, response times after receiving negative feedback were higher than after receiving positive feedback, indicating slower responding after seeing negative feedback, we think that such a mechanism may be at play here.

Since post-error slowing by default only occurs after an error has been made, and since in the present study erroneous responses were removed before analysis, one could argue that the main effect of feedbacktype could not be due to post-error slowing in the present study: responses seem to slow down after randomly generated negative feedback as well. However, because it was emphasized in the instructions that both speed and accuracy would be taken into account when generating the feedback, participants could still regard random negative feedback as indicative of an error in the speed department. Assuming that, when pressing the wrong button, one is aware of their erroneous response, randomly generated negative feedback after a correct response automatically conveys the message to the participant that their response may be indeed correct, but given too slowly. Since both speed and accuracy were emphasized

(29)

28

to be important measures of performance, participants could experience this as an error as well, hence resulting in post-error slowing.

Our last exploratory analysis also offers evidence for this explanation. It shows that the mean response time to expected positive feedback was faster but close to the mean response time to unexpected negative feedback, while the mean response times to expected negative and unexpected positive feedback were much slower. Hence, it seems that participants only display post-error slowing if they are aware of their ‘error’: in expected negative feedback-trials as well as in unexpected positive feedback-trials participants expected to see negative feedback, and were thus aware of their slowness. Even in the unexpected negative feedback trials, participants had slower response times and thus showed post-error slowing compared to the expected positive trials.

General Discussion

We attempted to replicate Kazén and Kuhl´s (2005) finding that positive feedback facilitates performance on the Stroop task. We did not succeed in this attempt in either of the two studies discussed above. It is possible that some of the changes we introduced into the design of the original study may have contributed to the failure to replicate. As mentioned previously, these changes were thoroughly discussed with one of the original authors, Miguel Kazén, through email, and neither he nor we had any reason to expect a failure to replicate based on these changes.

One of the changes that was introduced into the design is that participants could in fact receive performance contingent feedback (namely, negative feedback when making

(30)

29

an erroneous response or when not responding within the response window), while Kazén and Kuhl’s (2005) feedback was completely random. This could have led to participants being preoccupied with thinking about the validity of the feedback (since sometimes it was valid but sometimes it was not) rather than with performing well on the Stroop task.

Furthermore, due to the way in which the task was programmed, it is possible for some participants to have seen more negative feedback trials than others. However, in the second study the ranges of trials within which participants received negative feedback do not differ too greatly from the ranges of trials in which participants received positive or neutral feedback, so we do not consider this to be a real problem. Also, in all cases, it is quite likely that participants saw more negative feedback trials at the beginning of the experiment (due to them making more errors in the beginning of the experiment) than later on in the experiment, affecting the results. This has already been discussed in Study 2 and did not seem to be a problem.

A second change that was introduced into the design is that we did not administer a battery of questionnaires before having participants complete the Stroop task. These questionnaires measured problem solving styles, how people decide on and commit to a certain course of action, Big Five personality, action versus state orientation, and momentary mood. It is possible that filling out these questionnaires may have primed certain constructs, such as problem-solving ability, that facilitated performance on the Stroop task.

The third change that was introduced into the design was that every participant was assigned to one of four possible colour-key combinations, in contrast with the original study where every participant received an individual colour-key assignment. Also, in the

(31)

30

present experiment, stickers in the corresponding colours were stuck on the keyboard to serve as a reminder for participants. This could have made the task too easy. This has been tested in one of the exploratory analyses in Study 2 and did not turn out to be the cause of the effect’s absence.

The fourth change was that we included a response window of two seconds within which participants had to respond. This could have made the participants feel more hurried, resulting in paying less attention to the feedback and more attention to responding as fast as possible. Hence, this may have caused the similar response times in all feedback conditions.

Something that struck us about the data is that the participants’ response times were extraordinarily fast for a four-colour Stroop task. It was suggested that maybe the fact that people could prepare for the trial for longer than is generally the case (they could prepare for as long as the fixation point and the feedback sign was visible) led them to respond faster, but then participants in Kazén and Kuhl should have had equally fast response times, which is not the case. A second possibility is that participants nowadays, ten years after Kazén and Kuhl conducted their study, are more used to doing Stroop tasks, or working on a computer as such, and are therefore better at these computerized tasks in general. However, an examination of recent Stroop studies deemed this argument improbable, since their reported reaction times were all considerably higher than those reported in our study (Taylor et al., 2015; Navarrete, Sessa, Peressotti, & Dell’Acqua, 2015; Cacioppo, Balogh, & Cacioppo, 2015; Heidlmayr et al., 2014; Elchlepp, Lavric, Mizon, & Monsell, 2012). A third option is that our Dutch participants were much less motivated to perform well on the task, and were more preoccupied with being done and receiving their reward as soon as possible. This could have led to them responding

(32)

31

faster with less regard for correctness. However, there was no speed/accuracy trade off. Also, everyone indicated that they tried at least ‘a bit hard’ to perform well.

A last possible explanation why we did not succeed in replicating Kazén and Kuhl is that the effect simply does not exist, or at least not in our sample. Kazén and Kuhl’s Study 4 was their only reported example employing a single task structure: all other reported studies employed a multiple task structure. Also, the original study was already a little underpowered (N = 25) and the found effect was quite small (partial η2_{= .12). It}

is possible that the way in which achievement orientation, and thus intention memory, was supposed to be loaded, was not successful in doing that in the present study. The feedback-prime might not have conveyed enough of an achievement message to the participants. The effect might therefore not have been present in the current study.

In short, there could be several reasons why we did not succeed in replicating the finding that positive feedback can facilitate performance on the Stroop task when participants are allowed to load their intention memory. What seems to be the next step in this line of research is to go back and replicate Kazén and Kuhl’s Study 4 according to the original design of the study, without the changes that were introduced by us. If, and only if, such a replication attempt succeeds, can we start to decipher which of our alterations caused failure of our own replication attempts and why. In conclusion, the facilitating effect of positive feedback in a single-task paradigm does not seem to be very robust across samples and methods.

(33)

32

References

Burgers, C., Eden, A., Van Engelenburg, M.D., & Buningh, S. (2015). How feedback boosts motivation and play in a brain-training game. Computers in Human Behavior, 48, 94 – 103.

Cacioppo, S., Balogh, S., & Cacioppo, J.T. (2015). Implicit attention to negative social, in contrast to non-social, words in the Stroop task differs between individuals high and low in loneliness: Evidence from event-related brain microstates. Cortex, 70,

213 – 233.

Carpentier, J., & Mageau, G.A. (2013). When change-oriented feedback enhances motivation, well-being and performance: A look at autonomy-supportive feedback in sport. Psychology of Sport and Exercise, 14, 423 – 435.

Carver, C. (2003). Pleasure as a sign you can attend to something else: Placing positive feelings within a general model of affect. Cognition and Emotion, 17, 2, 241 – 261. Deci, E.L., Koestner, R., & Ryan, R.M. (1999). A meta-analytic review of experiments

examining the effects of extrinsic rewards on intrinsic motivation. Psychological

Bulletin, 125, 6, 627 – 668.

Elchlepp, H., Lavric, A. , Mizon, G.A., & Monsell, S. (2012). A brain-potential study of preparation for an execution of a task-switch with stimuli that afford only the relevant task. Human Brain Mapping, 33, 1137 – 1154.

Gable, P.A., & Harmon-Jones, E. (2011). Attentional consequences of pregoal and postgoal affects. Emotion, 11, 6, 1358 – 1367.

(34)

33

Heidlmayr, K., Moutier, S., Hemforth, B., Courtin, C., Tanzmeister, R., & Isel, F. (2014). Successive bilingualism and executive functions: The effect of second language use on inhibitory control in a behavioural Stroop Colour Word task. Bilingualism:

Language and Cognition, 1 – 16, DOI: 10.1017/S1366728913000539.

Kazén, M., & Kuhl, J. (2005). Intention memory and achievement motivation: Volitional facilitation and inhibition as a function of affective contents of need-related stimuli.

Journal of Personality and Social Psychology, 89, 3, 426 – 448.

Kohn, A. (1993). Punished by rewards: The trouble with gold stars, incentive plans, A’s,

praise and other bribes. New York, NY: Houghton Mifflin.

Kuhl, J. & Kazén, M. (1999). Volitional facilitation of difficult intentions: Joint activation of intention memory and positive affect removes Stroop interference. Journal of

Experimental Psychology: General, 128, 3, 382 – 399.

Ilies, R., & Judge, T.A. (2005). Goal regulation across time: The effects of feedback and affect. Journal of Applied Psychology, 90, 3, 453 – 467.

Mizruchi, M.A. (1991). Urgency, motivation and group performance: The effect of prior success on current success among professional basketball teams. Social Psychology

Quarterly, 54, 2, 181 – 189.

Navarrete, E., Sessa, P., Peressotti, F., & Dell’Acqua, R. (2015). The distractor frequency effect in the colour-naming Strook task: An overt naming event-related potential study. Journal of Cognitive Psychology, 27, 3, 277 – 289.

Notebaert, W., Houtman, F., Van Opstal, F., Gevers, W., Fias, W., & Verguts, T. (2009). Post-error slowing: An orienting account. Cognition, 111, 275 – 279.

(35)

34

Podsakoff, P.M., & Farh, J.L. (1989). Effects of feedback sign and credibility on goal setting and task performance. Organizational Behavior and Human Decision

Processes, 44, 45 – 67.

Webster, J., & Martocchio, J.J. (1992). Microcomputer playfulness: Development of a measure with workplace implications. MIS Quarterly, 16, 2, 201 – 226.

Stroop, J.R. (1935). Studies of interference in serial verbal reactions. Journal of

Experimental Psychology, 18, 643 – 662.

Taylor, R., Schaefer, B., Densmore, M., Neufeld, R.W.J., Rajakumar, N., Williamson, P.S., & Théberge, J. (2015). Increased glutamare levels observed upon functional activation in the anterior cingulate cortex using the Stroop Task and functional spectroscopy. Neuroreport, 26, 3, 107 – 112.

Van Dijk, D., & Klugel, A.N. (2011). Task type as a moderator of positive/negative feedback effects on motivation and performance: A regulatory focus perspective.

(36)

35 Appendix Demographic questionnaire 1. Wat is je leeftijd? 2. Wat is je geslacht? 0 man 0 vrouw 3. Ik studeer

0 aan de Universiteit van Amsterdam (ga naar vraag 6) 0 aan de Vrije Universiteit (ga naar vraag 6)

0 aan de Hogeschool van Amsterdam (ga naar vraag 6) 0 ergens anders (ga naar vraag 5)

0 niet (ga naar vraag 4) 4. Wat doe je in het dagelijks leven? 5. Waar studeer je?

6. Wat studeer je? 0 Psychologie

0 Communicatiewetenschappen

0 Pedagogiek of een soortgelijke studie 0 Economie/Bedrijfskunde

0 Anders

7. Draag je normaal gesproken een bril of lenzen als je op de computer werkt?

0 nee

0 ja, en die heb ik nu ook bij me 0 ja, maar die heb ik nu niet bij me

(37)

36

Instructions

In dit experiment zul je kleurwoorden (bijvoorbeeld GROEN of ROOD) of reeksen van XXXX'en te zien krijgen. Het is de bedoeling dat je reageert op de kleur van het woord of de letterreeks en de betekenis van het woord negeert. Wanneer je bijvoorbeeld groen

ziet, druk je zo snel mogelijk op de toets op het toetsenbord met de blauwe sticker erop. Wanneer je bijvoorbeeld XXXX ziet, druk je zo snel mogelijk op de toets op het

toetsenbord met de rode sticker erop.

Wanneer een woord of letterreeks op het scherm verschijnt, druk dan zo snel mogelijk op de toets op het toetsenbord die dezelfde kleur heeft als het woord of de letterreeks. Wanneer je hier langer dan 2 seconden over doet, zal de trial automatisch fout gerekend worden. Het is de bedoeling dat je de taak zo goed mogelijk uitvoert. Dit betekent dat je zowel snel als accuraat moet zijn.

Aan het begin van iedere trial zal je een fixatiepunt ( ) zien. Daarna zal er een serie van neutrale tekens ( ), plus tekens ( ), of min tekens ( ) verschijnen. Deze tekens zijn een voorbereidingssignaal voor de direct volgende trial, waarin je moet reageren op kleuren (zoals hiervoor uitgelegd). Daarnaast geven de tekens je informatie over of je snelheid en accuraatheid gelijk is gebleven, is verbeterd of is verslechterd. Met deze procedure willen we uitzoeken of je hoort bij de groep van personen die hun prestatie verbeteren door het krijgen van positieve feedback of dat je hoort bij de groep van mensen die gemotiveerd zijn om hun prestatie te verbeteren na het ontvangen van negatieve feedback.

Dus:

betekent dat je je prestatie hebt verbeterd vergeleken met de voorgaande trials of dat je prestatie vergeleken met je referentiegroep (mensen van een vergelijkbare leeftijd, geslacht en opleidingsniveau) bovengemiddeld was.

betekent dat je prestatie is verslechterd vergeleken met de voorgaande trials of dat je prestatie vergeleken met je referentiegroep (mensen van een vergelijkbare leeftijd, geslacht en opleidingsniveau) beneden gemiddeld was.

betekent dat je prestatie constant is gebleven vergeleken met de voorgaande trials of dat je prestatie vergeleken met je referentiegroep (mensen van een vergelijkbare leeftijd, geslacht en opleidingsniveau) gemiddeld was.

Houd je vingers gedurende het hele experiment als volgt op de toetsen: middelvinger linkerhand: groen

wijsvinger linkerhand: geel

wijsvinger rechterhand: rood middelvinger rechterhand: blauw

Probeer zo min mogelijk naar het toetsenbord te kijken. Mocht je nou echt vergeten welke kleur bij welke toets hoort, kijk dan tussen de trials door.

Voordat je echt gaat beginnen, herhalen we nog even kort wat de bedoeling is.

(38)

37

reageren door op de bijbehorende toets te drukken.

betekent dat je je prestatie hebt verbeterd of dat je prestatie bovengemiddeld was. betekent dat je prestatie is verslechterd of dat je prestatie beneden gemiddeld was.

betekent dat je prestatie constant is gebleven of dat je prestatie gemiddeld was.

(39)

38

Post-test questionnaire

Je bent nu bijna klaar met het experiment. Tot slot willen we je vragen om nog een aantal vragen te beantwoorden.

We willen graag weten hoe je het doen van dit experiment hebt ervaren. We willen je daarom vragen om je gedachten en gevoelens met betrekking tot dit experiment met ons te delen.

Iemand’s prestatie in een experiment staat niet op zichzelf. Naast situationele variabelen kunnen ook individuele motivatie en vaardigheden de prestatie voor een groot deel beïnvloeden. Om ons onderzoek naar de kwaliteit van prestatie optimaal te kunnen doen zijn we geïnteresseerd in bepaald factoren van jou, de presteerder. We zijn er in het bijzonder geïnteresseerd in om te weten of je wel de tijd neemt om de instructies goed te lezen. Als je dit niet doet kan het zijn dat sommige van onze manipulaties die afhankelijk zijn van deze instructies niet effectief zijn. Om aan te tonen dat je de instructies

inderdaad goed hebt gelezen willen we je vragen om de vraag hieronder te negeren en gewoon op de eerste optie (“heel erg slecht”) te klikken. Hartelijk bedankt.

Hoe slecht of goed voel je je vergeleken met gisteren? 0 heel erg slecht

0 slecht 0 neutraal 0 goed

0 heel erg goed

Gedurende het experiment ontving je feedback over je prestatie. We zijn geïnteresseerd in je ervaring tijdens het ontvangen van deze feedback. We willen je daarom vragen om je gedachten en gevoelens met betrekking tot het ontvangen van feedback gedurende dit experiment met ons te delen.

Hoe hard heb je je best gedaan om goed te presteren op de kleurentaak? 1 helemaal niet mijn best gedaan

2 niet mijn best gedaan

3 een beetje niet mijn best gedaan 4 neutraal

5 een beetje mijn best gedaan 6 wel mijn best gedaan

7 heel erg mijn best gedaan

Hoe leuk of vervelend vond je het om de kleurentaak te doen? 1 heel erg leuk

2 leuk

3 een beetje leuk

4 niet leuk maar ook niet vervelend 5 een beetje vervelend

6 vervelend

(40)

39

Hoe makkelijk of moeilijk vond je het om de kleurentaak te doen? 1 heel erg makkelijk

2 makkelijk

3 een beetje makkelijk

4 niet makkelijk maar ook niet moeilijk 5 een beetje moeilijk

6 moeilijk

7 heel erg moeilijk Wat betekende ?

0 Dat mijn prestatie verbeterd is ten opzichte van de vorige trials OF dat mijn prestatie bovengemiddeld is vergeleken met die van andere mensen uit mijn

referentiegroep (mensen van een equivalente leeftijd, geslacht en opleidingsniveau). 0 Dat mijn prestatie constant is gebleven ten opzichte van de vorige trials OF dat mijn prestatie gemiddeld is vergeleken met die van andere mensen uit mijn

referentiegroep(mensen van een equivalente leeftijd, geslacht en opleidingsniveau). 0 Dat mijn prestatie verslechterd is ten opzichte van de vorige trials OF dat mijn prestatie ondergemiddeld is vergeleken met die van andere mensen uit mijn

referentiegroep (mensen van een equivalente leeftijd, geslacht en opleidingsniveau). Wat betekende ?

referentiegroep (mensen van een equivalente leeftijd, geslacht en opleidingsniveau). Hoe beïnvloedde het ontvangen van positieve feedback je prestatie op de volgende trial? Het gaat hierbij om wat je eigen gevoel is.

0 Het verbeterde mijn prestatie 0 Het beïnvloedde mijn prestatie niet 0 Het verslechterde mijn prestatie 0 Ik weet het niet

(41)

40

Hoe beïnvloedde het ontvangen van neutrale feedback je prestatie op de volgende trial? Het gaat hierbij om wat je eigen gevoel is.

Hoe beïnvloedde het ontvangen van negatieve feedback je prestatie op de volgende trial? Het gaat hierbij om wat je eigen gevoel is.

Hoe vaak kwam de feedback overeen met jouw eigen subjectieve gevoel over je prestatie op de voorgaande trial? 0 nooit 0 zelden 0 soms 0 vaak 0 altijd

Hoe accuraat gaf de feedback in dit experiment weer hoe je had gepresteerd op de voorgaande trial?

1 helemaal niet accuraat 2 niet accuraat

3 een beetje niet accuraat 4 weet ik niet

5 een beetje accuraat 6 accuraat

7 heel accuraat

Wat denk je zelf dat voor jou in het dagelijks leven het meest motiverend werkt? 0 positieve feedback

0 negatieve feedback