Master Thesis Educational Sciences 70240485MY
The effect of Binding Study Advice on students’ extrinsic motivation and achievement emotions: an experimental investigation
Word count: 9,026
Author Coen Bakker
Student number: 10164952
Supervisor and first reviewer Second reviewer
2 Table of Contents Abstract ... 3 Introduction ... 4 Method ... 10 Participants ... 10 Materials ... 10 Academic motivation ... 10
Binding study advice ... 11
Academic emotions ... 11 Academic achievement ... 12 Procedure ... 12 Analysis ... 13 Preliminary analyses ... 13 Selectivity bias ... 13 Scaling... 13 Confirmatory analyses ... 14 Exploratory analyses ... 14 Results ... 14 Preliminary analyses ... 14 Selectivity bias ... 15 Scaling... 15 Confirmatory analyses ... 17 Exploratory analyses ... 18 Discussion ... 20 References ... 24
3 Abstract
The goal of the present study was to investigate the effect of Binding Study Advice (BSA-policy) on students’ extrinsic motivation and achievement emotions. BSA-policy is a method in Dutch higher education to select first-year students for the second year of their programs. It was predicted that BSA-policy increases extrinsic motivation. It was explored whether activating positive, activating negative, deactivating positive and deactivating negative emotions mediate this effect. Furthermore, extrinsic motivation was predicted to positively covariate with academic achievement. An
experimental design was used for which the participants were recruited from two methodology courses at the University of Amsterdam (N = 42). A BSA and a non-BSA condition were created. Students in the BSA-condition were manipulated into contemplating the possibility of being deselected because of the BSA-policy. The students in the non-BSA-condition were not. Extrinsic motivation was measured with the Academic Motivation Scale. Seven achievement emotions (anger, anxiety, enjoyment, hope, hopelessness, pride and shame) were measured with the Test Emotion Scale of the Achievement Emotions Questionnaire. Students in the BSA-condition were not found to have higher extrinsic motivation scores than the students in the non-BSA-condition. None of the
achievement emotions categories were found to play a mediating role. The results indicate that BSA-policy does not increase extrinsic motivation. Furthermore, activating positive, activating negative, deactivating positive and deactivating negative achievement emotions were not found to play a mediating role in the effect of BSA-policy on extrinsic motivation. Implications are discussed.
Key words: Binding Study Advice, extrinsic motivation, achievement emotions, academic achievement, study progress.
4 Introduction
Students’ study progress is a persisting cause of concern in higher education. Academic dismissal policies, with which universities decide on whether students are allowed to continue their study, are aimed at increasing study progress. In the Netherlands Binding Study Advice (BSA-policy) is such an academic dismissal policy in higher education. BSA-policy is aimed at selecting first-year students for their progression into their second year of their programs. When students do not attain a certain number of credits in their first year, they receive a negative study advice. Consequently, they are not allowed to continue to their second year. The number of credits students must attain depends on the specifics that are set by their university. Students are, in some cases, also unable to enrol into the same and similar programs at their university for a certain amount of years. Although BSA-policy is widely used in the Netherlands, it is unclear what its effects are on students’ motivation.
It is important to gain knowledge about the effect of BSA-policy on students’ motivation. Motivation is found to be a strong determinant of students’ study success (Meece, Anderman, & Anderman, 2006). Besides benefitting students, higher study success also increases the efficiency of higher education (Van den Berg & Hofman, 2005). The knowledge that we gain about the effect of BSA-policy on students’ motivational processes can, therefore, be used to improve educational policy aimed at increasing the efficiency of higher education. Also students’ wellbeing would likely be benefitted when their motivation is increased. A lack of motivation has, for example, been found to be a strong antecedent of burnout (David, 2010). For these reasons the present study focussed on
investigating the effect of BSA-policy on students’ motivational processes.
Theories on the effect of incentives on students’ motivation suggest that BSA-policy can be expected to increase motivation. More specifically, BSA-policy is expected to increase extrinsic motivation. Extrinsic motivation refers to a person’s drive to behave a certain way because of an external reward (e.g. money or praise) or punishment (e.g. a fee or pain) that results from their behavior (Deci & Ryan, 2008; Reeve, 2002). Extrinsic motivation is often compared with intrinsic motivation, which refers to human’s inherent drive to exert certain behavior (Deci & Ryan, 2008). Many theories propose that rewards and punishments regulate academic motivation. Among other theories, Self-Determination Theory (Deci & Ryan, 2008) and Expectancy-Value Theory of Achievement Motivation (Wigfield & Eccles, 2000) suggest that the effect of an incentive on motivation depends on certain factors. One factors that is commonly described is value. BSA-policy can be expected to extrinsically motivate students more when they value the consequences of a negative study advice more. Although this may vary among students, it is likely that these consequences would indeed be found important by most students, since being expelled from a program has negative consequences for students’ finances and career. Therefore, BSA-policy can be expected to provide students with extrinsic motivation to attain a certain minimal number of credits in their first year. In the present study it was, consequently, hypothesized that BSA-policy increases students’ extrinsic motivation (Hypothesis I; also see Table 1 on page 9).
5 Although theory predicts a positive effect of BSA-policy on extrinsic motivation, the scarce literature on BSA-policy suggests that BSA-policy does not motivate students. No research, however, has focussed on the effect of BSA-policy on students’ extrinsic motivational processes specifically. De Koning, Loyens, Rikers, Smeets and Van der Molen (2014) investigated the impact of the introduction of BSA-policy on students’ self-study time, a variable strongly related to students’ motivation (Doumen, Broeckmans & Masui, 2014). They did so by comparing four cohorts of students of the Erasmus University Rotterdam: two BSA cohorts and two pre-BSA cohorts. It was predicted that BSA-policy had improved students’ self-study time. However, no evidence was found for their hypothesis. The results of this study must, however, be approached carefully. Firstly, more research into the relation between BSA-policy self-study time is needed to either confirm or discard these findings, because presently research into this relation is scarce. In the absence of more research it cannot be excluded that the study’s findings are a mere fluke. Secondly, the quasi-experimental design of the study might have allowed for confounding factors to affect the outcome variables. It is possible that around the time of the introduction of BSA-policy some unforeseen change in context or student population occurred, impacting students in the BSA cohort in a way that influenced their learning behavior. This makes it difficult to draw conclusions about the effects found in the study.
A few studies have been dedicated to investigating the effect of BSA-policy on study
progress. Study progress is found to be positively related to motivation (e.g. Busato, Prins, Elshout, & Hamaker, 2000). Arnold (2014) measured the effect of BSA-policy on study progress and first-year dropout, by comparing bachelor programs that used BSA-policy to programs that did not, between 2002 and 2007. It was found that BSA-policy can help bring dropout forward in time. Both first-year dropout rates and the completion rates of first-year survivors were found to have significantly increased. Duijndam and Scheepers (2009) found similar results in their study, comparing BSA cohorts to pre-BSA cohorts. In the study by Arnold (2014), however, the overall completion rates, when measured from the start of the program to a cut-off date four years later, were not found to have increased significantly. In the programs with BSA-policy the completion rates increased by 5 to 9%, compared to the programs without BSA-policy. This increase was considered small by the author, given the much larger proportions of study delay in higher education and the anticipated effect of BSA-policy. An important conclusion that was drawn from the study was that, although students are shown to be prevented from languishing in their programs, BSA-policy does not prevent students from languishing in higher education. In other words, students who progress slowly in one program, are often found to progress slowly in their new programs as well, after having received a negative study advice (also see Arnold, & Van de Brink, 2010). These findings suggest that BSA-policy might not have a significant positive effect on motivation.
It is striking that BSA-policy does not seem to significantly motivate students, since it is commonly assumed by Dutch universities and the Dutch government that BSA-policy does increase students’ motivation (ResearchNed, 2009; Ministry of Education, Culture and Science, 2015). In a
6 study on the expectations of universities of the effect of BSA-policy on motivation, it was shown that many universities do expect BSA-policy to increase students’ motivation (Ministry of Education, Culture and Science, 2015). For universities and the government to gain a better understanding of the capacity of BSA-policy to increase students’ motivation, more research is needed. Consequently, universities and the government would be able make more informed decisions about the creation and adaptation of their educational policy.
As was outlined above, theory on incentives, like a negative study advice, predict BSA-policy to increase extrinsic motivation. The control-value theory of achievement emotions also predicts an effect of BSA-policy on extrinsic motivation (Pekrun, 2006). It, however, also provides an
explanation for this effect. Control-value theory offers an integrative framework, specifying the antecedents of students’ extrinsic motivation. One antecedent of students’ extrinsic motivation is achievement emotion. Achievement emotions are defined as emotions that are tied directly to individual’s achievement activities or outcomes (Pekrun, Goetz, Titz, & Perry, 2002). Achievement emotions that are commonly referred to in literature are enjoyment, hope, pride, relief, gratitude, shame, anger, disappointment, anxiety, boredom and hopelessness. In control-value theory these achievement emotions mediate the effect of certain contextual factors on students’ extrinsic
motivation. One contextual factor in the control-value theory are the consequences that derive from the achievement of students. The consequences of a negative study advice are an example of such achievement consequences. In other words, control-value theory proposes that BSA-policy has an effect on students’ extrinsic motivation and that this effect is mediated by achievement emotions. Literature, however, also proposes that the effect of achievement emotions on extrinsic motivation depends on the specific emotions that students experience and the properties of these emotions.
In the literature on achievement emotions two properties of achievement emotions are often focussed on (Pekrun, 2006; Lam, Chen, Zhang, & Liang, 2015). Firstly, achievement emotions can be categorized by their valence (Figure 1 on p. 7). Achievement emotions with positive valence (e.g. enjoyment, hope and relief) are intrinsically attractive for the person that is experiencing the emotion. Conversely, negative valence (e.g. shame, anger and hopelessness) is experienced as aversive.
Secondly, achievement emotions can be organized according to their degree of activation (Figure 1 on p. 7). Activating achievement emotions stimulate a person to engage in tasks that this person believes to be necessary in the given situation to improve his or her learning and achievement. Deactivating achievement emotions inhibit persons from engaging in these tasks. Examples of activating achievement emotions are enjoyment and hope. Relief, boredom and hopelessness are examples of deactivating emotions.
7
Students’ activating positive emotions have been found to correlate positively with several variables that are close related to academic motivation, namely, students’ interest, effort invested learning, elaboration of learning material and self-regulation of learning (Pekrun et al., 2011). Negative deactivating emotions were, on the other hand, found to be detrimental to these variables. Given these findings, it can be speculated that activating positive emotions and deactivating negative emotions respectively increase and decrease motivation. To the knowledge of the author, no research has, however, been done specifically into the effect of activating positive and deactivating negative emotions on motivation. To investigate in what way activating positive (Exploratory Hypothesis A) and deactivating negative emotions (Exploratory Hypothesis B) might possibly mediate the effect of BSA-policy on extrinsic motivation, explorative research was conducted (also see Table 1 on p. 9 and Figure 2 on p. 10). In particular, it was explored whether activating positive emotions positively covariate, and deactivating negative emotions negatively covariate, with extrinsic motivation as well as intrinsic motivation.
In the research on achievement emotions more ambiguous results have been found for the effect of deactivating positive and activating negative emotions on motivation. Research on test anxiety, for example, has yielded mixed results. Test anxiety has been found to decrease motivation in several studies (e.g. Hancock, 2001; Stoeber, Feast, & Hayward, 2009), while other studies suggest that test anxiety has a positive effect on motivation (e.g. Ergene, 2011). Moreover, in many of these
Activation
Activating
negative
e.g. anger & anxietyActivating
positive
e.g. enjoyment &hope
Deactivating
negative
e.g. hopelessness &boredom
Deactivaiting
positive
e.g. relief &prideValence
8 studies the variance of the effect of test anxiety on motivation has been found to be large (Turner & Schallert, 2001). These findings have led scientists to conclude that there are large individual differences in the effect of test anxiety on motivation, including both positive and negative effects. Similar findings have been found for shame (Turner & Schallert, 2001) and anger (Boekaerts, 1994; Pekrun et al., 2011). Literature on these topics is, however, scarce and little consensus exist about the effect of deactivating positive and activating negative emotions on motivation. Therefore, in the present study, the mediating effects of deactivating positive (Exploratory Hypothesis C) and
activating negative emotions (Exploratory Hypothesis D) were investigated by means of exploratory analysis (also see Table 1 on p. 9 and Figure 2 on p. 10)
In the present study, also, the effect of extrinsic motivation on academic achievement was investigated. Although some studies found a negative relation between extrinsic motivation and academic achievement, most research has shown that an extrinsic motivation increases academic achievement (Ayub, 2010; Fortier, Vallerand, & Guay, 1995; Vallerand, 1997). Therefore, in the present study, it is hypothesized that extrinsic motivation positively covariates with academic achievement (Hypothesis II; also see Table 1 on p. 9).
It was chosen in the present study to use an experimental design to manipulate BSA-policy. Using an experimental design has the benefit that it makes it possible to draw conclusions about the causality between variables. Studies that have focused on the effects of BSA-policy, typically utilize correlational or quasi-experimental designs. Drawing conclusions about the causal relations between variables is less justified for these designs, than for experimental designs. Therefore, an experimental design is an appropriate choice for the present study and a valuable addition to the existing literature. In the present study’s experimental design two conditions were created. Students in the BSA-condition were reminded of the consequences of a negative study advice and instructed to imagine that their study advice depended on whether they passed a specific exam. Students in the non-BSA-condition were not reminded of BSA-policy and not asked to imagine the above. There are numerous examples of psychological studies that have successfully utilized similar (e.g. Thomaes, Bushman, de Castro, Cohen, & Denissen, 2009).
To test the effect of BSA-policy on extrinsic motivation and achievement emotion, these variables were measured using self-reportage. It was expected that students’ scores on extrinsic motivation would be higher in the BSA-condition, than in the non-BSA-condition. Furthermore, it was explored whether students’ scores on activating positive, activating negative, deactivating positive and deactivating negative emotions, mediated the effect of the manipulation on extrinsic motivation scores.
It is important to keep in mind that the present study’s manipulation is designed to alter students’ acute experience of achievement emotions (i.e. achievement emotion as a state). The manipulation is not expected to change students’ structural experience of achievement emotions (i.e.
9 achievement emotion as a trait). A difference in achievement emotion scores between the two
conditions, therefore, indicated a difference in emotional state, not emotional trait.
Lastly, students’ scores on their final exam were used as measures of their academic
achievement. It was expected that students’ scores on extrinsic motivation would positively correlate with their final exam scores.
Table 1
An overview of the hypotheses
Hypothesis I BSA-policy increases students’ extrinsic motivation
Hypothesis II Extrinsic motivation positively covariates with students’ achievement
Exploratory A Do activating positive emotions mediate the effect of BSA-policy on extrinsic motivation?
Exploratory B Do activating negative emotions mediate the effect of BSA-policy on extrinsic motivation?
Exploratory C Do deactivating positive emotions mediate the effect of BSA-policy on extrinsic motivation?
Exploratory D Do deactivating negative emotions mediate the effect of BSA-policy on extrinsic motivation?
10
.
Method Participants
The sample consisted of students enrolled in two methodology courses (‘Research Design’ and ‘Applied Methodology and Statistics’) at the Department of Child Development and Education of the University of Amsterdam. Research Design and Applied Methodology and Statistics are part of, respectively, the first and the second year of the bachelor phase of the pedagogical and educational science programs at the university. In Research Design 87 students were enrolled, of which 70 students had taken the final exam of the course. In Applied Methodology and Statistics 175 students were enrolled and 150 had taken the final exam. In total 74 participants were recruited from the two courses. A recruiter visited a lecture of both Research Design and Applied Methodology and Statistics to asks the students to fill out the present study’s questionnaire. Earlier in the recruitment phase the teacher of the course Research Design had requested the students to fill out the questionnaire. However, the number of students who consequently filled out the questionnaire was considered too low. The mean age of the recruited students is 21.4 (SD = 2.3). The sample consists of 66 females and eight males.
Materials
Academic motivation
The Academic Motivation Scale (AMS) was used to measure students’ intrinsic and extrinsic motivation (Vallerand et al., 1992). The motivation measure consisted of 24 items in total. Intrinsic
Figure 2. Explored mediating effects of achievement emotions on the effect of BSA-policy on extrinsic motivation
Activating positive emotions Extrinsic motivation BSA-policy Exploratory Hypothesis A Activating negative emotions Extrinsic motivation BSA-policy Exploratory Hypothesis B Deactivating positive emotions Extrinsic motivation BSA-policy Exploratory Hypothesis C Deactivating negative emotions Extrinsic motivation BSA-policy Exploratory Hypothesis D
11 and extrinsic motivation were both measured with twelve items. All items were rated on a 5-point Likert scale (1 = does not correspond at all; 5 = corresponds exactly). An example of an items is: ‘I go to college because I experience pleasure and satisfaction while learning new things’. The AMS has been shown to be a reliable measure for students. Vallerand et al. (1992) found a high average value for the internal consistency (α = .81) and temporal stability over a one-month period (mean test-retest correlation = .79). Moreover, the AMS has been found to be a valid measure of motivation (Vallerand et al., 1992; Vallerand et al., 1993; Fairchild, Horst, Finney, & Barron, 2005).
The English version of the AMS was used because to the present date no Dutch version of the scale is available. To check to what extent students had run into problems understanding the questions of the AMS, students were asked to indicate how many questions they had been unable to answer because they had not understood the English of the questions. It was expected that this would be a unlikely problem because at Dutch universities students are accustomed to reading English and the majority of their study materials is in English.
Binding study advice
The participants were randomly assigned to an BSA and a non-BSA condition. Students in the BSA-condition were manipulated to contemplate BSA-policy and the consequences of a negative study advice, while students in the non-BSA-condition were not. At the start of the present study’s questionnaire the students in the BSA-condition were asked to carefully read a short text outlining the consequences of BSA-policy at their university. Afterwards, they were asked to imagine that they needed their present course’s credits to get a positive BSA. They were also instructed to imagine that they were in the test hall and that they were about to take the exam for their methodology class. Subsequently, the students were instructed to formulate at least five sentences about ‘what went through their minds’ as they waited for the exam to start. Students in the non-BSA condition received different instructions. As in the BSA-condition the students in the non-BSA-condition were instructed to imagine that their exam was about to start and to formulate at least five sentences about ‘what went through their minds’. However, they were not prompted with the informative text about BSA-policy and they were not asked to imagine that they needed the course’s credits to obtain a positive BSA.
Academic emotions
The Test Emotion Scale of the Achievement Emotions Questionnaire (AEQ; Pekrun, Goetz, Frenzel, Barchfeld, & Perry, 2011) was used to measure students’ achievement emotions regarding the exam for their methodology course. The 25 items of the Test Emotions Scale measured the student’s academic emotions prior to taking the exam. The scale focuses on seven different emotions (enjoyment, hope, pride, anger, anxiety, shame and hopelessness) rated on a five-point Likert scale (1 = strongly disagree; 5 = strongly agree). An example of an item is: ‘I look forward to the exam’. The AEQ has been shown to be a reliable measure for students. For the subscales of the Test Emotions
12 Scale Cronbach’s alpha values were found to be between α = .78 and α = .92 by Pekrun et al. (2011). The AEQ has also been shown to be a valid measure (Pekrun et al., 2011).
The English version of the AEQ was used because to the present date no Dutch version of the scale is available. To check to what extent students had run into problems understanding the questions of the AEQ students were asked to indicate how many questions they had been unable to answer because they did not understand the English of these questions. As was described for the AMS, no significant problems with the students’ comprehension of the English language were expected because at Dutch universities students are accustomed to reading English.
Academic achievement
Academic achievement was measured as the students’ final exam scores. The final exam scores of both Research Design and Applied Methodology and Statistics were available. The final exam scores of Research Design were available as the number of correctly answered questions. The final exam scores of Applied Methodology and Statistics were, however, scaled between 1 and 10, indicating the final grade according to the formula: 𝑔𝑟𝑎𝑑𝑒 =𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑐𝑜𝑟𝑟𝑒𝑐𝑡 𝑎𝑛𝑠𝑤𝑒𝑟𝑠
𝑡𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑞𝑢𝑒𝑠𝑡𝑖𝑜𝑛𝑠 ∗ 9 + 1 . Also, the final exam scores of both courses were standardized using the mean and standard deviation of the corresponding course.
Procedure
The students in Research Design were asked by their teacher to fill out the questionnaire of the present study in their free time. The students were explained that they could fill out the
questionnaire by clicking a link in the menu of their online study environment. Two weeks later the students of Research Design that had not completed the questionnaire yet, as well as all students from Applied Methodology and Statistics, were asked to fill out the questionnaire of the present study during the break of one of their classes. In exchange for their participation the students were offered biscuits. Also, they were informed that the questionnaire would be open for approximately one week. After that period participation was not possible anymore.
The participants were randomly assign to two different versions of the questionnaire. The students in the BSA-condition were prompted a short text explaining the formal consequences of a negative study advice at their university. Subsequently, these students were instructed to imagine that they needed the study credits of their current methodology course in order to obtain a positive BSA. Afterwards, they were instructed to imagine that they were in the examination hall waiting for the exam to start. Directly following, students were asked to type at least five sentences about ‘what went through their minds’ as they waited for the exam to start. Also the students in the non-BSA condition received instructions to imagine that they were about to take the exam and type at least five sentences about what went through their minds. However, these students did not receive the informative text on
13 BSA-policy and they were not instructed to imagine that they needed the course’s credits to obtain a positive study advice. All students then proceeded to the AEQ and the AMS, respectively. At the end of both these questionnaires students were asked to indicate how many questions in the AEQ and AMS they were unable to answer due to a difficulty in understanding the English in the questions. Subsequently, students were asked to indicate whether they were already certain of a positive or negative study advice. Lastly, students were explained that it was important for the experiment that the content of the questionnaire would not be discussed with other students as long as the
questionnaire was open for participation. Students were asked to agree not to talk about the questionnaire by clicking on a ‘I agree’ button on the last screen of the questionnaire.
Analysis
Preliminary analyses
It was investigated whether the participants who started the questionnaire also had completed it. Students who had not completed the questionnaire were excluded from the sample, because their data was considered unusable. Furthermore, the data set was investigated for outliers.
Prior to testing the hypotheses of the present study, it was investigated whether the
experimental manipulation had been successful. During the experiment students were instructed to type at least five sentences about ‘what went through their minds’ as they imagined waiting for their exam to start. The participant’s answers were coded as ‘contemplating study advice’ or not,
depending on whether the answers related to BSA-policy or its consequences. Subsequently, for each student the proportion of answers coded ‘contemplating study advice’ was calculated. Using an independent t-test it was tested whether the proportions were higher in the BSA-condition, than in the non-BSA condition.
Furthermore, it was calculated how many students in the sample were already certain of a positive or negative study advice at the time of the experiment. Also the number of students that were not certain of their study advice was calculated.
Selectivity bias
To test whether the study’s sample differed from the rest of the students in the courses Research Design and Applied Methodology and Statistics, a number of analyses were ran. A chi-square test was used to test for independence between students’ participation in a number of non-mandatory midterms and their participation in the present study. Also, t-tests were used to compare final exam scores of the response and the non-response group.
Scaling
The reliability of the subscales of the AEQ and the AMS was determined by calculating their Cronbach’s alpha values. For two-item scales Spearman-Brown coefficients were calculated, instead
14 of Cronbach’s alpha. The dimensionality of the 25 items of the AEQ was investigated with principal component analysis with Varimax rotation. Bartlett’s test of sphericity and Kaiser-Meyer-Olkin measure of sampling adequacy were performed.
Confirmatory analyses
To test whether BSA-policy increases students’ extrinsic emotions an independent t-test was used. Subsequently, effect size was measured using Cohen’s d. Linear regression analysis was used to test whether extrinsic motivation positively covariates with the standardized final exams scores of all students. Linear regression analyses were also used to test whether extrinsic motivation positively covariates with the final exams scores of the two separate methodology courses.
Exploratory analyses
PROCESS (Hayes, 2012) was used to investigate the mediation effect of achievement
emotions on the effect of BSA-policy on extrinsic motivation. More specifically, the mediation effects of activating positive, activating negative, deactivating positive and deactivating negative emotions were investigated in a four separate mediation models (see Figure 2 on p. 10; Preacher & Hayes, 2004). Using the same PROCESS-models it was investigated whether activating positive and deactivating negative emotions covaried with extrinsic motivation. Also the direction of the covariation was investigated. Additionally, it was investigated with two separate linear regression models whether activating positive and deactivating negative emotions predicted intrinsic motivation.
Results
Preliminary analyses
In total 74 students were recruited to participate in the present study. After excluding students who had not finished the questionnaire 42 students remained. No outliers were found in the remaining data. The mean age of the reduced sample was 21.1 (SD = 2.2). Furthermore, the sample consisted of 37 females and five males. This unbalanced distribution of gender was expected since the programs at the Department of Child Development and Education of the University of Amsterdam are known for their small number of male students.
The experimental manipulation was found to be successful. Using an independent t-test, students in the BSA-condition (N = 19, M = .47, SD = 0.30) were found to have a higher average proportion of BSA-coded answers (see preliminary analyses on p. 13), than the students in the non-BSA-condition (N = 23, M = 0.04, SD = 0.10) at a p<.05-level: t(20) = 6.02, p = .00, Cohen’s d = 1.923. Levene’s test for equality of variances indicated unequal variance: F = 14.28, p = .00, so the degrees of freedom were adjusted from 39 to 20.
15 Furthermore, it was investigated how many students in the sample were already certain of a positive or negative study advice at the time of the experiment. Thirty-six students were certain of a positive study advice, while none were certain of a negative advice. Six students were not certain yet.
Selectivity bias
Prior to testing the hypotheses of the present study, it was evaluated whether the students who participated in the experiment differed from students who chose not to participate. This was evaluated using two available measures: students’ participation in non-mandatory midterms and their final exam scores. For the evaluation of the participation in the non-mandatory midterm test, only the data of the course Research Design was available (response group: N = 28; nonresponse group: N = 43). The independence of the participation in the non-mandatory midterms and the participation in the present study was tested using a chi-square test. The percentage of participation in the midterms was higher in the response group (75.0%), than in the nonresponse group (48.8%). The chi-square test indicated a statistically significant dependence between the participation in the midterms and the participation in the present study at a p<.05-level, X2 (1, N = 71) = 4.80, p = .03, Cramer’s V = .260. Secondly, it was tested whether the students who participated in the study scored differently on their final exam, than the students had not participated. For this analysis also the data of Applied Methodology and Statistics were available (response group: N = 13; nonresponse group: N = 138). The response (N = 41, M = 0.24, SD = 0.98) and nonresponse group (N = 179, M = -0.06 , SD = 1.00) were not found to have significantly different mean standardized final grades, at a p<.5-level: t(218) = 1.738 , p = .08, Cohen’s d = 0.303. T-tests were also performed for the two methodology courses separately. In Research Design the difference in the number of correctly answered questions between the response (M = 12.89, SD = 2.44) and the non-response (M = 12.43, SD = 2.95) was not found to be
statistically significant: t(68) = 0.690 , p = .49, Cohen’s d = 0.170. In Applied Methodology and Statistics, however, the difference (response: M = 5.97, SD = 1.73; non-response: M = 5.06, SD = 1.48) was found to be statistically significant at a p<.5-level: t(148) = 2.085 , p = .04, Cohen’s d = 0.564. For the chi-square test, and all three the t-tests, the students who were enrolled in the courses but did not participate in the exam were excluded from the analyses. They were assumed to not have actively participated in their course.
Scaling
The reliability of both the AEQ and the AMS was studied using Cronbach’s alpha
coefficients. Only the anger subscale of the AEQ was calculated with Spearman-Brown coefficient, because it consisted of two items. The subscales of the AEQ for enjoyment and anger were found to have poor reliability (< .600; see Table 2). The subscales for hopelessness, hope and anxiety were found to have acceptable reliability (α > .750; see Table 2). The coefficients for shame and pride were not calculated because they were only measured with one item each. Pekrun (2011) found the
16 reliability of the full versions of all scales to be acceptable (N = 389, α > 75; see Table 3).The
reliability of the AMS for extrinsic and intrinsic motivation was found to be high (extrinsic
motivation: number of items = 12, α = .834; intrinsic motivation: number of items = 12, α = .903) . Vallerand et al. (1992) also found the reliability of the extrinsic (N = 745, α = 76) and intrinsic (N = 745, α = .85) motivation scales of the AMS to be acceptable.
To assess the dimensionality of the 25 items of the AEQ, principal component analysis with Varimax rotation was utilized. To assess whether the 25 items grouped conform the seven subscales of the AEQ (enjoyment, anxiety, anger, hope, hopelessness, shame, pride), seven components were requested. Bartletss’s test of sphericity was found to be significant (p = .00), indicating proper sphericity. The Kaiser-Meyer-Olkin measure of sampling adequacy indicated an acceptable sampling adequacy (KMO = .668). The 25 items were, overall, not found to group well according to their subscales (see Table 4 on p. 17). For example, the five items for hopelessness grouped relatively well, with a minimal and maximum loading on component 1 of respectively .35 and .87. However, three of the five items of anxiety loaded moderately strongly on the first component as well (> .40).
Table 2
Reliability of AEQ subscales
AEQ subscale Number of items α
Hopelessness 5 .83
Enjoyment 5 .59
Anxiety 5 .81
Hope 5 .79
Anger 2 .52 (Spearman-Brown coefficient)
Table 3
Reliability of AEQ subscales in a study by Pekrun (2011) with N = 389
AEQ subscale Number of items α
Hopelessness 11 .92 Enjoyment 10 .78 Anxiety 12 .90 Hope 8 .80 Anger 10 .86 Pride 10 .86 Shame 10 .87
17 Table 4
Principal component analysis of the 25 items of the AEQ
AEQ subscale Component 1 Component 2 Component 3 Component 4 Component 5 Component 6 Component 7 AEQ 17 hopelessness .87 .07 .18 .03 .07 -.07 .14 AEQ 9 hopelessness .74 .18 .22 -.21 -.15 -.20 -.24 AEQ 13 hopelessness .55 .24 .37 -.17 -.31 .26 -.09 AEQ 6 hopelessness .42 .48 .50 -.04 -.10 -.35 -.08 AEQ 23 hopelessness .35 .73 .14 -.08 .09 -.15 .01 AEQ 4 anger -.09 .69 .04 .50 -.13 -.10 -.10 AEQ 12 anger .22 .68 .23 -.22 -.13 .11 -.24 AEQ 15 anxiety .54 .11 .14 .33 -.60 -.02 -.06 AEQ 19 anxiety .50 .29 .26 .37 -.43 -.12 .13 AEQ 10 anxiety .41 .11 .64 .11 .01 -.06 -.11 AEQ 2 anxiety .16 .53 .53 -.12 -.28 -.13 .21 AEQ 24 anxiety .09 .34 .69 .36 -.17 .02 -.10 AEQ 14 enjoyment .09 -.16 .04 .23 -.01 -.10 .84 AEQ 22 enjoyment -.08 -.01 .23 .79 .11 -.14 .05 AEQ 8 enjoyment -.20 -.23 -.16 .63 .27 .37 .06 AEQ 18 enjoyment -.52 .03 -.32 .16 .38 .45 .23 AEQ 1 enjoyment -.29 .03 -.25 -.26 .07 .25 .73 AEQ 7 shame .17 .10 .02 .10 .03 -.86 .04 AEQ 5 pride -.10 -.05 -.09 .29 .87 .03 .02 AEQ 25 hope -.34 .29 -.34 .36 .30 .40 .28 AEQ 3 hope -.55 -.15 .27 .33 .03 -.25 .32 AEQ 20 hope -.56 -.16 .06 .18 .40 .44 .11 AEQ 11 hope -.58 -.19 -.27 -.16 .22 .38 .25 AEQ 16 hope -.59 -.17 -.30 .21 .22 .28 -.22 AEQ 21 hope -.75 -.15 -.14 .12 .27 .31 .09 Eigenvalues 9.14 2.80 1.84 1.63 1.21 0.92 0.91 % explained variance 36.58 11.19 7.36 6.51 5.14 4.83 3.67 Confirmatory analyses
The effect of BSA-policy on students’ extrinsic motivation was tested using a t-test. The students in the BSA-condition (M = 3.56, SD = .53) were not found to score higher on extrinsic motivation, than the students in the non-BSA-condition (M = 3.35, SD = 0.78) at a p<.5-level: t(40) = 1.01, p = .32, Cohen’s d = 0.319.
18 Linear regression was used to test whether extrinsic motivation and achievement positively covariated. Extrinsic motivation was not found to predict the standardized final exam scores of the students from both methodology courses: b = 0.02, t(39) = 0.087 , p = .93. Extrinsic motivation was also not found to explain a statistically significant proportion of the variance in achievement scores:
R2 = .00, F(39) = 0.008 , p = .93. Also, in the two courses separately no covariation was found (Research Design: b = 0.19, t(26) = 0.281 , p = .78; Applied Methodology and Statistics: b = -0.26,
t(11) = 0.304 , p = .77). In both courses extrinsic motivation did not explain a statistically significant
proportion of the final exam scores (Research Design: R2 = .00, F(39) = 0.079 , p = .78; Applied Methodology and Statistics: R2 = .01, F(11) = 0.092 , p = .77). Lastly, linear regression was used to test whether extrinsic motivation positively predicted standardized final exam scores, while
controlling for students’ intrinsic motivation. No covariation was found: b = -0.01, t(39) = 0.041 , p = .97. Therefore, it was concluded that extrinsic motivation does not positively covariate with
achievement.
Exploratory analyses
PROCESS (Hayes, 2012) was used to investigate the mediation effect of achievement emotions on the effect of BSA-policy on extrinsic motivation. Four separate mediation models were used. Activating positive, activating negative, deactivating positive or deactivating negative emotion were set as the mediators in these four models (see Figure 2 on p. 10). The seven measured
achievement emotions were assigned to these four categories as is shown in Table 5. This was done according to a subdivision that is commonly made in literature on the valence and activation of achievement emotions (e.g. Pekrun et al., 2002). All four categories were not found to mediate the effect of BSA-policy on motivation (see Figure 3 on p. 19).
Table 5
The measured achievement emotions categorized according to their activation and valence
Categorization Achievement emotions
Activating positive Enjoyment, Hope
Activating negative Anxiety, Shame
Deactivating positive Pride
19 0.21 (0.21) 0.06 0.08 Activating positive emotions Extrinsic motivation BSA-policy 0.23 (0.21) -0.32 0.04 Activating negative emotions Extrinsic motivation BSA-policy 0.18 (0.21) 0.33 0.10 Deactivating positive emotions Extrinsic motivation BSA-policy 0.26 (0.21) -0.33 0.13 Deactivating negative emotions Extrinsic motivation BSA-policy
Figure 3. Regression coefficients in four mediation models
A
B
C
D
20 Discussion
The goal of the present study was to investigate the effect of BSA-policy on students’ extrinsic motivation and achievement emotions by the use of an experimental research design. It was expected that BSA-policy increases extrinsic motivation. BSA-policy was, however, not found to affect extrinsic motivation. It was also explored whether achievement emotions mediate the effect of BSA-policy on extrinsic motivation. This was investigated for activating positive, activating negative, deactivating positive and deactivating negative emotions. None of these categories were found to play a mediating role. Literature (Pekrun et al., 2011) has suggested that activating positive emotions positively covariate with motivation. In the present study activating positive emotions did not predict extrinsic motivation. Intrinsic motivation, however, was positively predicted by activating positive emotions. Literature (Pekrun et al., 2011) has also suggested that deactivating negative emotions negatively covariate with motivation. In the present study, however, deactivating negative emotions did not predict extrinsic or intrinsic motivation. Furthermore, it was expected that extrinsic motivation positively covariates with achievement. No covariation was, however, found.
A number of reasons can be put forward for the fact that BSA-policy was not found to affect students’ extrinsic motivation, and that achievement emotions were not found to mediate the effect of BSA-policy on extrinsic motivation. In other studies on achievement emotions (Pekrun et al., 2002; Turner & Schallert, 2001) it is often argued that, due to large individual differences in the effect of contextual factors on achievement emotions, small effects on a group-level are found. These individual differences in achievement emotions are possibly explained by the different ways
individuals appraise the features of their context. It is likely that the same argument is also applicable to BSA-policy. A negative study advice will have substantial negative consequences for one student, while for another it hardly does. Some students will be able to coop with the pressure of having to perform, better than others. It is likely that these differences relate to different styles of appraising BSA-policy and the consequences of a negative study advice. These differences in appraisals will result in different emotional experiences, as is also proposes in control-value theory (Pekrun, 2006). Moreover, theories on motivation, like Self-Determination Theory (Deci & Ryan, 2008) and
Expectancy-Value Theory of Achievement Motivation (Wigfield & Eccles, 2000), propose that extrinsic motivation is also affected by the interpretation of contextual features. In the present study no exceptionally large variance in achievement emotions and extrinsic motivation weas, however, found in the BSA-condition, compared to the non-BSA condition. This might, however, be explained by a lack of effectiveness of the experimental manipulation.
Although research suggests that BSA-policy does not have a large effect on motivation on a group-level, it is also possible that a substantial proportion of the student population does not contemplate the consequences of a negative study advice. This would also explain why no effect of BSA-policy was presently found. However, this scenario is unlikely. Negative study advice is
21 designed to have important negative consequences for students. Considering the probable negative consequences for a student’s finances and career, negative study advice is likely to be contemplated. To test whether this is the case, more research is necessary.
To detect small effects, large statistical power is needed. The present study’s relatively small sample (N = 42) did, however, not allow for a large power. Consequently, it was unlikely for any existing effects in the population to have been found in the present sample. It is advisable for future investigators to recruit more participants to increase power.
It is also possible that the experimental manipulation was not powerful enough to create detectable differences in the BSA and the non-BSA condition. In the present study’s sample
participants were, in most cases, guaranteed of a positive study advice, because they were at the end of their first year or already in their second year. For this reason, the participants in the BSA-condition were asked to imagine that their study advice was not certain yet and depended on their result on the final exam of their methodology course. To yield more power, future research could replicate the present study’s experiment with a sample of first-year students at the beginning of their year. These students are not certain of their study advice yet. It is, therefore, likely that they would contemplate the negative consequences of a possible negative study advice more strongly, than students who are certain of a positive study advice. Achievement emotions and extrinsic motivation would,
consequently, be effected more strongly.
Another explanation for the fact that achievement emotions were not found to be effected by the manipulation, is the quality of the measure of achievement emotions. Enjoyment and anger were found to have poor reliability, and pride and shame were measured with only one item per subscale. Moreover, principal component analysis revealed that the 25 items of the AEQ did not structure well according to the seven subscales of the AEQ. For future research better measures of achievement emotions are advisable.
A possible explanation for the fact that extrinsic motivation was not found to positively covariate with students’ academic achievement, is that the positive effect of extrinsic motivation on learning behavior is overestimated in theory. Numerous studies have found extrinsic motivation to not have a positive effect on academic achievement. However, most research does find a positive effect. Additional research is needed to better explain why extrinsic motivation benefits academic
achievement in some instances, while it does not in other.
It is difficult to compare the present study’s findings on BSA-policy with the existing
research on BSA-policy. To the knowledge of the author, no research has been done into the effect of BSA-policy on extrinsic motivation specifically. One study on the effect of BSA-policy on students’ self-study time, however, did not find an effect (De Koning, Loyens, Rikers, Smeets, & Van der Molen, 2014). Self-study time is a variable that is closely related to motivation (Doumen, Broeckmans, & Masui, 2014). Moreover, research on BSA-policy and achievement has, in most cases, found no or small positive effects of BSA-policy on academic achievement (De Koning,
22 Loyens, Rikers, Smeets, & Van der Molen, 2014; Arnold, 2014). If BSA-policy increases students’ motivation, it would be expected that their achievement would benefit. Since academic achievement is not found to be effected substantially, a large effect of BSA-policy on motivation is unlikely.
Extrinsic motivation is, however, only one form of motivation. It is conceivable that a decrease in another form of motivation, like intrinsic motivation, undoes the effect of a possible increase of extrinsic motivation. To test this, more research is necessary.
A strong side of the present study is the fact that it investigated the effect of BSA-policy on extrinsic motivation and achievement emotions, with an experimental research design. Most research into BSA-policy have made use of correlational or quasi-experimental designs. Although these designs can be highly valuable, they lack the ability to thoroughly examine the causal relations between variables. Since it is important to gain more knowledge about the causal effect of BSA-policy on motivation, it is necessary that existing research is supplemented with experimental research. Experimental manipulations in education are in many cases morally rejectable, since they require scientists to create inequality in a group of students. For example, students who would receive a negative study advice in one condition, would be able to progress to the second year in the other condition. In the present study it was found possible to design an experiment that investigated the effect of BSA-policy on students in a useful and morally acceptable way. Future research could supplement the investigation of BSA-policy with the use of experimental designs, by strengthening the present design or applying new ones.
Research on the effect of BSA-policy on students’ motivation has a number of implications for higher education. To improve the way students coop with BSA-policy, or similar policies, universities could offer students mentoring. Mentoring could target the way students appraise the possibility of a negative study advice. This could prevent students from experiencing disadvantageous emotions and lower motivation. Consequently, students study success and wellbeing would likely be benefitted.
Also, it is advisable to re-evaluate the design of policy. Although literature on BSA-policy is scarce, it suggests that the BSA-policy has not had the desired effect on students’ motivation and the efficiency of higher education. BSA-policy has, however, been found to bring dropout forward in time at universities (Arnold, 2014; Arnold & Van de Brink, 2010; Duijndam & Scheepers, 2009). This increases the efficiency of education at those universities. Nonetheless, the efficiency of the entire Dutch higher education is less likely to benefit, since slowly progressing students are not found to progress faster under BSA-policy (Arnold, 2014; Arnold, & Van de Brink, 2010). Education policy that is targeted at intrinsically motivating students might, therefore, be a better alternative for, or a advisable addition to, BSA-policy.
Overall, research offers higher education numerous opportunities for improving the efficiency of education and the motivation of its students. As research continues to build, more knowledge about the mechanisms of education will be gained, which will provide higher education with even more
23 opportunities to improve itself. Not only psycho-educational research plays a part in this. By using the knowledge of the different perspectives on education, like the economical or sociological perspective, more powerful and varied ways of developing education can be found. As research and practice continue to learn from each other, higher education is likely to steadily evolve into evermore adaptive forms.
24 References
Arnold, I. J. (2015). The effectiveness of academic dismissal policies in Dutch university education: an empirical investigation. Studies in Higher Education,40(6), 1068-1084.
Arnold, I., & Van den Brink, W. (2010). Naar een effectiever bindend studieadvies [Towards a more effective binding study advice]. Tijdschrift voor Hoger Onderwijs en Management, 5(10), 10-13.
Ayub, N. (2010). Effect of intrinsic and extrinsic motivation on academic performance. Pakistan
business review, 27(1), 363-372.
Boekaerts, M. (1994). Anger in relation to school learning. Learning and Instruction, 3(4), 269–280. doi:10.1016/0959-4752(93)90019-v
Busato, V. V., Prins, F. J., Elshout, J. J., & Hamaker, C. (2000). Intellectual ability, learning style, personality, achievement motivation and academic success of psychology students in higher education. Personality and Individual Differences, 29(6), 1057–1068. doi:10.1016/s0191-8869(99)00253-6
David, A. P. (2010). Examining the relationship of personality and burnout in college students: The role of academic motivation. Educational Measurement and Evaluation Review, 1, 90-104. de Koning, B. B., Loyens, S. M. M., Rikers, R. M. J. P., Smeets, G., & van der Molen, H. T. (2013). Impact of binding study advice on study behavior and pre-university education qualification factors in a problem-based psychology bachelor program. Studies in Higher
Education, 39(5), 835–847. doi:10.1080/03075079.2012.754857
Deci, E. L., & Ryan, R. M. (2008). Self-determination theory: A macrotheory of human motivation, development, and health. Canadian Psychology/Psychologie canadienne,49(3), 182–185. doi:10.1037/a0012801
Doumen, S., Broeckmans, J., & Masui, C. (2013). The role of self-study time in freshmen’s
achievement. Educational Psychology, 34(3), 385–402. doi:10.1080/01443410.2013.785063 Duijndam, F., & Scheepers, A. (2009). Bindend Studieadvies Erasmus Universiteit Rotterdam. HO
Management, 1(5), 17-19.
Ergene, T. (2011). The relationships among test anxiety study habits achievement motivation and academic performance among Turkish secondary school students. Egitim ve Bilim, 36(160), 320.
Fairchild, A. J., Horst, S. J., Finney, S. J., & Barron, K. E. (2005). Evaluating existing and new validity evidence for the academic motivation scale. Contemporary Educational
Psychology, 30(3), 331–358. doi:10.1016/j.cedpsych.2004.11.001
Fortier, M. S., Vallerand, R. J., & Guay, F. (1995). Academic motivation and school performance: Toward a structural model. Contemporary Educational Psychology, 20(3), 257–274. doi:10.1006/ceps.1995.1017
25 Hancock, D. R. (2001). Effects of test anxiety and evaluative threat on students’ achievement and
motivation. The Journal of Educational Research, 94(5), 284–290. doi:10.1080/00220670109598764
Hayes, A. F. (2012). PROCESS: A versatile computational tool for observed variable mediation, moderation, and conditional process modeling [White paper]. Retrieved from
http://www.afhayes.com/ public/process2012.pdf
Fong Lam, U., Chen, W. ., Zhang, J., & Liang, T. (2015). It feels good to learn where I belong: School belonging, academic emotions, and academic achievement in adolescents. School
Psychology International, 36(4), 393–409. doi:10.1177/0143034315589649
Meece, J. L., Anderman, E. M., & Anderman, L. H. (2006). Classroom goal structure, student motivation, and academic achievement. Annual Review of Psychology, 57(1), 487–503. doi:10.1146/annurev.psych.56.091103.070258
Ministry of Education, Culture and Science, (2014). Financiering van het hoger onderwijs: meer
flexibiliteit, meer keuzevrijheid, meer kwaliteit.[Financing high education: more flexibility,
more choice, more quality]. Utrecht: Ministry of Education, Culture and Science. Ministry of Education, Culture and Science, (2015). Selectie en toegankelijkheid van het hoger
onderwijs.[Selection and accessibility of higher education]. Utrecht: Ministry of Education,
Culture and Science.
Ministry of Education, Culture and Science, (2015). Selectie en toegankelijkheid van het hoger
onderwijs [Selection and Accessibility of Higher Education]. Utrecht: Ministry of
Education, Culture and Science.
Organization for Economic Cooperation and Development (OECD), 2008. Education at a glance
2008: OECD indicators. Paris: Centre for Educational Research and Innovation.
Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic emotions in students’ self-regulated learning and achievement: A program of qualitative and quantitative research.Educational
Psychologist, 37(2), 91–105. doi:10.1207/s15326985ep3702_4
Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review, 18(4), 315–341. doi:10.1007/s10648-006-9029-9
Pekrun, R., Elliot, A. J., & Maier, M. A. (2009). Achievement goals and achievement emotions: Testing a model of their joint relations with academic performance. Journal of Educational
Psychology, 101(1), 115–135. doi:10.1037/a0013383
Pekrun, R., Goetz, T., Frenzel, A. C., Barchfeld, P., & Perry, R. P. (2011). Measuring emotions in students’ learning and performance: The achievement emotions questionnaire
(AEQ). Contemporary Educational Psychology, 36(1), 36–48. doi:10.1016/j.cedpsych.2010.10.002
26 Preacher, K. J., & Hayes, A. F. (2004). SPSS and SAS procedures for estimating indirect effects in
simple mediation models. Behavior Research Methods, Instruments, & Computers, 36(4), 717–731. doi:10.3758/bf03206553
Reeve, J. (2002). Self-determination theory applied to educational settings. In E. L. Deci (Ed); R. M. Ryan (Ed). (2002). Handbook of self-determination research (pp. 183-203). Rochester, NY, US: University of Rochester Press.
ResearchNed (2009). Bindend studieadvies: rapportage kwantitatieve resultaten [Binding study advice: report of quantitative results]. Utrecht: Dutch Inspectorate of Education. Rochester, NY: Rochester University Press.
Stoeber, J., Feast, A. R., & Hayward, J. A. (2009). Self-oriented and socially prescribed
perfectionism: Differential relationships with intrinsic and extrinsic motivation and test anxiety. Personality and Individual Differences, 47(5), 423–428.
doi:10.1016/j.paid.2009.04.014
The Educational Council, 2008. Een succesvolle start in het hoger onderwijs.[A successful start in higher education]. The Hague: The Educational Council.
Thomaes, S., Bushman, B. J., Orobio de Castro, B., Cohen, G. L., & Denissen, J. J. A. (2009). Reducing Narcissistic aggression by Buttressing self-esteem: An experimental field study. Psychological Science, 20(12), 1536–1542. doi:10.1111/j.1467-9280.2009.02478.x
Turner, J. E., & Schallert, D. L. (2001). Expectancy–value relationships of shame reactions and shame resiliency. Journal of Educational Psychology,93(2), 320.
Vallerand, R. J. (1997). Toward a hierarchical model of intrinsic and extrinsic motivation. Advances
in experimental social psychology, 29, 271-360.
Vallerand, R. J., Pelletier, L. G., Blais, M. R., Briere, N. M., Senecal, C., & Vallieres, E. F. (1992). The academic motivation scale: A measure of intrinsic, Extrinsic, and Amotivation in education. Educational and Psychological Measurement, 52(4), 1003–1017.
doi:10.1177/0013164492052004025
Vallerand, R. J., Pelletier, L. G., Blais, M. R., Briere, N. M., Senecal, C., & Vallieres, E. F. (1993). On the assessment of intrinsic, Extrinsic, and Amotivation in education: Evidence on the concurrent and construct validity of the academic motivation scale. Educational and
Psychological Measurement, 53(1), 159–172. doi:10.1177/0013164493053001018
Van Den Berg, M. N., & Hofman, W. H. A. (2005). Student success in university education: A multi-measurement study of the impact of student and faculty factors on study progress.Higher
Education, 50(3), 413–446. doi:10.1007/s10734-004-6361-1
Wigfield, A., & Eccles, J. S. (2000). Expectancy–Value theory of achievement motivation.Contemporary Educational Psychology, 25(1), 68–81. doi:10.1006/ceps.1999.1015
