Exploring
data
use
practices
around
Europe:
Identifying
enablers
and
barriers
Kim
Schildkamp
*,
Louisa
Karbautzki,
Jan
Vanhoof
UniversityofTwente,FacultyofBehaviouralSciences,P.O.Box217,7500AEEnschede,TheNetherlands
Introductionandtheoreticalframework
Data-baseddecisionmakingisreceivingincreasedattentionin
countriesaroundtheworld.Animportantreasonforthisisthat
somestudieshavefoundthateffectivedatausebyteachersand
school leaders can lead to school improvement in terms of
increasedstudentachievement(Campbell&Levin,2009;Carlson,
Borman, &Robinson, 2011; Lai, McNaughton, Amituanai-Toloa, Turner,&Hsiao,2009).Datacanbedefinedas‘‘informationthatis
collectedandrepresentssomeaspectofschools’’(Schildkamp,Lai,
&Earl,2013,p.10).
Schoolshaveaccesstomultipledata sources:input,process,
context and output data (Ikemoto & Marsh, 2007). Input data
includes, for example, data suchas demographics of students.
Process data refers to data such as data on the quality of
instruction.Contextdatareferstodataonpolicyandresources.
Output data includes data such as student achievement data
(Ikemoto&Marsh,2007).
These data can be used for decision making for school
improvement.Teachersandschoolleaderscanusedata,suchas
assessment and survey data, for different purposes: school
development purposes (e.g. policy development), instructional
purposes(e.g.instructionalchanges,suchasadaptinginstruction
totheneedsof thestudents),and accountabilitypurposes (e.g.
communicatingresultstoparents)(Breiter&Light,2006;Coburn&
Talbert,2006a;Diamond&Spillane,2004;Schildkamp&Kuiper,
2010;Schildkamp,Lai,etal.,2013;Wayman&Stringfield,2006; Wohlstetter,Datnow,&Park,2008;Young,2006).
Furthermore,differentdatausestudies(e.g.Coburn&Turner,
2011; Schildkamp & Lai, 2013; Supovitz, 2010) show that the
processofdatauseisinfluencedbyseveralfactorsthancaneither
enabledatauseorformabarriertowardeffectivedatause.Firstly,
datauseisenabledorconstrainedbycertainschoolorganization
and contextconditions. Organizationalstructures willinfluence
whatdataareusedinaschoolandforwhichpurposes.Aschool
leader can, for example, determine which data teachers have
accessto,theycansupportteachersintheuseofdatabymeansof
facilitatingthemintime,byputtingstructuresfordatauseinplace,
andbymodelingeffectivedatause.Furthermore,itisimportant
that there is a shared vision in the organization, and that
measurablegoalsexistatschool,classroom,andstudentlevel.If
therearenocleargoalsitisdifficulttousedata,becausethereare
no goalstocomparethedatato.Moreover,ifa schoolprovides
teacherswithopportunitiestocollaboratearoundtheuseofdata
this can lead to more effective data use as well (Schildkamp,
Poortman, & Handelzalts, 2013). The same goes for providing
teacherswithtrainingandsupportintheuseofdata(Coburn&
Turner, 2011; Datnow, Park, & Kennedy-Lewis, 2013; Honig & Venkateswaran, 2012; Jimerson & Wayman, 2012; Levin & Datnow,2012;Mandinach&Honey,2008;Marsh,2012; Schild-kamp &Kuiper, 2010;Spillane, 2012; Supovitz,2010;Vanhoof, Verhaeghe,VanPetegem,&Valcke,2011;Wayman,Spring,Lemke, &Lehr,2012;Wayman,Jimerson,&Cho,2012;Wohlstetteret al., 2008;Young,2006).
Secondly,characteristicsofdataanddatasystemscaninfluence
whetherdataareusedforschooldevelopmentaccountabilityand
ARTICLE INFO
Articlehistory:
Received25March2013
Receivedinrevisedform7October2013 Accepted8October2013
Availableonline14November2013 Keywords:
Data-baseddecisionmaking Schoolimprovement Professionaldevelopment
ABSTRACT
Inthisarticleweexplorewhatdata-baseddecisionmakinguselookslikeinschoolsinfivedifferent countries(UnitedKingdom,Germany,Poland,LithuaniaandtheNetherlands).Weexploreforwhat purposesdataareusedinthesecountriesandwhattheenablersandbarrierstodatauseare.Thecase studyresultsshowthatschoolsinallfivecountriesusedataforschooldevelopment,accountability,and instructionalimprovement. Also,theschools inthefive countriesstruggle withthesame typeof problems:e.g.lackofaccesstohighqualitydata,lackofprofessionaldevelopmentinusingdata,anda lackofcollaborationaroundtheuseofdata.Finally,wediscusshowsomeenablerscanturnintobarriers foreffectivedatause.
ß2013ElsevierLtd.Allrightsreserved.
* Correspondingauthor.Tel.:+310534894203.
E-mailaddress:k.schildkamp@utwente.nl(K.Schildkamp).
ContentslistsavailableatScienceDirect
Studies
in
Educational
Evaluation
j ou rna l h om e pa ge : w w w. e l s e v i e r. co m/ s tue duc
0191-491X/$–seefrontmatterß2013ElsevierLtd.Allrightsreserved.
instructional purposes. Schools that have good functioning
informationmanagementsystemsandaccesstorelevant,reliable
andvaliddataaremorelikelytoshowincreasedlevelofdatause.
Datauseislikelytobeconstrainedifteachershavedifficultiesin
accessingthedatatheyneed,oriftheyfeelthatthereareproblems
withthequalityofthedata(Breiter&Light,2006;Cho&Wayman,
2013; Coburn & Turner, 2011; Schildkamp & Kuiper, 2010; Wayman&Stringfield,2006;Wohlstetteretal.,2008).
Moreover,datauseisalsolargelydependentoncharacteristics
oftheuser.Schoolstaffismadeupofindividualpeople.Someof
themmighthavethenecessaryknowledge,skillsandattitudeto
usedata,whereasothersmaynot.Severalstudiestalkaboutthe
importanceofdataliteracy.Ittakescertainknowledgeandskillsto
analyze,interpretandtake actionbasedon data.Therefore,itis
importanttoalsolookatfactorsattheindividualdatauserlevel
(Coburn&Talbert,2006b;Earl&Katz,2006;Jimerson&Wayman, 2012;Little,2012;Wohlstetteretal.,2008;Young,2006).
Theuseofdatamayleadtoaneffectonteacher-,schoolleader-,
andstudentlearning.Forexample,basedonassessmentresultsin
combination with classroom observation results, teachers can
identifytheneedsofstudents(teacherlearning)andaddresstheir
instruction accordingly. This may lead to increased student
learning and increasedstudent achievement(Boudett &Steele,
2007).Animportantquestionthatiscurrentlylargelyunanswered,
however,iswhattypesofdataareusedandhowthesedataare
being used or not used. A related question is which factors
influencethepracticeofdatause,asstudiesshowthatthereare
distinct differences in the way schools use (or not use) data,
differences between schools in different countries, but also
differencesbetweenschoolswithinonecountry.
Therefore,thisarticleaddressesthreecentralquestionsinthe
context of five different countries (United Kingdom, Germany,
Poland,LithuaniaandtheNetherlands):
1.Whatdataareusedbyschoolsinthedifferentcountries?
2.Forwhichpurposesdoschoolleadersandteachersusedatain
thesecountries?
3.Which organizational, data and data systems and user
characteristicsinfluencetheuseofdata?
Researchcontextandmethodology
Contextdescription
Five countrieswereinvestigatedinthisstudy:Germany,The
Netherlands, United Kingdom, Poland, and Lithuania (see also
www.datauseproject.eu).Inthissection,wewillbrieflydescribe
the policy context (in terms of autonomy, accountability, the
curriculumanddataavailable)ofeachcountry.
Germanyhas16differentstatesandeachstateisresponsiblefor
providing education. The federal Ministry is mainly concerned
witheducation research, and educational planning. Within the
states,schoolsarecentrallyorganizedandverylimitedautonomy
exists for schools. Decisions are mostly taken at the state,
provincial/regionallevelandlocallevel(OECD,2008,2010).Only
withregardtoorganizationofinstructiontheschoolhasautonomy
regarding decision making. The state designs and selects the
programsthat areofferedand determinestherangeofsubjects
taught and the course content (OECD, 2008). Germany has a
standard curriculum or partly standardized curriculum that is
required, as well as mandatory national examinations and
assessments(OECD, 2010). Standardsareassessed bymeansof
state-widecentraltestsin9th/10thgrade, aswellasfor Abitur
(12th/13th grade).Additionally, independent state-widecentral
assessmentsare conducted in K-1, 3rdand 8th grade. Internal
evaluations are not compulsory, but school boards and other
organizationsoffertoolsandsupport.
SchoolsintheUnitedKingdomhavealotofautonomy.Almost
alldecisionsaremadeattheleveloftheschool(OECD,2008,2010).
Schoolsdecidewhichtextbookstheywanttouse,theyselectthe
programs that they will offer, decide on therange of subjects
taughtand thecoursecontentof thesesubjects(althoughthey
havetorefertoaframeworkatthecentrallevel)(OECD,2008).The
UnitedKingdomdoeshaveastandardcurriculum(OECD,2010).
Therearenationalassessmentsrequiredofallstateschoolsinthe
United Kingdom for all students of certain ages, and although
nationalexaminationsarenotcompulsory,onlyrarelydostudents
not take core subjectsas these are neededfor the majority of
subsequenttraining, educationandemployment needs.Schools
are inspected by Ofsted, whoprovides schools withinspection
reports.Internalevaluationsusinglessonobservation,perception
questionnaires, attainment and achievement data are highly
recommended. These evaluations are most frequently based
around the Ofsted inspection framework. Inspections from
external evaluation agencies are optional. Schools are likely to
feelpressuredtousedataastheyareevaluatedbyOfstedandtheir
performancewillappearinLeaguetables(Downey&Kelly,2013).
Also, theUnitedKingdomhasa nationalstudentdatabase,and
achievement and attainment tables, which makes information
availableinasystematicandaccessiblemanner.
In Lithuania, the Ministry of Education is responsible for
developingeducationalpolicy,approvingofthegeneralcontentof
teaching,organizingthefinalexaminations,anddeterminingthe
national standards for attained education level. The County’s
Manager’s Administration implements the national education
policyinthecounty,approveseducationplansforthecounty,and
supervises the education providers. Municipalities execute the
nationaleducationpolicyinthemunicipality,approveeducation
plans,andensurethecontextnecessaryforprovidingeducation.
Alsoschoolsensuretheexecutionofthenationaleducationpolicy.
Attheendof secondaryeducation,studentsparticipate infinal
examinations(e.g.matureexams)atschoolleveland/oratnational
level (Ministry of Education and Science of the Republic of
Lithuania, 2004). Schools are evaluated both externally and
internally.External evaluationsare carried out by theNational
AgencyforSchoolAssessment.Internalevaluationsareobligedas
well.Schoolscanusetheinternalauditmethodologydevelopedby
the National Agency for internal evaluation or use their own
system. Internal evaluations are carried out by the school
administrationincooperationwithteachers.
AnimportantactinPolandisthePedagogicalSupervisionAct
passed in 2009, which lists three areas of school supervision:
evaluation, control and support. The act provides also the
requirements according to which all schools in Poland are
externallyevaluatedbyeducationalauthorities. TheMinistryof
NationalEducationprovides curriculumstandards,districtsand
municipalitiescontroladministrationandfinancing,schoolleaders
choosewhichteacherstohireandteacherschooseacurriculum
fromapre-approvedlist.Schoolleadershaveautonomy
concern-ing hiring teachers, approving programs and textbooks, and
conductinginternalevaluations.Polandhasmandatorynational
examinationsandassessmentscoordinatedandimplementedby
theCentralandRegionalExaminationCommissions(OECD,2010),
for examplethe6th (primaryeducation), 9th(lowersecondary
education), and 12th grade (upper secondary education) exit
exams. Schools are both (in theory) internally and externally
evaluated.However,sincetheActonPedagogicalSupervisionisa
rathernewact,notallschoolshavebeenevaluatedexternally,nor
havetheyconductedinternalevaluations,yet.
IntheNetherlands,schoolshavealotofautonomy.Similartothe
school(OECD,2008,2010).Schoolsdecidewhichtextbooksthey
wanttouse,theyselecttheprogramsthattheywilloffer,decideon
the range of subjects taught and the course content of these
subjects(althoughtheyhavetorefertoaframeworkatthecentral
level)(OECD,2008). TheNetherlandsdoes not havea standard
curriculum that is required; they do have mandatory national
examinationsat theend of secondary education, but no other
mandatorynationalassessments(OECD,2010).However,schools
areheldaccountablefortheirfunctioningbytheDutch
Inspector-ate.Asschoolsareresponsibleforthequality ofeducationthey
provide,theyhavetoconductsomekindofschoolself-evaluation
tochecktheirqualityandimproveifnecessary.
Designandrespondents
Case studies were conducted in each of the five countries
participatingin this study. According toYin (1984,p. 23) case
studiesare:‘‘anempiricalinquirythatinvestigatesacontemporary
phenomenon withinits real-life context; when theboundaries
betweenphenomenonandcontextarenotclearlyevident;andin
whichmultiplesourcesofevidenceareused.Stake(2000)states
that a number of cases can be jointly studied toinvestigate a
phenomenon,inthisstudydatauseinfivedifferentcountries.
Ineachcountry,weaimedatstudyingdatauseinaminimumof
two secondaryeducationschools. We usedpurposive sampling
(Yin, 2009) toidentifyand selectschools and respondents. We
werelookingforschoolsonanadvancedlevelinthefieldof
data-baseddecisionmaking,comparedtootherschoolsintheircountry.
Theseschoolswereidentifiedbasedon(1)theprojectmembers’
previousresearchatthesesitesondatause,(2)nominationsfrom
professionalcontactsinthefieldofdatauseresearchanddatause
support,and(3)basedonpolicyandinspectiondocuments.This
resultedinalistofpossibleandwillingschoolsineachcountry(see
Table1).Theresultscanthusnotbegeneralizedtopopulation,but
theycanbegeneralizedtotheoreticalpropositions(Yin,2009).Our
aimwastogainmoreinsightsintodatauseandfactorsinfluencing
datauseindifferentcountries.
Interviewswereconducted withrespondentsof each of the
schools(seeTable1).Theserespondentsincludedschoolleaders,
andteachersnominatedbytheschoolleadersashavingexperience
withdatause.InGermany,sixteachersandsix(assistant)school
leadersoftwoschoolswereinterviewed.IntheNetherlands,11
teachers and 21 (assistant) school leaders of six schools were
interviewed. In Lithuania, 15 (assistant) school leaders of two
schools were interviewed.1 In Poland, 11 teachers and two
(assistant) school leaders of two schools were interviewed. In
the United Kingdom, six teachers and eight (assistant) school
leadersoffourschoolswereinterviewed.
Instruments
Fortheinterviewsweusedaninterviewscheduledevelopedby
Schildkampand Kuiper(2010) tostudytheuseofdata andits
influencing factors.Open questions were askedwith regard to
what datateachersandschoolleadershad available,whatdata
theyused,forwhichpurposesschoolleadersandteachersused
data,andwhichvariablespromotedandhinderedtheuseofdata.
The interviews started with an open question with regard to
currentschool-wideschoolimprovementinitiatives,whetheror
notdataplayedaroleintheseactivities,and,ifyes,how.Secondly,
respondentswereaskedwhetheror nottheyusedseveral data
sources, suchasassessment data,and forwhich purposesthey
usedthesedata.Finally,weaskedifrespondentscouldmention
factorsthat eitherenabled orhindered theuseof dataintheir
school. Interviewsapproximatelytookbetween 30 and 60min.
Also,documents(e.g.policyplans,literature,andOECDreports)
werecollectedtodescribethecontext(relatedtodatause)ineach
ofthecountries.ThedatafortheDutchcasestudieswerecollected
inapreviousstudy(Schildkamp&Kuiper,2010).
Dataanalysesandquality
We used a systematized approach to data collection and
analyses that is consistent withthe research questions (Riege,
2003).Inallcountriesthesameinterviewschedulewasusedand
the collected data were analyzed in the same manner. The
interviewdatawerecodedaccordingtoacommoncodingscheme.
Thecodingschemewasbasedonourtheoreticalframework.First,
we codedthedifferenttypesof datatherespondentsindicated
using.Weorganizedthesedatasourcesaroundinput,processand
context, and output data. Secondly, we looked for different
purposesfordatauseputforwardbytherespondents.Examples
ofcodes are:schooldevelopment/policydevelopment;
account-ability/communication with parents; instruction/instructional
changes. Next,wecoded theenablersand barriersfordatause
that the respondents mentioned. Examples of codes include:
organization/schoolleadersupport;dataanddatasystems/quality
ofthedata;user/knowledgeandskills.
Foreachschool,theprojectpartnersconductedthecodingand
filledout acase specifictemplate,includingthedatathatwere
usedbytheschools,theschoolleaderandteacherpurposefordata
use, thedataanddata systemscharacteristics,school
organiza-tionalcharacteristics,andusercharacteristicsinfluencingtheuse
ofdata.Foreachschool,onetemplatewasfilledout.Thisfacilitated
thecrosscaseanalyses,forwhichallthecasedescriptionswere
comparedandcontrastedwithinacountryandbetweencountries.
Furthermore,itenhancedvaliditybecauseitmadeitpossibleto
highlightmajorpatternsofsimilaritiesand differencesbetween
respondents, schools, and countries (Poortman & Schildkamp,
2011). For example, it enabled us to compare which factors
enabled orhinderedtheuseof datainallcountries,and which
factorsseemedtobemoreuniqueforacertaincountry.Thepolicy
andinspectiondocumentswereusedfordescribingthepolicyand
contextofeachoftheparticipatingcountries.Thefullresultsper
countrycanbefoundinthecasestudyreport(ComeniusProject
UsingDataforImprovingSchoolandStudentPerformance,2011).
Inthispaperwefocusedonthecrosscaseanalyses.
Results
Below,wedescribetheresultsofourcrosscaseanalyses.The
resultsoftheindividualcountriescanbefoundintheTables.In
Section‘Whatdataareusedbyschoolleadersandteachersinthe
differentcountries’wedescribewhatdatawereusedbytheschool
leadersandteachersinthedifferentcountries.Weorganizedthese
datain input,processand context,and outputdata.Inputdata
includesdatasuchasthedemographicsofthestudentpopulation.
Processdataincludesdataonthequalityofinstruction.Context
dataincludesdatasuchasdataonthecurriculum,material,and
Table1
Respondentspercountry.
Schools Schoolleaders Teachers
Germany 2 6 6 Netherlands 6 21 11 Lithuania 2 15 0 Poland 2 2 11 UnitedKingdom 4 8 6 Total 16 52 34 1
building.Output data refers to outcomes suchas student test
scoresandstudentwell-being(Ikemoto&Marsh,2007).InSection
‘Forwhichpurposesdo schoolleaders andteachersusedatain
differentcountries?’ we discuss for which purposes thesedata
sources wereused. This section is organized, according to our
theoreticalframework,inschooldevelopmentpurposes,
instruc-tionalpurposes,and accountabilitypurposes.InSection ‘Which
organizational, data and data systems, and usercharacteristics
influencetheuseofdata?’wepresenttheresultsofthedifferent
enablersandbarriersofdatause,organizedinschool
organiza-tionalcharacteristics,data and datasystem characteristics,and
usercharacteristics.
Whatdataareusedbyschoolleadersandteachersinthedifferent
countries
Table2liststhedatathataccordingtotherespondentswere
availableintheirschools.Schoolsinallcountrieshadawiderange
ofdataavailable.Allrespondentsmentionedtheavailabilityof
inputdata,suchasdemographicinformationonthestudentsin
theirschool,processandcontextdata(e.g.self-evaluationdata),
andoutputdata(e.g.achievementresults).However,thetypesof
data that were available in Germany differed per state: both
schools had student achievement results, one school had
inspection and self-evaluation results. English respondents
mentionedthewidestrangeofdatasourcesavailable,interms
ofinputdata,processdataandoutputdata,althoughthefocus
wasmostlyonachievementdata(e.g.valueaddedachievement
data,attainmentdata,progressdata).InLithuanianschools,there
seemedto be less data available, or at least the respondents
mentionedlessdatasources.AccordingtothePolishrespondents
thedata thatweremostlyused wereachievementdata. Polish
schoolshadelectronicdatasystemsinplaceandteacherscould
accessthesesystems tofinddata ontheirstudents. Schoolsin
Polandseemedtobeabitmorefocusedonoutputdata,whereas
intheNetherlandsawiderangeofprocessdatawerementioned
bytherespondents.
Forwhichpurposesdoschoolleadersandteachersusedataindifferent
countries?
Withregardtousingdataforschooldevelopment,respondents
inallfivecountriesindicatedthatdata,suchasdatafrominternal,
external evaluations and assessments, were used for policy
developmentandschoolimprovementplanning,andforteacher
development(seeTable3).However,severalrespondents
indicat-ed that data were used at a very superficial level. German
respondents indicated, for example, that a lot of data were
collected,butnotsystematicallyused.Dataweremostlyusedfor
administrativepurposes.Theresultsoftheinterviewsinthetwo
Lithuanianschoolsshowedthatschoolleaderswouldliketouse
data more extensively than they currently were. In the
Netherlands,several schoolleaders alsoadmittedthat datause
usuallydidnotmovebeyondthemonitoringandplanningstage.
ThiswasconfirmedbyDutchteachers,whoindicatedthatthey
werenot familiarwith theimprovementactions formulatedin
policyplansandthatthesewereoftennotimplemented.
InGermany,Poland,andtheUK,intake(assessment)datawere
usedforplacingandgroupingstudentsbasedontheirabilitiesand
needs.LithuanianandUKrespondentsalsospecificallymentioned
usingdata(e.g.assessmentdata,evaluationdata,Ofstedinspection
data)fortargetsetting.InthecaseofLithuanianschoolsthisreferred
totargetsettingatschoollevel,intheUKthisreferred totarget
setting for specific departments that were, according toOfsted,
underperforming.UKRespondentsindicatedthatdatawereusedto
motivatestaff,forexamplebycelebratingachievement.
OnlyintheUKitseemedthatschoolswereabletomovebeyond
asuperficiallevelofdatauseforschooldevelopment.Whenasked
for concrete examples of data use for policy and teacher
development,onlytheUKrespondentswereabletodeliverthese.
For example, they would describe how school leaders would
observe a teacher’s lesson and based on observation data,
combinedwithperformancedata,theywoulddiscusswhattypes
of improvements a teacher could make in the classroom. The
teacherwouldfollowuponthis,andthiswouldbeevaluatedagain.
Table2
Dataavailableineachofthecountriesaccordingtotherespondents.
GEa
UK LT PO NL
Inputdata
Studentintake/demographicdata X X X X X
Specialneedsdata X
Primaryschoolresults X X
Diagnosticentrancetest X
Processandcontextdata
Lessonobservations X
Externalevaluations X X X X X
Self-evaluation/internalevaluation X X X X X
Teacherperformancedata X
Staffdata,suchasattendance,hoursofwork,degrees,age X X
Timespendonsubjects X
Absentees/attendancedata X X X X
Transferdata X X
Studentbehavior X X
Schoolpolicyplansandinformation X X X X
Classmanagementinformation X
Exclusionrates X
Teenagepregnancy X
Staffsurveys X X
Studentsurveysand/orinterviews X X X
Parentsurveysand/orinterviews X X X
Outputdata
Schoolinspectionreport X X X
Assessment/achievementdata X X X X X
Schoolleavers X X X
Exitinterviews X
Graduatesurveys X
a
Respondentsinallfivecountriesindicatedthatdatawereused
forinstructionalpurposes(seeTable4).Inallcountries,student
progress was monitored based on student achievement data.
RespondentsfromGermany,theUK,PolandandtheNetherlands
mentioned that data were used for instructional changes.
However,again,respondentsfromtheUKwereabletodescribe
themostconcreteexamples(e.g.how valueaddedassessment
results were used to target instruction toward weak(er) and
strong(er)students).Germanteachersdidtalkaboutusingdatato
monitor progress of students and to determine the need for
individualstudentsupportorinstructionalchanges,however,this
was, according to the German respondents, not common in
German schools. In one of the Polish schools, respondents
mentionedusingdataforinstructionalchanges,suchasgrouping
of students, monitoring progress of individual students and
groupsofstudents,andadjustinglessonplansandgoalsaccording
to needs of students. Respondents from the other school
mentionedthatthiswasnotcommonpracticeinPolandeither.
IntheNetherlands,datauseforinstructionalpurposesdidnot
movebeyondmonitoringinmostschools.Onlyintwoschools,
teacherscouldnameconcreteexamplesofhowdatawereusedto
make instructional changes. For example, one teacher talked
about analyzing the assessment results of failing students,
discoveringthatthesestudentsallfailedspecifictopics,andthen
re-teachingthesetopics,butexplainingthesetopicsinadifferent
manner.
Also,respondentsinallfivecountriesmentionedusingdatafor
accountability purposes (see Table 5). Specifically, results of
Table3
Schooldevelopmentpurposesfordatausementionedbytherespondentspercountry.
GE UK LT PO NL
Policydevelopmentandschool improvementplanning(internal andexternalevaluations)
Policydevelopmentandschool improvementplanning(internal andexternalevaluations)
Policydevelopmentand schoolimprovement planning(internaland externalevaluations)
Policydevelopmentand schoolimprovement planning(internaland externalevaluations)
Policydevelopmentand schoolimprovement planning(internaland externalevaluations) Teacherdevelopment
(assessmentdata)
Teacherdevelopment,discuss andimproveteacherperformance (lessonobservations,performance dataandinternalinspections)
Teacherdevelopment (achievementand observationdata) Teacherdevelopment (assessmentdata) Teacherdevelopment, discussteacher performance(assessment data,intaketransferand schoolleaverdata) Groupingofstudentsand
placingstudents(intakedata)
Groupingofstudentsandplacing students(intakedata)
Groupingofstudents andplacingstudents (intakedata) Targetsettingfordepartments
(Ofstedreportsandachievement scores)
Targetsettingand monitoringgoals (assessmentdata, internalevaluations) Motivatingstaff(performancedata
andobservations)
Table4
Instructionalpurposesfordatausementionedbytherespondentspercountry.
GE UK LT PO NL
Monitoringprogress(studentdata) Monitoringprogress (studentdata) Monitoringprogress (studentdata) Monitoringprogress (studentdata) Monitoringprogress (studentdata) Instructionalchanges(assessment
andself-evaluation)
Instructionalchanges(assessment andself-evaluation,student voice,observations) Instructionalchanges (assessment andself-evaluation) Instructionalchanges (assessmentand self-evaluation) Curriculumdevelopment (assessmentdata) Curriculumdevelopment (intakedata)
Rewardandmotivatechildren (achievement
data)
Referself-evaluation resultstoInspection
Proofthattheschoolimproves (assessmentresults)
Referself-evaluation resultsto
Inspection
Table5
Accountabilitypurposesfordatausementionedbytherespondentspercountry.
GE UK LT PO NL
Communicationwith parents(studentdata)
Communicationwith parents(studentdata)
Communicationwith parents(studentdata)
Communicationwith parents(studentdata)
Communicationwith parents(studentdata) Publicrelations
(administrativedata)
Publicrelations(inspection results)
Communicationand
collaborationwithotherschools (achievement
andevaluationdata) Referself-evaluation
resultstoInspection
Proofthattheschoolimproves (assessmentresults)
Referself-evaluation resultstoInspection
students were communicated to parents (in all countries),
sometimessharedwithotherschools(UKandLithuania),andif
theresultsweregoodtheywereusedforpublicrelationpurposes
in Germany and the Netherlands (e.g. post high achievement
resultsontheschool’swebsiteorsharetheresultsinnewslettersto
thecommunity).Datawerealsousedtocomplywith
accountabil-ity demandsof an inspectorateor ministry.In theUKand the
Netherlands,forexample,schoolself-evaluationresultsweresent
totheinspectorate.
Whichorganizational,dataanddatasystems,andusercharacteristics
influencetheuseofdata?
Tables 6–8comparetheresultspercountryforthedifferent
influencingfactors(e.g.schoolorganizationalcharacteristics,data
anddatasystemcharacteristics,andusercharacteristics).In the
tables,itisindicatedwhetherafactorenabled(+)orhindered( )
theuseofdataaccordingtotherespondents.Forexample,some
respondentsintheNetherlandsindicatedthattheyfeltsupported
by their colleagues in the use of data (+), whereas other
respondentsintheNetherlandsfeltnotsupportedintheuseof
databytheircolleagues( ).Ifafactorisnotmentionedatallby
anyoftherespondentsinacountrythecellisleftempty(e.g.we
didnotaskquestionsaboutallofthefactors,butaskedanopen
question with regard to what factors the respondent could
mentionthateitherenabledorhindereddatauseintheirschool
orclassroom).
Table 6 shows that organizational characteristics of
influ-enced data-based decision making in the five countries. The
followingorganizationvariableswerementionedbythe
respon-dentsaseitherenablingorhinderingfactorsintheuseofdata.
German respondents stated that they were not collaborating
aroundtheuseofdata.Insomeschoolsteacherscollaborated
around the use of data inthe Netherlands, but this was not
common. Several teachers complained about the lack of
collaborationaroundtheuseofdataandalackofsupportfrom
theircolleagues.IntheUK,teachercollaborationarounddatause
was common. Teachers collaborated around the analysis,
interpretationanduse ofdatain,forexample,subject matter
teams,gradelevelteamsordatateams.Lithuanianrespondents
also indicated that they collaborated around the use of data.
TeachersinPolandcollaboratedaroundtheuseofdata,usuallyin
subjectspecificteammeeting,wherestudentoutcomedatawas
analyzed, sometimes at the request of the school leader.
However,most ofthecommunicationtook placebye-mailor
byinformalcommunication.
IntheNetherlandsandtheUK,somerespondentsindicatedthat
therewasadataexpertavailabletoanswertheirquestionsabout
datause.IntheNetherlands,thiswasintheformofaso-called
qualitymanager. However,this personoftenworkednot atthe
school,butattheleveloftheschoolboard,andonlyschoolleaders
turnedtothispersonforhelparoundtheuseofdata.IntheUK
schoolsthedataexpertsworkedattheschoolandteacherscould
also turn to thesepersons for help. The data experts in these
schoolsweremembersofschoolstaff,deputyandassistanthead
teachers,whowereappointedtotheroleofdatamanagerordata
administrator,buthadotherresponsibilitiesintheschoolaswell.
Theywere, forexample,responsiblefortheinput,warehousing
andexchangeofdata.Thesedataexpertscouldprovidetheneeded
datainatimelymatter,aswellasassistinanalyzing,interpreting,
andusingdata.
InGermany,alackofaclearvisionandgoalshinderedtheuseof
data.Germanrespondentsmentionedthatageneralstrategyofthe
educationauthoritieswithregardtodatausewasmissing.Often
nospecificinstructionortargetableimprovementvaluesorgoals
had beenformulated. Onlysomerespondents indicated thatin
theirschooltheyhadclearandmeasurablegoals.Insomeschools
intheNetherlandsaclearvision,normsandgoalsexisted,inother
schools not. UK and Lithuanian respondents mentioned that
havinga clearvision and goalswasimportant in theirschools.
Respondentsinbothcountriestalkedaboutusingdatatomonitor
theimplementationofthesevisionandgoals.
Some Dutchand Polish respondentstalkedabouthow their
schoolleadersactivelysupported,encouragedandfacilitatedthe
use of data. One of the Polish school leaders coordinated and
supportedtheworkoftheteamsinoneoftheschools.Heprovided
structures, put processes in place, participated in meetings,
supported the development of an improvement plan, and
monitored the implementation. However, in the other Polish
schooltheschoolleaderdidnotputinplaceastructuredprocess
Table6
Organizationalcharacteristicsinfluencingdatauseaccordingtotherespondents.
GE UK LT PO NL
Teachercollaborationandcolleaguesupport + + +/ +/
Dataexpert + +
Visionandgoalsfordatause +/ + + +/
Schoolleadersupport + +/
Trainingandsupportindatause +
Time +/ +
Table7
Dataanddatasystemcharacteristicinfluencingdatauseaccordingtotherespondents.
GE UK LT PO NL
Availabilityofandaccesstodifferenttypesofdata +/ + + +/
Qualityofthedata +/ +/ +/ + +/
Datasystem,toolsandsoftware +
Table8
Usercharacteristicinfluencingdatauseaccordingtotherespondents.
GE UK LT PO NL
Attitudetowarddata + + + + +/
for data use and monitoring. In the Netherlands, several
respondentsalso indicated that their school leaderencouraged
theuseofdata,althoughsomerespondentsindicatedthattheydid
notfeelsupportedbytheirschoolleader.
InGermany,littletonosupportexistsforschoolsintheuseof
data.Trainingindatausehappenedsporadicallyandwasusually
linked to studies conducted by external parties. After the
evaluation of national test results, for example, the executing
institutesofferedworkshops.Nonationwidetrainingwas
estab-lished.UKrespondentsindicatedthattheyhadreceivedextensive
trainingindatause,andthattheyalsoreceivedsupportfromtheir
LocalEducationAuthority.However,severalrespondents
indicat-edthattheystillfounditdifficulttocomeupwithimprovement
measuresbasedondata.Professionaldevelopmentaroundtheuse
ofdatawasnotastandardofferingtoteachersorschoolleadersin
Poland.Onlymotivatedandinnovativeteachersandschoolleaders
developed competenciesin this area, mainlythrough pursuing
conference participation or individual reading and on-the-job
learning. However, the drive toward developing data use
competencieswas gradually increasing as thestate exam data
andvalueaddeddataweregainingmoreattentionandaresubject
ofvariousregionalorstate-levelanalyses.DutchandLithuanian
teachersindicatedhavingreceivednotraining.
Respondentsinallcountriescomplainedaboutalackoftimeto
usedata. In someUKschools and in one of thePolishschools
structuredtimewassetasidetousedataandstructuredprocesses
fordatauseexistedwithintheschool.
Severalrespondentstalkedabouthowdifferentdataanddata
systems influenced theuseof data (see Table7). Interestingly,
someenablingfactorscouldalsoformbarrierstotheuseofdata.
For example, respondents in the Netherlands talked about the
importanceofhavingdifferenttypesofdataavailable(e.g.notonly
assessment data), but other respondentstalkedabout how the
availabilityofawiderangeofdifferentdatasourcescouldbecome
abarriertodatauseasrespondentsindicatedthattheyfeltlike
they were ‘‘drowning in data’’ or experienced ‘‘information
overload,becausetherewastoomanydataout there’’.German
respondentscomplainedthatwereseveraldatasourcesoutthere,
butthattheyhadonlylimitedaccesstodata,especiallywhenit
concernedstudentdata.Thiswasrelatedtostrictprivacypolicies
aroundstudent data.Moreover, in Germanydata,suchas final
examination results, werenot always timelyavailable. UK and
Polishrespondentsindicated that their schoolshad access toa
widerange ofdata sources,althoughPolishrespondentstalked
mostly only about assessment data. Lithuanian respondents
indicatedthattherewereproblemswiththeavailabilityofdata.
Severalrespondentstalkedaboutproblemswiththequalityof
data.Forexample,inGermany,thedatacollectionforthenational
learning performance measurements is carried out within the
schoolandisveryerror-prone,resultinginlowqualitydata.Inthe
UK,thequalityof datawasusually good,althoughrespondents
sometimescomplainedaboutthelackoftimelyandaccuratedata
(forexample,theycouldnotalwaysuseestimatesofattainment,
becausethese are influenced by deprivation factors). Lithuania
respondentsindicatedthatdatawerealwaysrelevant.Thefactthat
Lithuanian schools were able to use data to some extent was
probably due to the fact that both the external and internal
evaluationresultedinusable,relevant,reliableandaccuratedata.
In the Netherlands, respondents stated that there were some
problemswithtimelyandrelevantdatathatcoincideswiththe
needs of the user. The only respondents that seemed to be
completelysatisfiedwiththequalityofthedata(andspecifically
thequalityofthevalueaddedstudentachievementdata)werethe
Polishrespondents.
UK respondents talked about the data systems, tools, and
software available in their schools. Mostly, UK respondents
indicatedthatwiththesesystemstheywereabletofindthedata
they need easily and timely. Some departments even had
departmentspecificdatasystemsalignedwiththeirneeds.Finally,
they had toolsavailable to analyzeand usedata. According to
German respondents, there was a problem of interoperability
between thedifferentdata systemsintheir schools. Hence,the
relationbetween differentdata couldnotbe analyzed.German
informationsystemsareheterogeneousandteachersselectedtheir
owntoolswhich didnot fittothecentralinformation systems.
There were no data standards and the ICT infrastructure for
administrativepurposesinschoolsdidnotallowcollaborativeor
individualdatause.
Finally,respondentsmentionedseveraluser characteristicsas
enabler or barriers. A positive attitude toward data use was
mentionedbyrespondentsinallcountriesasanenablinginfluence
ondatause(seeTable8).IntwoDutchschools,respondentsshowed
amorenegativeattitudetowardtheuseofdata.Onerespondent
mentioned thathe‘‘didnotbelieve intheuseofdata’’.Another
teacherindicatedthat‘‘assessmentresultsaredifferenteachyear,
dependingonwhetheryouhavegoodornotsogoodstudents’’.
Also,schoolstaffneedknowledgeandskillstocollect,analyze,
interpretandusedata.German,DutchandLithuanianrespondents
indicated thattheylackeddataanalysisand datauseskills.UK
respondentsindicatedthattheyweretrainedintheuseofdataand
possessedtheknowledgeandskillsneededtousedataeffectively.
RespondentsofonePolishschoolindicatedthattheyneededto
workontheirskillstousedata,respondentsoftheotherschool
indicatedthattheyhadtheknowledgeandskillsneededtowork
withdata,astheseteacherswerecertifiedexaminers.
Conclusionanddiscussion
Beforediscussingtheresults,wehavetodiscussthelimitations
ofthis study.First ofall,theschoolsthatparticipated arenota
representativesample,but wereselectedbecausetheyaregood
examplesofhowdataisusedineachofthecountries.Wewantto
emphasizethatthegoalofthispartoftheprojectwasnottomake
firmgeneralizations,buttogainmoreinsightsintotheuseofdata
indifferentcountries.Teachers’andschoolleaders’self-perception
isusedtostudytheiruseofdata.Wecheckedthecommentsmade
bytherespondentsbyaskingformoredetailsandbyaskingfor
examples.
Byidentifyingthepurposesforwhichdataarebeingusedin
schoolsitmayseemthatwepresentdatauseasaratherlinear,
rationalprocess,whichitis not.Datauseinvolvesa numberof
processes,conditionsandcontextswhichinteractincomplexways
and context interacts with user characteristics, data use and
stakeholderlearning.Datauseinvolvesaninterpretativeprocess,
in which data has to be identified, collected, analyzed and
interpretedtobecomemeaningfulandusefulforactions(Coburn
& Turner, 2011;Coburn, Toure,&Yamashita, 2009). Allfactors
(datause,organizationalcharacteristics,datacharacteristics,user
characteristics)areinterlinkedandcaninfluenceeachother.
Ourfirstresearchquestionwas‘‘whatdataareusedbyschools
in thedifferentcountries?’’.The resultsof ourstudy showthat
schoolsinthefiveEuropeancountrieshaveawiderangeofdata
availabletothem.TherespondentsoftheUKmentionedthewidest
rangeofdataavailable,whichisnotsurprising,asthiscountryalso
hasthelongestEuropeantraditionindatause.Thefocusseemedto
bemostlyonvalueaddedachievementdata,attainmentdata,and
progress data,although respondentsalsoindicated that a wide
range ofinput,process andcontext datawereavailable. Inour
view,thenarrowfocusonachievementdatacanleadtoanarrow
formof data-baseddecisionmaking focusingpredominantlyon
cognitiveoutcomes.Wewouldarguethatdatashouldbeusedto
reached,andifnot,takeactionaccordingly.Severalofthesegoals
pertain tostudent achievement, but schools pursue other
out-comesaswell,forexamplewithregardtoemotionalwell-beingof
students,socialskills,andcitizenshipcompetences.
German and Lithuanian respondents mentioned the least
amountofdatasources.Thequestioniswhethertheseotherdata
sourcesdonotexists,respondentsdonothaveaccesstothesedata
sources,orrespondentsdonotknowthesedatasourcesexists.In
the case of Germany, respondents did indicate that problems
existedwithregardtoaccesstodifferentdatasourcesduetostrict
privacypolicies.
Our second research question was ‘‘for which purposes do
schoolleadersand teachersusedatainthesecountries?’’.With
regardtothepurposesofusingdataforschoolandinstructional
improvement,respondentsinallcountriestalkedabouttheuseof
data for policy development, school improvement planning,
teacherdevelopment and instructional improvement. However,
inallthecountries,withtheexceptionoftheUK,datausedseemed
tobeusedataverysuperficiallevelanddidnotmovebeyondthe
monitoringphase.RespondentsfromtheUKwereabletodescribe
themostconcrete examples (e.g. howvalue added assessment
results were used to target instruction toward weak(er) and
strong(er)students).Mostrespondentsfromtheothercountries
werenot able to provide concrete examples of how identified
weaknessesbasedondatalettoactionsintheclassroom,although
thereweresomeexceptionsfromDutchandPolishrespondents.
Datauseforaccountabilitywascommoninallfivecountries.
Thefactthatthefocusoftheuseofdatainallfivecountriesseems
tobemore on accountabilitythan on school development and
instructionalimprovement is worrisome. In everycountry it is
importanttoholdschoolsaccountablefortheirfunctioning,but
thisshouldneitherbethesolenorthemostimportantaspectof
datause.Thefocusshouldbeontheuseofdataforimprovement.A
solefocusondatauseforaccountabilityalsocomeswithadanger
ofnegativesideeffects.IntheUnitedStates,wehaveseenseveral
examplesofthesenegativesideeffects,includingfocusingonlyon
aspecifictypeofstudentswhocanhelpimproveyourstatuson
accountabilityindicators(e.g.bubble kids),cheatingtoimprove
the status on accountability indicators, teaching to the test,
excluding certain students from a test, and encouraging low
performing students to drop out (Ehren & Swanborn, 2012;
Hamilton,Stecher,&Yuan,2009).
Withregardtoourthirdresearchquestion,‘‘which
organiza-tional,dataanddatasystemsandusercharacteristicsinfluencethe
useofdata?’’,theresultsshowthatallthreefactorsinfluencethe
useofdata.Theresultsofthis studyalsoprovideuswithsome
insightintowhyseveralschoolsarenotusingdatafor
improve-mentpurposes,especiallywhenwecomparetheschoolsin the
Netherlands,Germany,PolandandLithuaniawiththeschoolsin
theUK. UK schools seem muchmore equiped to use (student
achievement)dataeffectively,asseveralorganizationalstructures
wereputinplaceinUKschools.Importantdifferenceswiththe
other four countries include that in UK schools, respondents
collaboratedaroundtheuseofdata,forexampleingradelevels,
theyhad a dataexpertavailableon site, andmabybe themost
importantdifference, UK respondentsindicated that they were
trainedintheuseof(achievement)data.
AnotherimportantdifferencebetweentheUKandtheotherfour
countries pertains to data and data system characteristics,
specificallydatainformationsystems,tools,andsoftwaretoanalyze
data.IntheUK,teachershadaccesstoverysophisticatedsystems
andtools,althoughsomerespondentsindicatednotknowinghowto
usethese,inwhichcaseaccesstothesesystemsandtoolsbecamea
barrierinsteadofanenabler.Intheothercountries,schoolsdidnot
havethesesophisticatedsystemsandtoolsavailable.Forexample,In
Germanschoolstherewasaproblemofinteroperabilitybetweenthe
differentdatasystemsintheirschools,andteacherswerenotableto
combineandanalyzecertaintypesofdata.
Whatisstrikingisthattheavailabilityofdifferenttypesofdata,
which is generally seen as an enabling factor, can also forma
barrier to data use. In the Netherlands, for example, some
respondentsindicated thatthere wastoomuchdataout there,
andtheydidnotknowwheretostart.Datnowetal.(2013)talk
aboutsimilarfindingsintheirstudyinwhichseveralfactorscan
becomeanaffordance(enabler)orconstraint(barrier).They,for
example,foundthatprovidingteacherswithtimeforcollaboration
arounddatause,andprotocolsandstructureswithregardhowto
usedata,generallyseenasenablers,sometimesworkedasbarriers
asteachersfeltthattherewasyetanotherthingimposedonthem.
AsstatedbyDatnowetal.(2013,p.346):‘‘someoftheverysame
conditionscanbebothanaffordanceandconstraintatthesame
time.Agreatdealdependsonthecontextinwhichtheworktakes
place,aswellasindividuals’experiencesandknowledge.’’
The final importance difference we found between the
countries is that respondents in the UK and Poland indicated
havingtheknowledgeandskillstoanalyzeandusedata,although
inPolandthispertainedtoanalyzingspecificassessmentdataby
certified examiners. Respondents from the other countries
indicated lacking knowledge and skills to use data effectively
(e.g.dataliteracy),whichisalsoaresultofalackoftraining.
Theresultsofthisstudyconfirmthatorganizational,dataand
datasystem,andusercharacteristicsallinfluencetheuseofdatain
thedifferentschoolsinthedifferentcountries.However, inone
countryorschoolafactor canworkasa enabler(e.g.accessto
differenttypesofdata)whereasinanotherschoolorinanother
countrythatsamefactorcanbeabsentorworkasabarrier(e.g.
thereistoomuchdataoutthere).Whetherafactorworksasan
enablerorbarriercanalsodependonthedatauser.Inthecaseof
ourexample,theuserdoesnotknowwheretostartbecausethere
istoomuchdataoutthere.Itcanalsodependontheorganization.
Inourexample,theschoolcouldstreamlinethedataandprovide
userswithaccesstodatathatarerelevanttothem.
Althoughitwasnottheexplicitaimofthisstudywedidnotice
thatpressurefromtheaccountabilitysystemseemedtoplayan
importantroleinallcountries.However,intheUKpressurewas
combinedwithsupport.Effectivedatauseprobablyrequiressome
pressure from the accountability system, but is needs to be
combinedwithsupportintermsofagoodfunctioningdatasystem,
dataanalysestools,andprofessionaldevelopmentintheuseof
data(Schildkamp&Lai,2013).
Also,theamountofautonomyschoolshaveindecisionmaking
canaffectdatause.IntheUnitedKingdomandtheNetherlands,
schools have a lot of autonomyand theycan make almost all
decisionsthemselves(withregardtothecurriculum,instruction,
personnel and resources). In Germany, schools have a lot less
autonomy.AlackofautonomyinGermanymayhavehinderedthe
useofdata,simplebecauseschoolleadersandteacherswereoften
notallowedtomakecertaindecisions.Ifyoucombinethiswitha
lackofdataliteracy,andverystrictprivacypolicieswhichprevent
accesstoseveral studentsources, itis easy tounderstand why
Germanrespondentsindicatedthatdatawerenotusedalot.
Itisimportanttonotethatthereisadifferencebetweenthe
actualavailabilityofpolicyspaceandtheexperiencedpolicyspace.
InschoolsintheUnitedKingdomandintheNetherlandsample
policyspaceexistsforschoolstomakeallsortsofdecisionsbased
ondata,forexamplewithregardtothecurriculum,instruction,
andevenwithregardtoattainmenttargetstosomeextent.For
example,intheNetherlands,overtheyearsschoolshavereceived
morepolicyspace.Thenumberofattainmenttargetsdecreased
from122(in1993)to58.Furthermore,theattainmenttargetsthat
still exist are much less detailed and do not specify teaching
inspirationforschoolstobasedecisionson.Theydostillforma
frameofreferenceforaccountability,forexamplewithregardto
outcomes(Nieveen&Kuiper,2012).However,thisobjectivepolicy
spacemightdifferfromtheperceptionofthispolicyspace.Several
schools and teachers do not perceive having this policy space
(Nieveen&Kuiper,2012;Nieveen,VandenAkker,&Resink,2010),
andconsequentlyarenotinclinedtomakedecisionwithregardto
thecurriculumandinstructionbasedondata.
Whatisinterestingtonoteisthatalthoughtherewerehuge
variationsinthecontextsofschools participatingin this study,
almostalltherespondentsacknowledgedtheimportanceofdata
use,andalsoacknowledgedtheproblemswithdatauseintheir
schools.Themostcommonproblemswereproblemswithaccessto
dataandappropriatedatasystemsandtools,alackofknowledge
and skills in the use of data (and related to this a lack of
professionaldevelopmentandtrainingintheuseofdata),alackof
teachercollaboration.Therefore,webelievethatthewayforward
istoinvestinprofessionaldevelopmentofteamsofteachersand
schoolleadersintheuseofdata.
Professionaldevelopmentintheuseofdataisurgentlyneeded
and is crucial for improving the quality of schools (Desimone,
2009; Van Veen, Zwart, Meirink, & Verloop, 2010). These
professionaldevelopment activitiesof courseneedtotake into
accountthecontextoftheschoolandcountry,butwebelievethat
theseactivitiescanlooksimilarinallofthecountries.Asresults
fromthisstudyindicate,teachersandschoolleaderssometimesdo
notknowwithwhatdatatostart.Paradoxically,webelievethat
thebestwaytostartwithdatauseisnottostartwithdatabutto
startwithaproblemaschoolwantstosolveorgoalstheywantto
achieve.Thenextstepis collectingdataon theseproblemsand
goals.Thisapproachcanworkinallcountriesaroundtheglobe,but
only ifwe invest in training for data literacy and investin an
effectivedatainfrastructure.
Acknowledgements
Thispaperincludespartsofthereports‘‘Comparativeanalyses
data use in Germany, The Netherlands, Lithuania, Poland and
England’’,workpackage lead: Dr Kim Schildkamp,University of
Twente and ‘‘Survey Data Analysis’’, workpackage lead: ifib
InstituteforInformationManagementBremenGmbH.Theauthors
of these reports are: PCG Polska (PL), Specialist Schools and
AcademiesTrust(UK),ModernDidacticsCenter(LT),Universityof
Twente(NL),InstituteforInformationManagementBremenGmbH
(GE). All rights reserved to DATAUSE project partners. More
information can be found here: www.datauseproject.eu. The
DATAUSEprojecthasbeenfundedwithsupportfromtheEuropean
Commission.Thiscommunication reflectstheviewsonly ofthe
authors,andtheCommissioncannotbeheldresponsibleforany
usewhichmaybemadeoftheinformationcontainedtherein.
References
Boudett,K.P.,&Steele,J.L.(2007).Datawiseinaction.Storiesofschoolsusingdatato improveteachingandlearning.Cambridge:HarvardEducationPress.
Breiter,A.,&Light,D.(2006).Dataforschoolimprovement:Factorsfordesigning effectiveinformationsystemstosupportdecision-makinginschools.Educational Technology&Society,9(3),206–217.
Campbell,C.,&Levin,B.(2009).Usingdatatosupporteducationalimprovement. EducationalAssessment,EvaluationandAccountability,21(1),47–65.
Carlson,D.,Borman,G.,&Robinson,M.(2011).Amultistate district-levelcluster randomizedtrialoftheimpactofdata-drivenreformonreadingandmathematics achievement.EducationandEvaluationandPolicyAnalysis,33(3),378–398.
Cho,V.,&Wayman,J.(2013).Datasystemsandsensemaking.TeachersCollegeRecord. (inpress).
Coburn,C.E.,&Talbert,J.E.(2006a).Conceptionsofevidenceuseinschooldistricts: Mappingtheterrain.AmericanJournalofEducation,112,469–495.
Coburn,C.E.,&Talbert,J.E.(2006b).Conceptionsofevidenceuseinschooldistricts: Mappingtheterrain.AmericanJournalofEducation,112(4),469–495.
Coburn,C.E.,Toure,J.,&Yamashita,M.(2009).Evidence,interpretation,and persua-sion:Instructionaldecisionmakinginthedistrictcentraloffice.TeachersCollege Record,111(4),1115–1161.
Coburn,C.E.,&Turner,E.O.(2011).Researchondatause:Aframeworkandanalysis. Measurement,9,173–206.
ComeniusProjectUsingDataforImprovingSchoolandStudentPerformance.(2011). ComparativeanalysesdatauseinGermany,TheNetherlands,Lithuania,Polandand The United Kingdom. Accessible at: http://www.datauseproject.eu/ Retrieved 16.11.13.
Datnow,A.,Park,V.,&Kennedy-Lewis,B.(2013).Affordancesandconstraintsinthe contextofteachercollaborationforthepurposeofdatause.JournalofEducational Administration,51(3),341–362.
Desimone,L.M.(2009).Improvingimpactstudiesofteacher’sprofessional develop-ment:Towardbetterconceptualizationsandmeasures.EducationalResearcher, 38(3),181–199.
Diamond,J.B.,&Spillane,J.P.(2004).High-stakesaccountabilityinurbanelementary schools:Challengingorreproducinginequality.TeachersCollegeRecord,106(6), 1145–1176.
Downey,C.,&Kelly,A.(2013).ProfessionalattitudestotheuseofdatainEngland.InK. Schildkamp,M.K.Lai,&L.Earl(Eds.),Data-baseddecisionmakingineducation: Challengesandopportunities(pp.69–89).Dordrecht:Springer.
Earl,L.,&Katz,S.(2006).Leadinginadatarichworld.ThousandOaks,CA:CorwinPress.
Ehren,M.C.M.,&Swanborn,M.S.L.(2012).Strategicdatauseofschoolsinaccountability systems.SchoolEffectivenessandSchoolImprovement,23(2),257–280.
Hamilton,L.S.,Stecher,B.M.,&Yuan,K.(2009).Standards-basedreformintheUnited States:History,research,andfuturedirections.SantaMonica,CA:RANDCorporation. Retrievedfromhttp://www.rand.org/pubs/reprints/RP1384.
Honig,M.I.,&Venkateswaran,N.(2012).School–centralofficerelationshipsin evi-denceuse:Understandingevidenceuseasasystemsproblem.AmericanJournalof Education,118(2),199–222.
Ikemoto,G.S.,&Marsh,J.A.(2007).Cuttingthroughthedata-drivenmantra:Different conceptionsofdata-drivendecisionmaking.InP.A.Moss(Ed.),Evidenceand decisionmaking.USA:Wiley-Blackwell.
Jimerson, J. B.,&Wayman, J.C. (2012).Brandingeducational data usethrough professionallearning:Findingsfromastudyinthreedistricts.Paperpresented attheAmericanEducationalResearchAssociationConference.
Lai,M.K.,McNaughton,S.,Amituanai-Toloa,M.,Turner,R.,&Hsiao,S.(2009).Sustained accelerationofachievementinreadingcomprehension:TheNewZealand experi-ence.ReadingResearchQuarterly,44(1),30–56.
Levin,J.A.,&Datnow,A.(2012).Theprincipalroleindata-drivendecisionmaking: Usingcase-studydatatodevelopmulti-mediatormodelsofeducationalreform. SchoolEffectivenessandSchoolImprovement,23(2),179–202.
Little,J.W.(2012).Understandingdatausepracticeamongteachers:Thecontribution ofmicro-processstudies.AmericanJournalofEducation,118(2),143–166.
Mandinach,E.B.,&Honey,M.(Eds.).(2008).Datadrivenschoolimprovement:Linking dataandlearning.NewYork:TeachersCollegePress.
Marsh,J.A.(2012).Interventionspromotingeducators’useofdata:Researchinsights andgaps.TeachersCollegeRecord,114(11),1–48.
MinistryofEducationandScienceoftheRepublicofLithuania.(2004).Educationin Lithuania. Vilnius: Ministryof Education and Science of the Republic of Lithuania.
Nieveen,N.,&Kuiper,W.(2012).Balancingcurriculumfreedomandregulationsinthe Netherlands.EuropeanEducationalResearchJournal,11(3),357–368.
Nieveen,N.,VandenAkker,J.,&Resink,F.(2010).School-basedcurriculum develop-mentintheNetherlands.PaperpresentedattheEuropeanConferenceonEducational Research(ECER).
OECD.(2008).Educationat aglance2008:OECDindicators.Paris:Organisationfor EconomicCo-operationandDevelopment(OECD).
OECD.(2010).Educationat aglance2010:OECDindicators.Paris:Organisationfor EconomicCo-operationandDevelopment(OECD).
Poortman,C.L.,&Schildkamp,K.(2011).Alternativequalitystandardsinqualitative research?QualityandQuantity,46(6),1727–1751.
Riege,A.M.(2003).Validityandreliabilitytestsincasestudyresearch:Aliterature reviewwithhands-onapplicationsforeachresearchphase.QualitativeMarket Research:AninternationalJournal,6(2),75–86.
Schildkamp,K.,&Kuiper,W.(2010).Datainformedcurriculumreform:Whichdata, whatpurposes,andpromotingandhinderingfactors.TeachingandTeacher Educa-tion,26,482–496.
Schildkamp,K.,Lai,M.K.,&Earl,L.(2013a).Data-drivendecisionmakingaroundthe world:Challengesandopportunities.Dordrecht:Springer.
Schildkamp,K.,&Lai,M.K.(2013).Data-baseddecisionmaking:Conclusionsand theoreticalframework.InK.Schildkamp,M.K.Lai,&L.Earl(Eds.),Data-based decisionmakingineducation:ChallengesandopportunitiesPLDordrecht(pp.177– 191).Springer.
Schildkamp,K.,Poortman,C.L.,&Handelzalts,A.(2013b).Datateamsforschool improvement.TeachingandTeacherEducation.(submittedforpublication).
Spillane,J.P.(2012).Datainpractice:Conceptualizingthedata-baseddecisionmaking phenomena.AmericanJournalofEducation,118(2),113–141.
Stake,R.E.(2000).Casestudies.InN.K.Denzin&Y.S.Lincoln(Eds.),Handbookof qualitativeresearch(pp.435–454).ThousandOaks,CA:SagePublications.
Supovitz,J.(2010).Knowledge-basedorganizationallearningforinstructional im-provement.InA.Hargreaves,A.Lieberman,M.Fullan,&D.Hopkins(Eds.),Second internationalhandbookofeducationalchange(pp.707–723).NewYork:Springer.
Vanhoof,J.,Verhaeghe,G.,VanPetegem,P.,&Valcke,M.(2011).Theinfluenceof competencesandsupportonschoolperformancefeedbackuse.EducationalStudies, 37(2),141–154.
VanVeen,K.,Zwart,R.,Meirink,J.,&Verloop,N.(2010).Professionele ontwikkel-ingvanleraren.Eenreviewstudienaareffectievekenmerkenvan professio-naliseringsinterventiesvanleraren[Professionaldevelomentofteachers.A reviewintoeffectivecharacteristicsofprofessionaldevelopment interven-tions].ICLON/ExpertisecentrumLerenvanDocenten.
Wayman,J.C.,&Stringfield,S.(2006). Datauseforschoolimprovement:School practices and research perspectives. American Journal of Education, 112, 463–468.
Wayman,J.C.,Spring,S.D.,Lemke,M.A.,&Lehr,M.D.(2012a).Usingdatatoinform practice:Effectiveprincipalleadershipstrategies.PaperpresentedattheAmerican EducationalResearchAssociationConference.
Wayman,J.C.,Jimerson,J.B.,&Cho,V.(2012b).Organizationalconsiderationsin establishingthedata-informeddistrict.SchoolEffectivenessandSchool Improve-ment,23(2),159–178.
Wohlstetter,P.,Datnow,A.,&Park,V.(2008).Creatingasystemfordata-driven decision-making:Applyingtheprincipal-agentframework.SchoolEffectiveness andSchoolImprovement,19(3),239–259.
Yin,R.K.(1984).Casestudyresearch:Designandmethods.BeverlyHills,CA:Sage Publications.
Yin,R.K.(2009).Casestudyresearch(4thed.).BeverlyHills,CA:SagePublications.
Young,V.M.(2006).Teachers’useofdata:Loosecoupling,agendasetting,andteam norms.AmericanJournalofEducation,112,521–548.
