Citation for this paper:
Kaipio, J.; Lääveri, T.; Hyppönen, H.; Vainiomäki, S.; Reponen, J.; Kushniruk, A.; …
& Vänskä, J. (2017). Usability problems do not heal by themselves: National survey
on physicians’ experiences with EHRs in Finland. International Journal of Medical
Informatics, 97, 266-281.
UVicSPACE: Research & Learning Repository
_____________________________________________________________
Faculty of Human and Social Development
Faculty Publications
_____________________________________________________________
Usability problems do not heal by themselves: National survey on physicians’
experiences with EHRs in Finland
Johanna Kaipio, Tinja Lääveri, Hannele Hyppönen, Suvi Vainiomäki, Jarmo
Reponen, Andre Kushniruk, Elizabeth Borycki, & Jukka Vänskä
January 2017
© 2017 Kaipio et al. This is an open access article distributed under the terms of the
Creative Commons Attribution NC-ND License.
http://creativecommons.org/licenses/by-nc-nd/4.0/
This article was originally published at:
ContentslistsavailableatScienceDirect
International
Journal
of
Medical
Informatics
jo u r n al h om e p a g e :w w w . i j m i j o u r n a l . c o m
Usability
problems
do
not
heal
by
themselves:
National
survey
on
physicians’
experiences
with
EHRs
in
Finland
Johanna
Kaipio
a,∗,
Tinja
Lääveri
b,
Hannele
Hyppönen
c,
Suvi
Vainiomäki
d,
Jarmo
Reponen
e,f,
Andre
Kushniruk
g,
Elizabeth
Borycki
g,
Jukka
Vänskä
haAaltoUniversity,SchoolofScience,DepartmentofComputerScience,Espoo,Finland
bUniversityofHelsinkiandHelsinkiUniversityHospital,InflammationCenter,ClinicofInfectiousDiseases,Helsinki,Finland cNationalInstituteforHealthandWelfare(THL),InformationDepartment,Helsinki,Finland
dUniversityofTurku,DepartmentofClinicalMedicine,GeneralPractice,Turku,Finland
eUniversityofOulu,Finntelemedicum,ResearchUnitofMedicalImaging,PhysicsandTechnology,Oulu,Finland fHospitalofRaahe,DepartmentofRadiology,Raahe,Finland
gUniversityofVictoria,SchoolofHealthInformationScience,Victoria,BritishColumbia,Canada hFinnishMedicalAssociation,Helsinki,Finland
a
r
t
i
c
l
e
i
n
f
o
Articlehistory:
Received15December2015
Receivedinrevisedform2October2016 Accepted10October2016 Keywords: Usability Userexperience Physician Questionnaire
Electronichealthrecordsystem Healthinformationsystem Nationalsurvey
a
b
s
t
r
a
c
t
Purpose:Surveystudiesofhealthinformationsystemsusetendtofocusonavailabilityoffunctionalities, adoptionandintensityofuse.Usabilitysurveyshavenotbeensystematicallyconductedbyanyhealthcare professionalgroupsonanationalscaleonarepeatedbasis.Thispaperpresentsresultsfromtwo cross-sectionalsurveysofphysicians’experienceswiththeusabilityofcurrentlyusedEHRsystemsinFinland. Theresearchquestionswere:Towhatextenthastheoverallsituationimprovedbetween2010and2014? Whatdifferencesaretherebetweenhealthcaresectors?
Methods:Inthespringof2014,asurveywasconductedinFinlandusingaquestionnairethatmeasures usabilityandrespondents’userexperienceswithelectronichealthrecord(EHR)systems.Thesurveywas targetedtophysicianswhowereactivelydoingclinicalwork.Twenty-fourusability-relatedstatements, thatwereidenticalin2010and2014,wereanalysedfromthesurvey.Therespondentswerealsoasked togiveanoverallratingoftheEHRsystemtheyused.Thestudydatacomprisedresponsesfrom3081 physiciansfromtheyear2014andfrom3223physiciansintheyear2010,whowereusingtheninemost commonlyusedEHRsystembrandsinFinland.
Results:Physicians’assessmentsoftheusabilityoftheirEHRsystemremainascriticalastheywerein 2010.Onascalefrom1(‘fail’)to7(‘excellent’)theaverageofoverallratingsoftheirprincipallyusedEHR systemsvariedfrom3.2to4.4in2014(andin2010from2.5to4.3).Theresultsshowsome improve-mentsinthefollowingEHRfunctionalitiesandcharacteristics:summaryviewofpatient’shealthstatus, preventionoferrorsassociatedwithmedicationordering,patient’smedicationlistaswellassupport forcollaborationandinformationexchangebetweenthephysicianandthenurses.Evenso,supportfor cross-organizationalcollaborationbetweenphysiciansandforphysician-patientcollaborationwerestill consideredinadequate.Satisfactionwithtechnicalfeatureshadnotimprovedinfouryears.Theresults showmarkeddifferencesbetweentheEHRsystembrandsaswellasbetweenhealthcaresectors(private sector,publichospitals,primaryhealthcare).Comparedtoresponsesfromthepublicsector,physicians workingintheprivatesectorweremoresatisfiedwiththeirEHRsystemswithregardstostatements aboutuserinterfacecharacteristicsandsupportforroutinetasks.Overall,thestudyfindingsaresimilar toourpreviousstudyconductedin2010.
Conclusions:SurveysabouttheusabilityofEHRsystemsareneededtomonitortheirdevelopmentat regionalandnationallevels.Toourknowledge,thisstudyisthefirstnationaleHealthobservatory ques-tionnairethatfocusesonusabilityandisusedtomonitorthelong-termdevelopmentofEHRs.The resultsdonotshownotableimprovementsinphysician’sratingsfortheirEHRsbetweentheyears2010 and2014inFinland.Instead,theresultsindicatetheexistenceofseriousproblemsanddeficiencies whichconsiderablyhindertheefficiencyofEHRuseandphysician’sroutinework.Thesurveyresults callforconsiderableamountofdevelopmentworkinordertoachievetheexpectedbenefitsofEHR systemsandtoavoidtechnology-inducederrorswhichmayendangerpatientsafety.Thefindingsof repeatedsurveyscanbeusedtoinformhealthcareproviders,decisionmakersandpoliticiansaboutthe ∗ Correspondingauthor.
E-mailaddress:Johanna.Kaipio@aalto.fi(J.Kaipio). http://dx.doi.org/10.1016/j.ijmedinf.2016.10.010
1386-5056/©2016TheAuthors.PublishedbyElsevierIrelandLtd.ThisisanopenaccessarticleundertheCCBY-NC-NDlicense( http://creativecommons.org/licenses/by-nc-nd/4.0/).
J.Kaipioetal./InternationalJournalofMedicalInformatics97(2017)266–281 267 currentstateofEHRusabilityanddifferencesbetweenbrandsaswellasforimprovementsofEHRusability. Thissurveywillberepeatedin2017andthereisaplantoincludeotherhealthcareprofessionalgroups infuturesurveys.
©2016TheAuthors.PublishedbyElsevierIrelandLtd.ThisisanopenaccessarticleundertheCC BY-NC-NDlicense(http://creativecommons.org/licenses/by-nc-nd/4.0/).
1. Introduction
Healthcare IT (information technology) adoption rates are rapidly increasing along withthe expected benefits of system usage.InmostmodernhealthcareorganisationsITplaysan essen-tialroleincaredeliveryandclinicians’dailywork.IntheNordic countriestheavailabilityanduseoflocalElectronicHealthRecord (EHR)functionalitieshasreachedahighlevel(i.e.closeto satura-tion)[1].IntheEUcountriesaccesstobasicEHRsisbynownearly universalamonggeneralpractitioners[2].IntheUSAtheadoption ratesofEHRsystemsinhospitalshaveincreasedfrom15.6%in2010 to75.5%in2014[3].
TheeffectsoftheadoptionanduseofEHRsystemshavenot all been positive. Several studies have revealed that usability problems,technology-induced errors and lackof end-user par-ticipationin EHR developmentare continuingissues that need to be addressed (e.g. [4–11]). Poorly designed user interfaces havebeenrecognizedtoleadtotechnology-inducederrors and thereby may detrimentally affect patient safety [8,12]. Indeed, many technology-associated adverse events in medicine have beenattributedtopoorinterfacedesignratherthanhumanerror alone[13].
Clinicians’ acceptanceof and attitudestowardsEHRsystems havebeenshowntorelatecloselytosystemusability,forinstance easeofuse,integrationofthesystemsintoclinicians’workflows andhelpfulnessof thesystems inthecare ofpatients [14–16]. In addition,poorsystemdesign, systemslowdown and system downtimehavebeenconsideredthemostcommonfactorsin influ-encingclinicians’negativeattitudes towardsclinicalITsystems
[15].Usabilityandhumanfactorsapproachesneedtobeintegrated intothedesignandmonitoringofEHRsystemdevelopmentinorder toovercometheprevailingmismatchbetweenclinicalworkandIT systemsandtosupportpracticesthatimprovepatientsafety.Asa result,thereareincreasingattemptstounderstandhowsystems shouldandcouldbeimproved(e.g.[17,18]).
Currently, survey studiesof healthcare IT usetend to focus onavailabilityoffunctionalities(e.g.[19])alongwithaspectsof technologyadoptionandacceptance(e.g.[20,21]).TheOECD (Orga-nizationforEconomicCo-operationandDevelopment)hasledan effortto provide member states withreliable data in order to compareinformationandcommunicationtechnology(ICT) avail-abilityandadoptionratesinthehealthcaresector[22].Moreover, pre-implementationandpost-implementationsurveyshavebeen conductedtoinvestigateclinicians’attitudes,satisfactionand reac-tions towards systems (and their new releases) (e.g. [23]). By contrast,usabilityanduserexperiencerelatedquestionnaireshave mainlybeenappliedduringITdevelopmentprocesses.The ques-tionnaireshavebeenusedforlearningaboutinitialuseexperiences or tocompare two or more versions of differing systems (e.g.
[14,24–27])ratherthangatheringlong-termdataonexperiences aboutfullyadoptedsystemsafterlongerperiodsofuse.
Intheacademicliteratureonhuman-computerinteraction(HCI) andusabilityengineering(UE)severaldefinitionshavebeen pre-sented for the conceptsof usability and user experience (UX). CommonlyciteddefinitionsforusabilityaregivenbytheISO 9241-11standard[28]andJakobNielsen[29].Thesedefinitionsshare similarusabilitycomponentsincommon–forinstanceefficiency, satisfactionandeffectiveness−andemphasizetheroleofcontext. Atamoreconcretelevel,usabilityhasbeendescribedasfollows:“A
systemwithgoodusabilityiseasytouseandeffective.Itisintuitive, for-givingofmistakesandallowsonetoperformnecessarytasksquickly, efficientlyandwithaminimumofmentaleffort.Taskswhichcanbe performedbythesoftware(suchasdataretrieval,organisation, sum-mary,cross-checking,calculating,etc.)aredoneinthebackground, improvingaccuracyandfreeinguptheuser’scognitiveresourcesfor othertasks.”[30].Moreover,asusabilityliesintheinteractionofthe userandthesystem[31],qualityofusehasbeendescribedasthe objectofusability.Aqualityofusemodel,describedbytheISO 25010standard [32],includesfivecharacteristics: effectiveness, efficiency,satisfaction,freedomfromriskandcontext coverage. Thefirstthreeofthesecomponentsarealsopartofwidelyknown usabilitydefinitions[28,29].
Incontrast,UX(userexperience)asaconceptstillremainsvague despitedozensofattemptstodefineit[33–35].Severalofthese definitionsdescribeUXasapersonalexperienceincludingaspects ofemotions,beliefsandperceptionsthatoccurbefore,duringand aftersystemuse[36–38].Theseaspectscanbealsoseenaspart oftheconceptofusabilityassuggestedbyISO9241-210standard
[36].Usabilityshouldbeunderstoodasacontextualproperty.In thefieldofhealthinformaticsthismeansthataspectsofsafetyand preventionofmedicalerrorsaswellascharacteristicsofhealthcare workneedtobetakenintoconsiderationwhendesigningusability studies.Kushniruketal.[8]havestatedthat“theabilityofmethods fromusabilityengineeringtobeabletopredictmedicalerrorsholds considerablepotentialforassessinghealthcareinformationsystems regardingsafetyandensuringthatsuchsystemsdonotinadvertently introducemedicalerrors”.
Inourown studies[4,39–41]wehave applieddefinitionsof usabilityfromtheHCIfieldwhendescribingtheusabilityof clin-icalICTsystemsfromtheviewpointofdifferentend-usergroups withtheaimofincreasingtheunderstandingofcontextualaspects uniqueinclinicalcontexts.Theobjectiveofdesigningsystemsfor usabilitycanbedescribedasenablinguserstoachievegoalsand meettheirneedsinaparticularcontextofuse[28,36].Following fromthis,wehavepresentedadescriptionforusabilityof clini-calICTsystemsfromthephysician’sviewpoint[4]:Theusability ofclinicalICTsystemsreferstotheabilityofthesystemstohavea positiveimpactonpatientcarebysupportingphysiciansin achiev-ingtheirgoalswithapleasantuserexperience.Inordertosupport physiciansintheirdailyclinicalwork,ICTsystemsneedtobe com-patiblewithphysicians’tasks.Atamoreconcretelevel,thisindicates thatsystemsshouldprovidephysicianswithkey(context-matching) functionalities,beefficient(especiallyintermsofrecord-keepingand informationretrieval),andhaveintuitiveuserinterfaces.Inaddition, ICTsystemsshouldsupportinformationexchange,communicationand collaborationinclinicalworkandbeinteroperableandreliable.Since clinicalICTsystemsareusedinnumerousenvironments,theyshould alsoadjusttovarioususerneedsandorganisationalsettings.
ThefocusofthisarticleisonusabilityofEHRsystemsand physi-cians’experiencesinusingthesesystems.Onlyafewstudieshave beenconductedonalargescaleabouttheusefulnessand usabil-ityofEHRsystemsfromtheend-users’viewpointoverthepast fewyears.Nationalsurveysthatincludeusability-relatedquestions havebeenconductedinsomeNordiccountries[1,42–44],butthe focusoftheseworkshasmainlybeenonadoptionandintensityof use.Toourknowledge,besidesourpreviousstudy[4,9,41,45–49], specificnational usabilitysurveyshave notbeen systematically conductedamonganyhealthcareprofessionalgroups.
Longitudi-naleHealthobservatorystudiesonusabilitywillbecomeessential whenmovingfromadoptiontothenextlevel:monitoringofuse, usabilityandend-userexperiencesofEHRsystemsastoolsfor sup-portingandrenewingworkprocessesandcareoutcomesaswell astheimpactsofdevelopmentactivitiesandupdatesduring estab-lisheduse.
1.1. Contextofthestudy 1.1.1. HealthcareinFinland
Healthcare inFinlandconsists ofa decentralized,three-level publichealthcaresystemandapartlypubliclyreimbursedprivate sector.Municipalitiesareresponsibleforprimaryhealthcare. More-over, alloftheover300municipalitiesbelongtoone ofthe21 hospitaldistricts,whichprovidepublicsecondarycare,both inpa-tient andoutpatient.Outof these,five universityhospitalsalso providetertiarycare.Thevastmajorityofseverelyillpatientsare treatedbythepublichealthcaresystem,forinstancethereareno privateintensivecareunitsandallorgantransplantationsare per-formedin thepublic sector.Generalpractitioners in healthcare centresactasgatekeeperstoservicesatsecondaryandtertiary lev-elsofcare,andareferralfromaprimarycareproviderisnecessary innon-urgentcases.However,theprivatesectorcoversasmuchas onethirdofoutpatientvisits[50].Manyoftheseareoccupational healthcarevisits.ManyFinnishpatientschooseprivateprovidersin ordertochoosetheirphysicianinasecondary/tertiarycaresetting ortoavoidlongwaittimesforprimarycarephysicianappointments
[51].
1.1.2. PhysiciansinFinland
In2014,therewere16,350working-age(<65years)physicians living in Finland that were doing clinical work [52,53]. Physi-ciansworkinginhospitalsoftenworkbothinoutpatientclinics andinpatientwards.Thenumberofphysicianshasincreasedby approximately2000since2010.Theproportionofphysiciansin theyoungestandoldestagegroupsaswellasfemalephysicians hasgrownsince2010[54].
1.1.3. Healthinformationsystemsandinformationexchangein Finland
EHR coveragereached 100%in publichealthcare in 2010.In addition,thevastmajorityofprivatehealthcareprovidersuseEHR systems[42,55].Asingleadministrativeregisterofpatient infor-mationgeneratedbydifferentpublichealthcareproviderswithina hospitaldistrictwasmadepossibleonlybythenewhealthcareact in2011.Thishasincreaseddatasharingbetweenprimaryand sec-ondarycare[42]viajointregistersorregionalhealthinformation systems.
Atthetimeofthesurveyinthespring2014,healthinformation systemswereundergoingaremarkablereformintermsof connec-tivityandnewfunctionalities:thee-prescriptionfunctionalityhad alreadybeenfullyimplementedinallpublichealthcaresettingsand intwooutofthreeprivatesectorEHRsystemsthatareincludedin thisstudy.Allprescriptionsareinonesingledatabasethatcanbe accessedbypharmacies,healthcareprovidersandpatientsalike. Bycontrast,theimplementationofthenationalpatientdata repos-itory(Kanta) hadonlystartedinonesmallhospitaldistrict,but someoftheEHRsystemshadalreadydeployedthefunctionalities neededforintegrationwiththeKantaservices.1
1TheNationalArchiveofHealthInformation(Kanta)isthenameofthenational
datasystemservicesforhealthcareservices,pharmaciesandcitizens.Theservices includetheelectronicprescription,pharmaceuticaldatabase,MyKantapages,and patientdatarepository.TheservicesaredeployedinphasesthroughoutFinland. Moreinformationisavailableat:http://www.kanta.fi/en/kanta-palvelut.
1.2. Aimofthepaper
This paper presents a follow-up of a cross-sectional study of physicians’ experiences with currently used EHR systems in Finland. The first national study was conducted in 2010
[4,45,46,49].Thisstudyformedthebaselineforthefollow-upstudy resultstobereportedinthisarticle.
Theaimofthispaperistopresentthe2014surveyresultson Finnishphysicians’experienceswithEHRuseandcomparethose withthe2010resultsbyanalysingdatafromtwoperspectives:
a)To what extent has theoverall situation regarding usability changed?.
b)What differences are there between public hospitals (both outpatient and inpatient),public primary healthcare centres (outpatient)andprivateproviders(mainlyoutpatient)?.
2. Relatedresearch:questionnairestudiesonEHRusability
TheapproachandfocusofsurveysonEHRadoptionand usabil-ityseemtodifferacrosscountries[56].Forexample,Canadaisatan earlierstageinEHRadoptionanddoesnothaveanational question-nairefocusedexclusivelyonquestionsaboutEHRuse.Questions aboutITusagearepartofalargerCanadianphysiciansurvey,but thecurrentsurveydoesnotcontainquestionsaboutphysicians’ assessmentoftheusabilityofEHRs[56].TheNordiccountrieshave setupaneHealthgrouptobenchmarkthedeploymentanduse ofhealthITwithinthefiveNordiccountries[1].AlbeittheNordic surveysshareseveralcommonvariablesabouttheusabilityofthe systems,they arenottied tospecificfunctionalities ortypesof informationcollected[1].Usabilitydatahasbeencollected com-prehensivelywithnationalusabilitysurveysonlyinFinlandandin Iceland[1].InFinland,thefirstusabilitysurveyforphysicianswas conductedin2010[4]andtheIcelandicsurvey,conductedin2014, wasbuiltbasedontheFinnishsurvey[1].
2.1. Nationalusability-focusedquestionnairestudyinFinlandin 2010
Thenationalquestionnaire studyaimedtostudyphysicians’ experiencesof use, usabilityand developmentclinical informa-tion and communication (ICT) systems, particularly EHRs, and therebyprovidegeneralizedpictureabouttheadvantages, prob-lems,andchallengesthatwererelatedtothesesystems.Forthe studyatailoredusability-focusedquestionnairewasdesignedby amultidisciplinarygrouporresearchers[4].Usabilityquestions inthequestionnairewerederivedfromtheconceptualizationfor usabilityofclinicalICTsystems(describedin[4]).Thequestions addressedvariousaspectsofclinicalICTsystemusefroma physi-cian’sviewpoint:
-Compatibilitybetweensystemsandphysicians’tasksincluding statementsaboutkeyfunctionalities,efficiencyofuse, intuitive-nessofEHRuserinterface
-Systemsupportforinformationexchange,communicationand collaborationinclinicalwork
-Integrationandinteroperabilitybetweenthesystems,aswellas reliabilityandtechnicalfunctionality[4].
Inaddition,theusabilityquestionsreflectedvarioususability aspects,forinstance
-Efficiencyofconductingroutinetasks
-Physicians’abilitiestoutilizekeyfunctionalitiessuchassummary views
J.Kaipioetal./InternationalJournalofMedicalInformatics97(2017)266–281 269
-Learnabilitywithrespecttotherequiredamountoftraining -Safetyorerrors(asinlowerrorrates)ofmedicationordering
pro-cessesandseriousadverseeventsforthepatientcausedbyfaulty EHRsystemfunctions.
Mostoftheusabilityquestionswereformulatedasstatements. Thequestionnairealsoincludedasummativequestionaboutthe overallrating(socalledschoolgrade)tothephysician’sprincipally usedEHRsystem.
Altogether3929physiciansactivelyworkinginpatientcarein thepublicandprivatesectorsrespondedtotheweb-based ques-tionnairein2010[4].
Results from the study have been reported in several pub-lications [4,9,41,45–49]. The main usability findings were that physicians’estimatesabouttheusabilityoftheirclinicalICT sys-temsindicatedthatthey werecriticaloftheir system’susablity
[4]. The overall grades given to EHR systems varied signifi-cantly: scores for systems used in private sector were higher thanfor those used in public hospitalsand healthcare centres. In general, the physicians indicated that the systems lack a properpatientoverview/dashboardandthey shouldbetter sup-portroutinetasks,decisionmaking,preventionofmedicalerrors, cross-organizationalinformationexchangeandcommunication,as wellascollaborationbetweenphysicians,nursesandpatients[4]. Theresultsalsoshoweddifferencesbetweenresponsesfrompublic hospitals,publichealthcarecentresandprivatesector organisa-tions[4,48].Physiciansworkinginpublichospitalsandinwards weremorecriticalthantheircolleaguesinotherorganisations[48]. Thequestionnairestudyforphysiciansconductedin2010canbe seentohaveanimpactonmonitoringuserexperiencesoneHealth: thesurveytoolandresultshavebeenexploitedwithinFinlandas wellasinternationally.InFinland,thestudyresultspromoted dis-cussionsamongvendors,ministry,healthcare organisationsand researchinstitutesabouthow tobetterunderstand thecurrent challengesandenhancecollaborativeactionsinimprovingthe sit-uation.ThestudyhasalsoimpactedtheneweHealthstrategyin Finland[57].InFinlandthereisacontinuedcommitmentto mon-itoringusabilityandend-userexperiences.Suchsurveyswillbe continued,andextendedfromphysicianstonursingstaff.Many oftheFinnishsurveyvariableshavebeentakenasabenchmark intheNordiceHealthindicatorwork[58].Icelandusedidentical variablestomonitoruserexperienceswithindifferentstakeholder groupsin2014.Futureworkincludesestablishingapermanent sys-temforgathering,analysisandpublicationofresultsofthecommon benchmarkingvariables[59].
2.2. LiteraturereviewonquestionnairestudiesonEHRusability Basedonourpriorquestionnairestudyin2010(describedin Section2.1)andrelatedreviewofliterature[4],theassumption wasthattherearenotmanymonitoringstudiesaboutEHRusein thehealthinformaticsfield.Tocomplementourknowledgeon,we conductedaliteraturesearchtofindrecentlypublishedacademic articlesonusabilitysurveyspublishedafter2009.Ouraimwasto findarticlesreportingwide-scalesurveysonclinicians’experiences withusabilityofEHRsystemswhichwecouldcomparetheresults ofourcross-sectionalstudywith.
Articles were searchedfollowing a scopingreview approach fromPubMedusingthekeywords:“usability”and“questionnaire” combinedwith“EHR”;“EPR”(abbr.ofelectronicpatientrecord); “healthinformation exchangesystem”; and “national”.In addi-tion;searchterms “longitudinal+study+usability”wereusedto specificallysearcharticlesreportingusabilityrelatedlongitudinal; cross-sectional;and follow-upstudies.Theinclusioncriteriafor articleswasthefollowing:
-publishedbetween2010and2015,inEnglish
-studiesusabilityofhealthcareITsystemsinuseorrecently imple-mented
-focusesonlargehealthinformationsystemslikeEHRs(notina mobileapplicationorasmallpartofalargerITsystem)
-studiesusabilityfromtheperspectiveofclinicians’experienceon use
-reportsanempiricalstudyincludingdatagathering
-reportsa study,in whichsurveysareusedasanindependent method(e.g.notcomplementingusabilitytesting).
Intotal163citationswerereturned.Thetitlesofallthefound citationswerereviewed.Thereviewwasconductedbythefirst authorofthispaper.Ninearticlesmetthecriteriabasedontheir titleandabstractreview.Twoofthesearticlesreportedresultsfrom ourpriorquestionnairestudyin2010inFinland(Refs.[4]and[49]) andwerenotincludedinfurtherin-depthreview.Thesummary ofthe remainingsevenarticlesispresented in Table1.Twoof thearticlesfocusedonclinicalITsystems(includingEHRs)inuse
[64,69],oneonhealthinformationexchange(HIE)system[61]and oneonanorderentrysystem[70],whereasthreestudieswereon theimplementationofEHRsystems[60,63,66].Thestudiesapplied variouskindsofquestionnaires(e.g.Avl[60],IsoMetrics[65])and themesofquestionsrangedfromgraphicallayoutandadequacyof trainingtoperceivedpatientoutcomes.Thenumberofrespondents perstudyvariedfrom32toabout1000.
Based onthereviewit appearsthat long-termresearchand follow-upstudiesofusabilityofEHRsystemsinusearescarce.Only theGermanstudy[64]reportedresults,whichcanbeusedtoget anoverviewofthecurrentsituationofusabilityandcomparea numberofassessedsystems.Thestudycovereda widerangeof ITsystemsinhospitalsandwastargetedtodifferentusergroups: clinicians,radiologistsandlaboratorypersonnel,aswellas admin-istrative,managementandIT-employees.
EHRimplementationstudies[60,63,66]showedthatclinicians weresatisfiedwiththeEHRingeneral,butdissatisfiedwithEHR usability[63].Factorsrelatedtosystemdesignhadstrongeffectson acceptance,evenoneyearafterimplementation[66].Skilled clini-ciansmaybeabletoovercomeusabilityobstaclesbutthisrequires thattheyaregiventheproperresources,educationandtraining
[60].A four-yearstudyof ITsupportedclinicalpathways found thattheend-users’estimatesofusabilitywererathersatisfactory: onafourpointLikert-scale(1=bad,2=acceptable,3=good,and 4=verygood)theaveragewascloseto2.5.Basedonanalysis,the subjectiveestimatesslightlyimprovedtwotofouryearsafterHIS softwareimplementation,however,nostatisticalsignificant dif-ferenceswerefoundinthetimecourse[69].InregardtoHIE,the findingsinastudyinUSAconcerningtheusersatisfactionon inter-facedesignshowencouragingresults[61]:Allthe35itemsofQUIS (QuestionnaireforUserInteractionSatisfaction)hadscoresover 50%,whichwereaboveneutral(aQUISscoreof5isneutralanda scoreof>5favourable)whilethemeanscorewas6.5.
3. Methods
Theaim ofournational questionnairestudy wastoexplore Finnishphysicians’experienceswiththeuseoftheircurrent clini-calITsystems,particularlyEHRsystems.Theideawastoutilizethe usability-focusedquestionnairedesignedforthestudyin2010to gatherrepeateddataandtofindouttowhatextenttheoverall sit-uationregardingusabilityhaschangedandwhatdifferencesthere arebetweenhealthcaresectors.
ThebackboneEHRsystemsincludedinthisstudyarebasedon anarchitecturewherecorepatientinformation,narrativepatient recordtextsfrommedicalspecialitiesandadministrativehospital
Table1
Summaryofarticles. Authors/Yearof
publication/Referencenumber
HISconcerned Researchmethods Themes/questionsinthe
questionnaire
Numberofrespondents
Janols,Lind,Göranssonand Sandblad,2014[60]
Deploymentofthreemodules ofEPRsystemsinSweden
AvalidatedquestionnaireAvl wasoneoftheusedmethods
16questionsonsystem development,usage,utility, competence,stressandhealth, relations
Physicians,nurses,clerks(N notreported)
Gadd,Ho,Cala,Blakemore, Chen,FrisseandJohnson,2011 [61]
HIEsysteminuseinUSA Selecteditemsfromavalidated instrumentQuestionnairefor UserInterfaceSatisfaction (QUIS)[62]
Themes:Overallreactions, screen,terminology andsysteminformation, learning,systemcapabilities, systemfunctionality
165physicians,nursesand others(70%
responserate)
Sockolow,Weiner,Bowles, AbbottandLehmann,2011[63]
RecentlyimplementedEHRin USA
Cliniciansatisfactionsurvey wasoneoftheusedmethods inthestudy
22questionsonimpactof theEHRonclinician satisfactionwithteam communicationandperceived patientoutcomes
37+32physicians(11and17 monthspostimplementation, responseratesof95%and82%)
Bundschuh,Majeed,Bürkle, Kuhn,Sax,Seggewies,Vosseler andRöhrig,2011[64]
ClinicalITsystemsinGerman hospitals
Web-basedquestionnaire, basedonIsoMetricsinventory [65]
37questionsonsuitabilityfor thetask,suitabilityforthe learning,conformitywithuser expectations,effectiveness
1003respondents(including 658cliniciansand73 non-bed-sidemedical personnel)from158hospitals (11%responserate) Carayon,Cartmill,Blosky,
Brown,Hackenberg, Hoonakker,Hundt,Norfolk, WetterneckandWalker,2011 [66]
RecentlyimplementedEHR systeminintensivecareunits (ICU)inUSAregionalmedical centre
Surveybasedonestablished instrumentstomeasure technologyacceptance,EHR usability,andEHRusefulness [62,67,68]
Themes:Overallacceptanceof theEHRtechnology,perceived usability,perceptionsof usefulness,information receivedbytheend-users aboutEHRimplementation, participationin
implementationactivities
121+161nurses(3and12 monthspostimplementation, responseratesof51%and72%)
Schuld,Schäfer,Nickel,Jacob, SchillingandRichter,2011[69]
IT-supportedclinicalpathways inaGermanhospital(HIS softwareincludinge.g.patient datamanagementsystem)
Annualsurvey2006–2009 (interviewsandstandardized questionnaires)toresearch staffsatisfaction
Themes:comprehensibility, usabilityandgraphicallayout (thearticledoesnotdescribe detailsoftheuser questionnaire)
Physiciansandnurses(Nnot reported)
Tan,FloresandTay,2010[70] Orderentrysystemusedin hospitalsinSingapore
Asurveytooldesignedforthe purposesofthisstudy
16questionsonreliability, speedofthesystems,easeof use,adequacyoftraining, impactonproductivity,impact onpatientcare,overall satisfaction
52physiciansandnurses(52% responserate)
Abbreviations:
EHR=Electronichealthrecord. EPR=Electronicpatientrecord. HIE=Healthinformationexchange. HIS=Healthinformationsystem.
informationsysteminformationareavailableindigitalformatvia acommonuserinterface.Thesameuserinterfacegivesaccessto localmedicationdata,nursingdocumentsandadiagnosishistory. DependingontheEHRsystem,radiologyimagesandlaboratory dataarestoredinaseparatedatabase,butusuallytheyareretrieved byexchangingthepatientcontextandseenthroughthemainEHR system.Alsothecomputerizedorderentriesaremanagedinthe mainEHR.Therearesomeelectronicdepartmentalsystems typ-ically in intensive care, operation theatre or emergency rooms whicharenotintegratedtothemainsystem.Virtuallyall docu-mentationinFinnishhealthcareisnowadayselectronic[42].
Therespondentswereaskedtogivetheirassessmentsin rela-tiontothebackboneEHRsystemtheyhadchosenearlierinthe questionnaire.Thetradenamesofthesesystemsandtheir geo-graphicaldistributionhadremainedthesamebetweenthestudies in2010and2014.Also,thebasicarchitectureanduserinterface concepthadremainedthesame.Inpublichospitalsandin pub-licprimaryhealthcarecentressomefunctionsoftheEHRsystem hadchangedfrom2010:theyincludedtheadditionofamodule toconnectwiththenationalelectronicprescriptiondatabase.Also, thisbasicconnectivitytothenationalhealthinformationexchange (HIE)hascausedmodificationstotheinternalstructureofthe soft-ware:stronguserauthenticationwithanationalsmartcardwas takenintouseandmanycodes andclassificationswereunified. However,fullconnectivitytothenationalpatientrecordarchive
wasstillaheadas itwasplannedfortheyears2014and 2015. AmongprivateprovidersofEHRsystems,thesechangeshadnot yettakenplaceatthetimeofthesurvey.Forsoftwareproviders, thistransitionperiodinconnectivitytothenationalHIEhad prob-ablytakenalotofdevelopmentresourceswhichotherwisecould havebeenusedforuserinterfacedevelopment.
3.1. Questionnaire
Theusability-focused questionnaire included18 background questions,38corestatements withafive-point Likert-scaleand a questionfortheoverall ratingof theEHR-systeminuse(see Appendix1inSupplementarymaterial).Inaddition,therewere other groups of questions addressing issues of management, patientsafety,workwell-being,informationsystemsdevelopment, EHRfeaturesthatareworkingwellorareconsideredasthemost importantdevelopmenttargets.Thephysicianswereinstructedto answerquestionsabouttheEHRsystemtheyprimarilyuseinthe contextoftheirdailywork.Outof38corestatements,24usability statementswhichwereidenticalin2010and2014wereselected foranalysis inthis study.Based onearlier experiencewiththe 2010responsessomequestionswhererephrasedtoincrease clar-ity.Sincethesequestionsmaynowcarryadifferentmeaning,they havenotbeentakenintoaccountintheanalysis.
J.Kaipioetal./InternationalJournalofMedicalInformatics97(2017)266–281 271
3.2. Datagathering
ThedatawasgatheredfromFebruarytoMarch2014byusinga web-basedquestionnaire.Theindividuallinkstothequestionnaire weresentviae-mailtoallphysicianswhowerecurrentlylivingin Finlandandwhowereundertheageof65.Thee-mailaddressesof thestudypopulationwereobtainedfromtheregisteroftheFinnish MedicalAssociation,whichcoversmorethan90%ofactive physi-cianaddresses.Theresearcherswerenotabletoidentifyindividual respondents.
The invitationtothesurvey wasemailedtoall workingage physiciansinFinland. Theactualtargetgroup,physicians doing clinicalwork,washighlightedinthee-mailmessage.Basedonthe FinnishMedicalAssociation’sregister,therewere16350 working-agephysiciansengagedinclinicalworkin2014[52,54]whilethe correspondingfigurein2010was14411.Thenumberof respon-dentswas3781(23%)in2014and3929(27%)in2010.Inthe2014 surveysomeoftherespondentsweredifferentfromthoseinthe 2010survey(i.e.respondentswereatleastpartiallydifferent indi-vidualsineachsurvey).
Therewasapossibilitytochoose“other”asamainEHRsystem iftherespondentdidnotconsideranyoftheoneslistedastheir pri-maryEHRsystem.Thereweredozensofsmallerbrandswithless than25respondentseach.Theresponseswereanonymizedafter therespondentshadusedthelinksentbytheFinnishMedical Asso-ciation.Inordertomakethesituationmorecomparablebetween 2010and2014theresearchersdecidedtodiscardthebrandswith theleastnumberofphysicianusersbecausewecouldnottrack thechanges thathaveoccurredin eithertheuserorganisations andpractices,usergroupsortheEHRsystemsthemselvesbetween 2010and2014.
3.3. Analysis
Ananalysiswasconductedofuserresponsestothe24 usability-relatedstatementsthatwereidenticalinthe2010and2014survey. Sincetherespondentswerenotidentifiedandinordertomakethe resultsmorecomparable,onlytherespondentsthathadreplied aboutusingtheninemostcommonlyusedEHRsystembrandswere included.Thisresultedintheexclusionof706respondentsfrom 2010and700from2014data(seeFig.1).
The following modifications to the original data were per-formed:
1)Theoriginalscaleofoverallratingswasfrom4to10,which followsthetypicalscaleusedatFinnishschools(4meaning‘fail’ and10meaning‘excellent’)andthereforewasfoundintuitive fortheFinnishrespondents.Thescalewaschangedtobefrom1 (‘fail’)to7(‘excellent’)forthepurposesofpresentingtheresults toaninternationalaudience.
2)Thefive-pointLikert-scaleanswers‘Fullyagree’and‘Somewhat agree’werecombinedtoformthecategory ‘Agree’.Similarly theanswers‘Fullydisagree’and‘Somewhatdisagree’were com-binedtoformthecategory‘Disagree’.
Forcategorical variablesthestatisticalanalyseswerecarried outwithChi-squaretestsorFisher’sexacttest,whenapplicable. Continuousvariableswerecomparedusingaone-wayanalysisof variance.StatisticalsignificancewasdefinedasP<0.05.The sta-tisticalanalysiswasconductedusingSPSS22software(IBMCorp, Armonk,NY).
4. Results:changesbetween2010and2014
Theresultsaredividedintothreesections:respondents’ demo-graphics2010and2014(Table2),overallratingsforEHRsystems (Table 3), and responses to usability statements presented by healthcare sector: public hospitals (both outpatient and inpa-tient),publicprimaryhealthcarecentres(outpatient)andprivate providers(mainlyoutpatient)(Table4).
4.1. Respondentdemographics
Thedemographicsoftheusersoftheninemostcommonlyused EHRsystembrandsaredescribedinTable2whichshowssimilar demographic features ofoursubset ofphysicians withthe tar-getpopulation(describedin [4]and[53,54]).Theproportionof femalesamongrespondentshadincreasedbetween2010and2014. Theyoungestagegroupofrespondentshadalsogrowninsizein bothsurveys.Correspondingchangescouldalsobeseeninthe tar-getpopulation.Theproportionofdifferentworkingsectorshad remainedsimilar.
Fig.1.Selectionofdatafortheanalysis.
*EHRbrandsthatwereusedbylessthan25respondentsortherespondentsdidnotdefinethebrand. **r.g.GovernmentsectororUniversityorundefined.
Table2
Respondents’demographicsinthe2010and2014surveys.
2010 2014 PhysiciansinFinland
2014under65years [52]
p-valueforchange between2010and 2014
Numberofrespondents inthewholesurvey
3929 3781 Numberofrespondents inthisstudy 3223 3081 Numberofphysicians inFinlandunder65 years 18933 20110 n(%ofrespondentsin thestudy) n(%ofrespondentsin thestudy) % Gender <0.001 Female 1840(57) 1909(63) 58 Men 1370(43) 1130(37) 42 Age <0.001 <35years 454(14) 633(21) 21 35–45years 797(25) 669(22) 25 46–55years 1156(36) 841(28) 29 >55years 785(25) 873(29) 25 Healthcaresector 0.232 Publichospital 1807(56) 1667(54) 53 Publichealthcare centre 894(28) 875(28) 26 Privateproviders 522(16) 539(18) 21 Table3
Physician’soverallratingswithscalefrom1(or‘fail’)to7(or‘excellent’)fortheirprincipallyusedEHRsystems(N>30).In2014thephysicianshadtogiveanoverallrating scaleorchoose“Idonotwishtoanswer”.Theproportionsofdifferentbrandsweresimilarbetween2010and2014(p=0,232).
EHRsystem 2010 2014 pfordifference
between2010and 2014 Respondents (total=3223)N(%) Respondents havinggivena score(total=2041) N(%) Meanopinion score(scale1–7) (std.deviation) Respondents (total=3081)N(%) Respondents havinggivena score(total=3057) N(%) Meanopinion score(scale1–7) (std.deviation) Publichospital K 218(7) 148(7) 4.2(1.2) 195(6) 194(6) 4.3(1.2) 0.341 Lh 75(2) 51(2) 2.5(1.3) 47(2) 46(2) 4.2(1.3) <0.001 Mh 462(14) 289(14) 3.7(1.1) 432(14) 428(14) 3.5(1.2) 0.043 Nh 102(3) 60(3) 3.1(1.2) 107(3) 106(3) 3.4(1.7) 0.186 O 950(29) 580(28) 3.1(1.2) 886(29) 881(29) 3.2(1.2) 0.295
Publichealthcarecentre
P 44(1) 31(2) 3.9(1.4) 37(1) 37(1) 4.4(1.0) 0.092 Lc 50(2) 34(2) 3.9(1.5) 71(2) 71(2) 4.0(1.2) 0.952 Mc 439(14) 274(13) 4.1(1.1) 424(14) 419(14) 3.6(1.2) <0.001 Nc 361(11) 231(11) 3.2(1.2) 343(11) 343(11) 3.4(1.1) 0.031 Privateproviders Q 375(12)2 249(12) 4.2(1.0) 379(12) 373(12) 4.3(1.1) 0.324 R 87(3) 60(3) 4.3(1.0) 96(3) 96(3) 4.0(1.3) 0.097 S 60(2) 34(2) 3.5(1.1) 64(2) 63(2) 3.5(1.3) 0.937
4.2. Physicians’overallratingsoftheirEHRsystems
Table3presentstheEHRsystemsusedandtheircontextofuse, aswellasthemeanopinionscoresin2010and2014onascale from1(‘fail’)to7(‘excellent’).Theaverageoftheratingsvaried from3.2to4.4in2014(from2.5to4.3in2010).ThreeoftheEHR systems(systemslabelledL,MandN)areusedbothinpublic hospi-tals(denotedwithasubscript‘h’)andinpublichealthcarecentres (denotedwiththesubscript‘c’).Theratingswereratherlow in 2010,especiallyforthehospitalEHRsystems.Ingeneral,themean opinionscoresgivenin2014accordwithourearlierstudy find-ings.However,somechangesarealsoshown:theaveragerating hadimprovedforthesystemsLh(from2.5to4.2)andNh(from
3.1to3.4),whereasthemeanopinionscorehaddecreasedforthe systemM(Mhfrom3.7to3.5andMcfrom4.1to3.6).
4.3. Responsestousabilitystatements:2010comparedto2014
Table4presentsthesummaryofthefindingsforeach24 usabil-itystatement.Thetableshowschangesbetween2010and2014 ‘agree’responsesbyhealthcaresector.Thesimilaranalysisof ‘dis-agree’responsesofthesamedataisavailableasSupplementary material(Appendix2inSupplementarymaterial).
Analysisof‘agree’responsesfrom2010and2014shownotable improvementsinthefollowingEHRfunctionalitiesand character-istics:
-Helpinpreventingerrorsandmistakesassociatedwith medica-tions(allagree(%)from28to41)(statement1)
-Summaryviewonpatient’shealthstatus(allagree(%)from14to 24)(statement2)
J. Kaipio et al. / International Journal of Medical Informatics 97 (2017) 266–281 273 Table4
Summaryofquestionnaireitems(usabilitystatements)andphysicians’responses:Changesbetween2010and2014forAGREEresponsesbyhealthcaresector.Theboldnumbersrepresentthe”best”responseinthetwoyears thatarecomparedandreflectwhetherthesignificantchangeisanimprovementornot.
Itemno. Statement Publichospitals
AGREE(%of respondentsin workingsector) (N=3474) p-value (between 2010and 2014) Publichealthcare centresAGREE(% ofrespondentsin workingsector) (N=1769) p-value (between 2010and 2014) Privatesector AGREE(%of respondentsin workingsector) (N=1061) p-value (between 2010and 2014)
AllAGREE(%) p-value (between 2010and 2014) Totalnrof responsesto statement 2010 2014 2010 2014 2010 2014 2010 2014
Totalnrofrespondentsinyear andworkingsector
1807 1667 894 875 522 539 3223 3081 6304
1 Informationsystemshelpin
preventingerrorsandmistakes associatedwithmedications.
19 38 <0.001 46 52 0.005 30 34 0.136 28 41 <0.001 6203
2 TheEHRsystemgeneratesa
summaryview(e.g.ona timeline)thathelpstodevelop anoverallpictureofthe patient’shealthstatus.
13 28 <0.001 14 18 0.137 17 17 0.975 14 24 <0.001 3970
3 Informationsystemshelpto
improvethequalityofcare.
30 33 0.022 38 40 0.308 37 42 0.120 33 37 0.003 6215
4 Routinetaskscanbeperformed
inastraightforwardmanner withouttheneedforextra stepsusingthesystem.
28 25 0.092 36 25 <0.001 56 54 0.238 35 30 <0.001 6265
5 Thesystemrespondsquicklyto
inputs.
36 26 <0.001 47 34 <0.001 72 67 0.065 45 36 <0.001 6262
6 Thearrangementoffieldsand
functionsislogicalon computerscreen.
36 42 0.001 44 40 0.173 58 63 0.090 42 45 0.008 6252
7 Thesystemkeepsmeclearly
informedaboutwhatitisdoing (forexamplesavingdata).
28 29 0.558 44 34 <0.001 45 47 0.451 35 34 0.164 6253
8 Terminologyonthescreenis
clearandunderstandable(for exampletitlesandlabels).
40 39 0.459 54 45 <0.001 60 59 0.636 47 44 0.012 6254
9 Learningtousetheelectronic healthrecordsystemdoesnot requirealotoftraining.
38 37 0.253 37 24 <0.001 61 50 <0.001 42 36 <0.001 6270
10 Theinformationinthenursing recordisineasilyreadable.
27 26 0.400 35 34 0.431 47 48 0.886 33 32 0.391 5981
11 Thepatient’scurrent
medicationlistispresentedin aclearformat.
J. Kaipio et al. / International Journal of Medical Informatics 97 (2017) 266–281 Table4(Continued)
Itemno. Statement Publichospitals
AGREE(%of respondentsin workingsector) (N=3474) p-value (between 2010and 2014) Publichealthcare centresAGREE(% ofrespondentsin workingsector) (N=1769) p-value (between 2010and 2014) Privatesector AGREE(%of respondentsin workingsector) (N=1061) p-value (between 2010and 2014)
AllAGREE(%) p-value (between 2010and 2014) Totalnrof responsesto statement 2010 2014 2010 2014 2010 2014 2010 2014 12 Informationonmedications
orderedinotherorganisations iseasilyavailable.
5 9 <0.001 5 12 <0.001 4 9 0.002 5 10 <0.001 6142
13 Obtainingpatientinformation
fromanotherorganisation oftentakestoomuchtime.
76 81 <0.001 75 75 0.991 72 79 0.005 75 79 <0.001 6143
14 Informationsystemshelpto
ensurecontinuityofcare.
37 42 0.002 46 47 0.977 46 54 0.012 41 45 <0.001 6233
15 Informationsystemssupport
collaborationandinformation exchangebetweenphysicians workinginthesame organisation.
67 63 0.312 70 68 0.299 67 68 0.785 65 65 0.673 6258
16 Informationsystemssupport
collaborationandinformation exchangebetweenphysicians workingindifferent organisations.
17 17 0.564 15 19 0.010 5 9 0.016 14 16 0.016 6201
17 Informationsystemssupport
collaborationandinformation exchangebetweenthe physicianandthenurses.
34 45 <0.001 57 60 0.196 48 48 0.992 43 50 <0.001 6203
18 Thesystemmonitorsand
notifieswhentheordersgiven tonurseshavebeencompleted.
14 18 0.010 5 10 0.008 3 7 0.084 11 15 <0.001 3947
19 Informationsystemssupport
collaborationandinformation exchangebetweenthe physicianandthepatients.
12 9 0.004 9 9 0.889 21 20 0.388 12 11 0.024 6106
20 Useoftheinformationsystems
frequentlytakesmyattention awayfromthepatient.
71 76 <0.001 67 71 0.056 50 58 0.008 66 71 <0.001 6246
21 Thesystemisstableintermsof technicalfunctionality(does notcrash,nodowntime).
46 41 0.001 54 46 <0.001 73 72 0.680 53 48 <0.001 6283
22 Information
entered/documented occasionallydisappearsfrom theinformationsystem.
34 28 <0.001 27 25 0.259 27 21 0.015 31 26 <0.001 6225
23 IfIhaveproblemswiththe systemIcaneasilygethelp.
45 45 0.913 52 45 0.005 61 60 0.826 49 47 0.139 6246
24 Faultysystemfunctionhas
causedorhasnearlycauseda seriousadverseeventforthe patient.
J.Kaipioetal./InternationalJournalofMedicalInformatics97(2017)266–281 275 -Theformatofpatient’smedicationlist(allagree(%)from24to
37)(statement11)
-Supportforcollaborationandinformationexchangebetweenthe physicianandthenurses(allagree(%)from43to50)(statement 17).
Incontrast,theanalysisshownotablenegativechangesinthe followingEHRfunctionalitiesandcharacteristics:
-Quickresponsetoinputs(allagree(%)from45to36)(statement 5)
-TheamountoftrainingrequiredtolearntousetheEHR(allagree (%)from42to36)(statement9).
Overall,theportionofrespondentsagreeingwiththepositively formattedusabilitystatementsisratherlow.Thefollowing find-ingscanbepointedoutasexamples.TheEHRsystemsstilllack summaryviewsordashboards(inallsectorsonly17–28%ofthe respondentsagreedwiththestatementabouttheEHRgenerating anappropriatesummaryview),althoughinhospitalEHRsystems thesituationhadimproved(from13%to28%).ITsystems’ability tosupportcollaborationcontinuetoberatedaspoor:Physicians inallsectors weredissatisfiedwiththesupportforinformation exchangebetweenprofessionalsworkingindifferentorganisations (16%ofall agree, statement16) aswellas betweenthe physi-ciansandpatients(11%ofallagree,statement19).Thisfindings isstronglysupportedbytheanalysisof‘disagree’responses(see Appendix2inSupplementarymaterial),whichshowthatthe por-tionofdisagreeanswershasgrowninallsectorswhencompared toyear2010(inpublichospitalsfrom59%to63%;inpublic health-carecentresfrom60%to65%;inprivatesectorfrom41%to50%). Furthermore,allphysicianscontinuesharingtheirconcernsabout ITsystemusetakingtimeawayfromanddisruptingcaregiving activities.Threeoutoffourphysiciansinpublicsector organisa-tions(76%inhealthcarecentresand71%inhospitals)andmorethan half(58%)intheprivatesectorbelievedthattheuseofthesystems frequentlytakestheirattentionawayfromthepatient(statement 20).
Theanalysisshowsomedifferencesbetweenresponsesfrom differenthealthcaresectors.Comparedtoresponsesfromthe pub-lic sector,physicians workingin the privatesector were more satisfiedwiththeirEHRsystemswithregardstosupportfor rou-tinetasks(statement4),responsivenesstoinputs(statement5), intuitivenessoftheEHRuserinterface(statements6–8),andthe requiredamountoftraining(statement9).Incontrast,the assess-mentsbyphysiciansworkinginpublichealthcarecentrestowards theseaspectshadbecomemorecriticalthanin2010.Onlyinthe privatesector,themajorityofphysicians(72%in2014)seemto besatisfiedwiththeir EHRsystemswithregardstostability of thesystems(statement21).Publicsectorusersweremoreeven moredissatisfiedthanin2010(physiciansinpublichospitals41% andinpublichealthcarecentres46%in2014).Onthestatement of‘faultyfunctionshavecausedornearlycausedaseriousadverse eventforapatient’(statement24)8%ofphysiciansintheprivate sectoragreedwiththestatementwhereastheportionof physi-ciansworkinginpublichospitalsandinpublichealthcarecentres andagreeingwiththestatementremainmuchhigher: 42%and 31%.
In general, the analysis of ‘agree’ responses (presented in Table 4)and ‘disagree’ responses (presented in Appendix 2 in Supplementary material) show similar patterns. The compari-sonofthefindings,particularly“best” responsesperhealthcare sector,indicate onlysome small differenceswhich are marked inAppendix2 inSupplementarymaterial.Mostofthese differ-encessupporttheconclusionsmadebasedonanalysisof‘agree’ responses.
5. Discussion
5.1. Themaincontributions
Inthispaper,wereportFinnishphysicians’experienceswiththe usabilityofcurrentlyusedEHRsystems,aswellasonthechanges intheirperceptionsbetween2010and2014.Overall,theresults indicatethesituationhasnotimproved.Physicians’assessments oftheirEHRsystemsstillindicateinadequacies:onascale1–7the averageoftheratingsvariedfrom3.2to4.4.Incontrast,therewere markeddifferencesbetweentheEHRsystembrandswithineachof theusercategories(privatesector,publichospitalsandprimary healthcare).Althoughthesituationhadnot improved consider-ablyingeneral,twointerestingchangesinphysicians’assessments arose:vendorLhwasgiventhemostnegativeassessmentsin2010,
butwasamongthebestin2014,andtheassessmentsofoneof thebiggestvendors(vendorM)haddeterioratedoverafouryear period.Wespeculatethatthereasonsbehindthesechangesarethe following:In2010LhhadreplacedthepreviousEHRsystemonly
threemonthspriortothestudy.Thiswasthebrand’sfirst imple-mentationin a hospital environment. The reason behindmore positiveassessmentscouldhavetheiroriginsinsuccessful develop-mentoftheproduct,unsuccessfulinitialimplementationprojects, orboth.Itisunlikelythatinitialphysicianchangeresistancewould havebeentheonlyexplanationforthecriticalassessmentsof2010. Importantly, most of theusers are within one hospital district where healthcare centresand hospitalsusethesame EHR sys-temandthebenefitsofsharingdatacouldexplainatleastpart oftheincrease inusersatisfaction.ForbrandMthesituationis morecomplex.Oneofthereasonscouldbethatintegrationwith nationalhealthandinformationsystemserviceshasrequiredmajor revisionsoftheEHRsystems’functionalities,andthismayhave disrupted physicianworkflows. In particular,thee-prescription functionalityhas evokedmorecriticismthanwithotherbrands (datanotshown).AtleastfourchangesintheITenvironmentcould explainthelackofimprovementinusers’opinionsabouttherest oftheEHRbrands:Firstly,thenationaleHealthservices(Kanta ser-vices)haverequiredmajorrevisionstoalloftheEHRsystems,and thesecouldhavecausedahaltinallotherdevelopments.Secondly, theimplementationoftheserequirementsmayhavenotimproved theusabilityofthesesystems.Thirdly, theexpectedbenefitsof theKantaservices(standardizationofdataandsharing informa-tion)hadnotbeenrealizedasyetatthetimeofoursurvey;only e-prescriptionhadbeenwidelyimplemented.Fourthly,otherIT solutionsthattherespondentsuseintheirdailyliveshave devel-opedduringthepastfouryears;theEHRsystemsseemtolagbehind inthisdevelopment.
Physiciansshouldbeabletostartgettingtheworkdoneinthe wayitissupposedtobedonewithouterrors.Therefore,the com-ponentsofuserinterfacesthatprovideinformationaboutcontrols fortheusershouldbeimmediately understandable.Theresults showthatlessthanhalfofallrespondentsagreedwiththe state-mentsaboutintuitivenessofEHRuserinterfaces.Thesefindings reflectexperiencedlearnabilityandsuccessofuserinterfacedesign fromthephysicians’viewpoint.Albeittheend-usersweremore experiencedinusingEHRsystems,theyexperiencedanincreasein thetimerequiredtolearntousethesystems.Ontheotherhand, clinicalworkpracticestendtobecomemoredependentondigital processesovertheyears.Atthesametimelessexternalguiding informationisavailableoutsidetheinformationsystems.
Itisalsoeasytoseethelinkbetweenintuitivenessanderror rateorsafetyofITuse.Thephysicianhasabetterchancetoavoid technology-inducederrorsandmistakeswhenusingtheEHR sys-tem,ifthesystemincludesfunctionalitytopreventerrors,ifthe systemkeepstheuserclearlyinformedaboutwhatitisdoing,and iftheusercaneasilygethelpwhenneeded.Inparticular,the
find-ingthatonethirdofallrespondents,andevena higherportion (42%)ofrespondentsfrompublichospitals,hadexperiencedthat afaultysystemfunctionhascausedorhasnearlycausedaserious adverseeventforthepatientindicatesthattheuseofthecurrent EHRsystemsmayposeaseriousrisktopatientsafety.
EHRsystemsshouldbeefficienttouse,sothatahighlevelof productivityinahecticandcriticalenvironmentispossible.The surveyresultsindicatethatwiththecurrentEHRsystems physi-ciansarenotabletoconducttheirworkinanefficientway.Thisis particularlyshownbystatementsabouttheavailabilityofkey func-tionalitiesandthestatementaboutperformanceofroutinetasks. Withregardstophysicians’responsestothesestatements,twoout ofthreephysicianssharethisconcern:thesystemslackthose prop-erties thatareneededtoperformkeyclinicalcaringtaskswith patientsandthesystemsforcethephysicianstoperformadditional tasksoradaptnewinappropriateworkprocesses.
Fromtheusabilityviewpointitcanbearguedthatefficiency ofworkisalsohinderedbecauseofpoorITsupportfor informa-tionexchange,communicationandcollaboration.EHRandrelated IT systems shouldserve a singlephysician but alsotheir work withnumerousotherpartiessinceclinicalprocessesare charac-terized witha high degreeof communication and cooperation. Theresultspointoutparticularlythefollowingareasof improve-ment:availabilityofinformationaboutmedicationsprescribedat anotherorganisation,supportforcross-organizational collabora-tion betweenphysicians, monitoring of ordersgiven tonurses, and support for physician-patient collaboration. Less than 20% of allrespondents agreed withthese four positively formatted statements. It appears that after its first years of deployment, thenationale-prescription databasehasnot asyetameliorated the availability of medication information acrossorganisations. The study findings providethe baseline statusfor the national information exchange in Finland. It will be interesting to see howphysicianexperienceswillevolveinthepost-implementation situationof 2017.Based onresults, ITsystems’supportfor col-laborationandinformationexchangebetweenthephysiciansand patientshadnotimproved;apparentlyfewsolutionshaveemerged to supportpatient-centredcare and patient abilitiesto partici-pateintheirowncare.Therefore,basedontheresultsofanother nationalsurvey,theusageratesforthepatientportal functionali-tieswerestillverymodest[71],andinformationexchangebetween patientsandphysicianswasonlyavailableinafeworganisations
[42].
Satisfactionwithtechnicalfeatures hasnotimprovedinfour years. We regard theavailability ofthe EHRsystems asone of thecentralfeatureswhenthephysiciansassesstheuseofthese systems.However,theproportionofphysiciansagreeingwiththe statementconcerningthedisappearanceofdocumenteddatafrom thesystem(statement22)hadslightlydiminishedinallworking environments.Aswearenotawareofmajortechnicalfailuresin datastorageintheEHRsystemsincludedinthisstudy,we inter-pret“disappearingdata”asanindicatorofthemostsevereusability problems.Theseincludeforexample(a)theuserdoesnotknow wheretofindtheinformationneeded,(b)theuserhasdocumented it inincorrectfieldsor modulesoftheEHRsystemor,(c)most importantly,atthefinalstagesofdocumentationprocess,theuser doesnotnoticethathe/sheneedstopressanotherbuttoninorder toactuallysavetheinformation.Userstendtofindwaystomove aroundtheusabilityproblemsovertime[72,73]sowellthatthe developersandtrainersmaynotseethemasusabilityproblems at allandblame theend-usersfor nothavingattendedenough trainingsessions.
Theresultsalsoshowsomedifferencesbetweenhealthcare sec-tors.EHRsystemsusedintheprivatesectorappearedmorestable andrespondedmorequicklytoinputsaswellasprovidingbetter supportforphysicians’routinetasks.Thepatientsintheprivate
sectorarelessseverelyillandseemtohavefewercomorbidities; accordinglythelackofdashboardsorpatientoverviewsislesslikely tointerferewithroutinetasks.Theresponsesfromprimary health-careindicatednegativechangesintheseaspectsascomparedtothe year2010.Oneofthereasonsbehindthiscouldbetheincreased documentationneedsfornationalreporting(otheraugmented doc-umentationneeds aremorelocal).Theresultsalsoindicatethat physiciansworkinginthepublicsector(hospitalsandhealthcare centres)wererathersatisfiedwiththeirITsystemabilitiesto sup-portpreventionoferrorsrelatedtomedication.Healthcarecentre EHRsystembrandshadimplementedmedicationinteractionalerts alreadyby2010whereasthisfunctionalitywasintroducedtothe largestbrandusedinhospitalsonlyafterourprevioussurvey,and wasstilllackingintheprivatesector.
Theanalysisof‘disagree’responsesshowhighlysimilarpatterns withtheanalysisof‘agree’responsesandtherebysupportedthe conclusionsmadebasedonTable4.However,thecomparisonof thefindingspointoutsomedifferences(whichareallshownin Appendix2inSupplementarymaterial).Probablythemostnotable differencesarerelatedtothestatementaboutsysteminforming theuseraboutwhatitiscurrentlyperforming(statement7),the statementaboutthearrangementoffieldsandfunctionson com-puterscreen(statement6)andthestatementaboutinformation systemsupportforphysician-patientcollaboration(statement19). Inpublichospitalsandinprivatesectortheportionofphysicians disagreeingwiththestatementno.7hasgrown(inhospitalsfrom 50%to54%andinprivatesectorfrom30%to36%)eventhoughthe analysisof‘agree’responsesshowthatthephysiciansinboththese sectorshavegivenmorepositiveestimatesthaninyear2010.On thestatementno.6theanalysisof‘agree’responsessuggestthat theportionofallphysiciansagreeingwiththestatementhasgrown (from39%to42%).Interestingly,theportionofdisagreeinghasalso grownfrom42%to45%.Basedonourexperience,anadditional analysiscanbeparticularlyusefulregardingthosestatementsand resultswhichshowonlylittleornochangeatallbetweenresponses fromdifferentyears.
Oneofthecentralfindingsofthisstudyconcurswiththe find-ingsfrom earlier usability studies[60,72]: Time does not heal usabilityproblemseventhoughtimeallowsuserstolearn strate-giesforovercomingsomeoftheproblems.Kjeldskovetal.[72]
conductedlaboratory-basedusabilitytestingwithanaimto com-paretheusabilityofthesystemasexperiencedbynoviceandexpert users.Theexpertuserswerenotmoreefficientoncomplextasks anda remarkablenumberofseriousand criticalproblemswith thesystemstillremainedafteroneyearofextensiveuse.Janols
[60]reportssimilarfindings:Evenaftertwoyearsofdeployment, thesystemwasnotconsideredtobeassupportiveastheold sys-tem.Inaddition,cliniciansoftenfoundwaystoovercomeproblems withsystemusage;however,thesework-aroundstypically gener-atednewproblems.Inconclusion:poordesignremainspooreven thoughtimeallowspeopletolearnstrategiestoovercominga sys-tem’sspecificpeculiarities.
Earlierstudieshavealsoshownthatdifferentusergroups(e.g. physiciansandnurses) havedifferentjobrolesand responsibil-ities leadingto differentneeds and expectationsof the clinical systems [70]. Similarly, physicians workingin different sectors havedifferentneedsandrequirementsfortheirsystems. Special-izedinformationsystemswithdefinedfunctionalitieshavebeen reported to receive more favourable assessments than clinical informationsystemsingeneral[64].Thefindingcanbeattributed totheimprovedcustomizationofthespecializedsystemsfor spe-cificworkingenvironments[64].Itisnoteworthythatoursurvey did not cover IT systems usedtypically in intensive careunits oroperatingtheatres,and,accordingly,moreinternational stud-iesareneededtoreassessthecurrentsituationintheseworking contexts.
J.Kaipioetal./InternationalJournalofMedicalInformatics97(2017)266–281 277
5.2. Relevanceoftheresearch
Fromtheacademicliteraturewe foundonlyafew examples ofusabilitysurveyswhichhavebeenusedtoresearchthe usabil-ityofEHRsystemsinuseovertimeinrealsettings.Approaches toresearching adoption and usability of healthcare IT systems varybetweencountries[56],however,tothebestofour knowl-edgethestudiesconductedinFinlandin2010and2014arethe onlycross-sectionalquestionnairestudiesfocusedonusabilityand aimedtomonitorandfollow-upthedevelopmentofEHRsystems atanationallevel.Reliableacademicsurveysareneededtostudy theusabilityanduserexperiencesofcurrentlyusedEHRsystems andtomonitortheirdevelopmentatregionalandnationallevels. Resultsofimplementationanddeploymentstudiessuggestthat EHRfunctionalityandusabilityimpactcliniciansatisfaction, effi-ciency,andclinicaluseoftheEHR.Therefore,oneofouraimswas toinformdecisionmakers,healthcareorganisationsandpoliticians abouttheusabilityofcurrentlyusedEHRsystemsand improve-mentsthathaveoccurredduringthepastfewyears.Assuggested byBundschuhet al.[64],resultsfromnationalusabilitystudies canbeusedasreferencedataforevaluationand benchmarking ofuser-orientedsoftwareengineeringforclinicalIT,whichis rele-vantforthedevelopmentandmarketingofthesesystems,aswell asforclinicalpracticeandcarequality.Furthermore,Carayonetal.
[66]havestatedthat“itisimportantforhealthcareorganisationsto continuetheireffortstooptimizethedesignanduseofEHRafterthe technologyisimplemented,sincethecharacteristicsofEHR technol-ogy,particularlyusabilityandusefulness,haveasignificantimpacton acceptanceanduseofthetechnology”.Theyalsosuggestthatmore researchusingalong-termdesignisneededtofurtherunderstand howEHR-relatedpredictorsoftechnologyacceptance,including usability,maychangeovertime.
5.3. Historyofourusability-focusedquestionnaire
Designingausabilityquestionnairestudyforphysiciansis chal-lenging.Itrequiresin-depthknowledgeaboutusabilityresearch issuesandaboutdomainspecificcharacteristicsofphysicians’ con-textofwork.Comparedtostandardizedusabilityquestionnaires (suchasSUMI[74],SUS [75]orQUIS [62]), thestrengthofour questionnaireisthatitisfocusedandspecializedaround physi-cians’workandtheiruseofEHRsystems.Ashasbeenstatedin ourearlierarticle[4],thereasonsfornotusingthesestandardized usabilityquestionnaireswerethattheyfocusonasinglesystemof software,evaluatetheusabilityofauserinterface,andarecontext anddomainindependent.However,weutilisedthose question-naireswhentheoriginalquestionnairewasdesignedandalsowhen weupdatedtheformbeforedatagatheringin2014.
Questionnaireitemsneedtobecarefullyformulatedsothatthey arecorrectandappropriatefromtherespondent’spointofview.Itis importantforthequestionnairetooltoaddressissuesitissupposed toaddressandtobespecializedforthecontextofitsintendeduse. Forexample,Tanetal.[70]foundthatintheirstudyphysicianshad adifferentconceptofusersatisfactionascomparedtonursesand thedatacollectiontoolwithitsconceptswasmoreeffectivein mea-suringnursingconstructionsthanaphysician’smindset.Therefore, thequestionnaireneedstobecarefullypilot-testedwith poten-tialrespondents.Afterourstudy,wefoundthatsomespecialist groupssuchasradiologistsandlaboratoryphysiciansdidnotfind allstatementsrelevanttotheireverydaywork.OnlyEHRsystem brandsthatwereusedas“foundationsystems”wereavailablefor selection,therefore,physiciansworkinginintensivecareunitsor operatingtheatrescouldnotreplybasedonthespecialityITsystem brandtheyused.MostphysiciansuseseveralotherITsystemsthat storepatientinformation(suchasradiologyinformationsystems, laboratoryinformationsystems)daily.Therespondentswerenot
askedtoassessthedevelopmentofEHRsystemswithregardsto thesituationfouryearsearlier.
The design of our national usability-focused questionnaire startedin2009andhashadseveraliterationssince.Theoriginal versionof it wasdesigned bya multidisciplinarygroup,whose sevenmemberswereexpertsintheareasof usabilityresearch, medicalinformatics,sociologyoftechnology,medicineand medi-calpractices,andoccupationalhealthresearch[4].Thetheoretical background of thequestionnaire development work originated fromareviewoftheusabilityliterature,particularlyfromwidely knowndefinitions ofusabilityandananalysisofcontextofuse characteristics[4].Beforethefirstdatagatheringintheyear2010 thequestionnaire had two pilottest phases[4]. Afterthat, the questionnairewasmodifiedbasedonourexperiencesfromdata gathering,analysisandacademicdiscussions.Asdescribedin ear-lier,somestatements weremodifiedand addedfor exampleto reflectdevelopmentsatthenationallevel.Thiskindofiteration andupdateneedstotakeplaceinthefutureaswell.Furthermore, itisimportanttopilottestthequestionnaireeverytimebeforeit isusedwithsufficientnumberofpotentialrespondents.Although themodifiedquestionnaireusedin2014wastestedbeforehand, itislikely thatphysiciansfromdifferentworkingenvironments mayunderstandthestatementsdifferently.Ontheotherhand,user experienceis context-relatedand respondentsassess the state-mentsbasedtheirownexperiences.
Eventhoughourquestionnairehassuchhistory,onecould ques-tionthevalidityandthereliabilityofthemethodandtheresults.To ourbestknowledge,oursurveywithFinnishphysiciansisthefirst nationaleHealthobservatoryquestionnairefocusedonusability andusedtomonitorthelong-termdevelopment.Therefore,wefind themethodandtheresultsarenovel,valuableandofhigh impor-tance.Inaddition,theresultshavepractical relevanceandthey havebeenexploitedwithinFinlandaswellasinternationally.Our planistokeepupmonitoringthelevelofusabilityofclinicalIT sys-temsaswellastocontinuethequestionnairedevelopmentwork.At thesametime,wecallforcollaborationwithotherresearchersand communitiesaroundthisworktodevelopvalidatedandreliable academicmonitoringsurveysforhealthcareITfield.
5.4. Strengthsandweaknessesofthestudy
Theuseofquestionnairesisavaluablemethodforgathering subjectiveexperiences–directclinicalresponse–withITusage. Compared with other more qualitative usability methods (e.g. usabilitytesting,expertevaluation,observationsorinterviews)use ofquestionnaires isa suitabletechniquefor gathering informa-tionfromalargetargetgroupofend-users.Aweb-basedformat makesiteasytoreachahighnumberofdesiredrespondentsand inquireaboutnumerousIT-userelatedthemes.Forthesereasons, aweb-basedquestionnairewasfoundsuitabletobeusedinour study,whichaimedatresearchingandmonitoringtheoveralllevel ofachievedusabilityofEHRsystemsandimpactsofdevelopment activitiesatanationallevelwithinrecentyears.
The studysuffered from generic limitations typicalof stud-iesconductedwiththeinternetsurveymethod[76]:TheFinnish MedicalAssociationregisterdid nothaveemailaddressesof all physicians.Theinvitationemails maynotnecessarily reach the respondentsbecauseoffirewallsettingsorothertechnicalissues. Therefore, we can’t be surehow many physicians actually got theinvitation.Thosethatwerereachedmightnothaveanswered online surveys.Surveyswithclosed-endedquestions mayhave alowervalidityratethanotherquestiontypes.Dataerrorsdue toquestionnon-responsesmayexist.Thenumberofrespondents whochoosetorespondtoasurveyquestionmaybedifferentfrom thosewhochosenottorespond,thuscreatingbias.Surveyquestion answeroptionscouldleadtouncleardatabecausecertainanswer
optionsmaybeinterpreteddifferentlybyrespondents.For exam-ple,theansweroption“somewhatagree”mayrepresentdifferent thingsto differentsubjects, andhave itsown meaningtoeach individualrespondent.‘Yes’or‘no’answeroptionscanalsobe prob-lematic.Respondentsmayanswer“no”iftheoption“onlyonce” is notavailable.Inaddition,issues ofanonymousquestionnaire methodarerelevanttopointout.Inourstudy,wefindassuranceof respondentanonymityakeyissue.Wethinkthishasinfluencedthe responserateinapositiveway.Multipleresponsesfromasingle respondentwerenotpossiblesincepersonalizedlinksweresendto respondents.Whenusinganonymousquestionnaireitis,however, notpossibletoevaluatethecausality.
Whenestimatedaschangesinpercentagesofphysicians agree-ingordisagreeingwithastatement,ourresultsdonotshownotable improvementsregardingtheusabilityofEHRsystems.However, as thenumber ofrespondents washigh, thestatisticalmethod used(ChiSquaretest)mayoverestimatethedifferencesbetween thefindingsof2010and2014.When consideringtherelevance ofthedifferencesbetweenthefindings,itisrecommendedtopay attentiontoboththep-valuesandthesignificanceofthechange expressedaspercentagevalues.
Itisunlikelythatthechangeinrespondentdemographicswould explainthelackofimprovementsintheopinionsofphysicians:The proportionofwomenhadincreasedamongbothrespondentsand physiciansinFinlandbetween2010and2014.Womenhadgiven higheropinionscoresthanmeninbothsurveys[45,53]. Respon-dentswereyoungerin2014thanin2010,buttheopinionsscores oftheyoungestagegrouphadslightlyrisenbetween2010and2014 (datanotshown).Theproportionsofworkingsectorsanddifferent EHRsystemshad remainedsimilar.Sincetherespondentswere notidentified,someoftherespondents,atleasttheonesthathad finishedtheirstudiesorretiredafter2010,werenotthesamein 2010and2014.However,itisunlikelythatphysicianswhofeelless positivelyabouttheirEHRsystemswouldhaverespondedtothis survey,butnotthepreviousone.Ofthelargestusergroups,only onehospitaldistricthadchangeditsEHRsystembrandbetween 2010and2014,manyoftherespondentsinthissurveyarelikely tohavefouryearsmoreexperienceintheuseofthesystemsthan inthepreviousone.
ThefindingsonthecurrentstateofusabilityofEHRandrelated clinicalITsystems havenovelvalue,sincetheappliedresearch approachwasnottypicalofhealthinformationstudies.The liter-aturereviewofrelatedstudiessuggeststhata usability-focused nationalquestionnairewithnearly4000respondentscanbe con-sidered exceptionalascompared toothersimilarstudiesinthe field.However,ourscopingreviewhadsomelimitationsaswell. Thereviewwasfocusedontheacademicliteraturepublishedin PubMedbetween2010and2015.Weareawarethatsome stud-iesonthetopic“questionnairestudiesonEHRusability”havebeen reportedbefore,forinstancetheacademicstudiesbyChristensen et al.[77]and Edwardset al.[78]aswellassome comprehen-sive reviews on EHR systems in the USA market (e.g. surveys by American EHR [79–81]). These non-academic surveysabout usersatisfactionwithEHRsintheUSAhavebeenconductedfor someyears.Conclusionsbasedonthesestudies,however,seem tobesomewhatcontradictoryandmanifold:HIMMS13reported EHRsatisfactiondiminishing[82],whereasrecentlypublished sur-vey reportby AmericanEHRPartnersindicate agrowing overall satisfactionamongphysicianswiththeirEHRsystems[81]. Inter-estingly, the report also points out how usability ratings vary betweenspecifictasks(e.g.refillingaprescriptionwasratedaseasy whereas importationof apatient’smedication listdifficult) and betweenspecialities(primarycarephysiciansgivingmorepositive evaluationsthanotherspecialists)[81].Whatismore,theuseof othersearchterms(e.g.cross-sectionalorusersatisfaction)could haveresultedinmorerelatedarticles.However,forthekeyterms
weselected“usability”sinceourgeneralfindingisthattheterms satisfaction,usability,userexperience,usefulnessand meaning-fuluseareusedinhealthinformaticsliteraturewithsomewhat contradictorymeanings.
5.5. Furtherresearch
Thisarticleisthefirstinternationalpublicationwhichreports resultsfromournationalsurveydatagatheredin2014.Resultsfrom the2010surveywerereportedinseveralarticles[4,9,47–49]. Like-wise,infuturestudieswewillanalysethequestionnairedatafrom otherperspectives, forinstancehealthinformationexchange.In addition,ouraimistoresearchforinterveningvariablestofindfor exampleifmoreexperiencedITusersmaybecomemorecriticalof theEHRsystemstheyuseforwork.
Ouraimistocontinuethemonitoringofdevelopmentof health-careITsystemsinFinlandasseriesofcross-sectionalstudieson physicians’experiencesofEHRsystemuseandusability.Thisalso meansthatthesurveyquestionnaireneedstobeupdatedtoreflect thechangesinthefield(e.g.regionalandnationalregulations). UpdatestoeHealthstrategyandpolicygoals(e.g.patient empow-ermentviapatientportalfunctionalities)callforupdatesof the surveyinstrument.Thenextnationaldatagatheringwith physi-ciansisplannedtotake placein2017.Alotofexpectationsare associatedwiththeforthcomingmoregeneralHIEanduser experi-encesmightshowdifferentresultsafterKanta(thenationalarchive ofhealthinformationinFinland)installationsareinfullservice. Thestudyresultscanbeusedtoinformhealthcareproviders, deci-sionmakersandpoliticiansaboutthecurrentstateofEHRusability anddifferencesbetweenbrandsaswellasimprovementsofEHR usabilityatanationallevel.
Our current survey did not cover other health professional groupssuchasnurses.AstheEHRisamultidisciplinaryplatform thatisexpectedtosupportteamwork,otherprofessionalsmightbe includedinfuturequestionnaires(e.g.physiotherapistsandvarious specialityprofessionalsuseacommonEHR).Medicalsecretaries performimportanttasksininformationlogisticsandscheduling. However,thesurveyquestionnaireneedstobemodifiedforuse withtheotherprofessions,sincetheworktasksandresponsibilities ofotherprofessionsdifferfromthoseofphysicians.
Ournationalquestionnairestudyhasgainedinterestinseveral countries,includingCanadaandNordiccountries.Thereis ongo-ingresearchactivitybetweenFinlandand Canadatodevelop a generalizedusability-focusedquestionnaireforvariousgroupsof healthcareprofessionals,includingnurses,basedontheFinnish nationalquestionnaireforphysicians.Itwouldbeinterestingtobe abletocomparetheresultsfromnationalusabilitystudiesacross morecountriesandmonitorthedevelopmentofhealthcareIT sys-temsataninternationallevel.Fromaconceptualperspective,our studysuggeststhatmoreresearchisneededtounderstandthe rela-tionshipbetweenconceptsofusability,technology-inducederror andpatientsafety.
6. Conclusion
Thehealthcare fieldiscontinuouslychanging.Political, orga-nizational and technological changes as well as increasing digitalizationhaveeffects onhealthcare ITsystemdevelopment andimplementation.ResearchonexperiencedusabilitywithEHR systemsisessentialtofindouthowthesechangesappearin clin-icians’ daily work in clinical environments. Based onour best knowledge,thereportedcross-sectional surveywithphysicians inFinlandisthefirstnationaleHealthobservatoryquestionnaire focusedonusabilityandusedtomonitorlong-termdevelopment inthisarea.