Familial hypercholesterolemia. The determination of phenotype - 5 Data collection from medical records: Guidelines for the physician-scientist

UvA-DARE is a service provided by the library of the University of Amsterdam (https://dare.uva.nl)

Familial hypercholesterolemia. The determination of phenotype

Jansen, A.C.M.

Publication date

2003

Link to publication

Citation for published version (APA):

Jansen, A. C. M. (2003). Familial hypercholesterolemia. The determination of phenotype.

General rights

It is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s)

and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an open

content license (like Creative Commons).

Disclaimer/Complaints regulations

If you believe that digital publication of certain material infringes any of your rights or (privacy) interests, please

let the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the material

inaccessible and/or remove it from the website. Please Ask the Library: https://uba.uva.nl/en/contact, or a letter

to: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. You

will be contacted as soon as possible.

Dataa collection from medical records:

Guideliness for the physician-scientist

Angeliquee CM Jansen

, Emily S van Aalst-Cohen

, Barbara A Hutten

,

Harryy R Büller

, John JP Kastelein

and Martin H Prins

Departmentss of Vascular Medicine1 and Clinical Epidemiology and Biostatistics2, Academicc Medical Center, University of Amsterdam, Amsterdam. Department of Clinical

Epidemiologyy and Medical Technology Assessment3, Academic Hospital Maastricht, the Netherlands. .

Abstract t

Objective e

Too construct guidelines for data collection from medical records.

Studyy design and setting

Retrospectivee analysis of clinical data is often performed by physician-scientists. In such research,, the source of clinical data is the patient's medical record. However, medical records aree intended for patient care and the data are not systematically recorded for research purposes.. We developed guidelines for data collection from medical records. These guidelines weree constructed using recommendations from the literature and our own experience with aa retrospective cohort study that uses a DNA bank.

Results s

Wee developed guidelines which incorporate a number of strategies for accurate data collectionn and demonstrate their application.

Conclusion n

Byy using guidelines for data collection the quality of research data is enhanced. A well-designed casee record form and a handbook for standardized data collection are essential for training thee data collectors and for ensuring fastidious searching of the record. However, certain informationn is not always well-documented in patient records. Consequently, it is essential to performm a pilot study to assess the study design and to use additional questionnaires. Correct interpretationn of clinical outcomes documented in the medical records often necessitates an independentt adjudication committee to prevent bias in outcome definition.

Introduction n

Duringg recent years, bio-banks of patient materials such as serum, DNA and pathology specimenss have become a rich source for scientific research. Such patient materials are storedd in laboratory freezers in order to be subjected to novel diagnostic techniques when theyy become available. And, indeed, retrospective examination and analysis of bio-bank materialss and other clinical data is performed increasingly by physician-scientists and epidemiologists. .

!nn such a retrospective study, the primary source nf clinical data is almost always the medical recordss of the participating patients. However, medical records are primarily intended for patientt care and the data are not systematically recorded for research purposes. N evert he-less, retrospectivee studies using such data should be of high quality, without incomplete, inappropriatelyy recorded or missing data. In analogy, it is expected that data collection in randomisedd controlled trials (RCTs) is of the highest quality u as their unbiased evaluation of medicall treatment has a major impact on medicine. Observational studies, such as cohort studiess using patient's records, have a considerable impact on medical practice as well, in fact,, such studies are performed even more often than RCTs, because it is relatively easy to collectt the necessary data and the attached costs are comparatively low.3

Inn the process of designing one of our current research projects that uses a large DNA bank, thee GIRaFH study (Genetic Identification of Risk Factors in Familial Hypercholesterolemia), we performedd a systematic search of the published literature for the design, execution and reporting off retrospective studies using medical records for data collection. No comprehensive guidelines weree found for the execution or reporting of such studies. Therefore, we decided to develop aa set of guidelines which we discuss in this paper.

Thesee guidelines were developed using recommendations from the published literature andd our own experience with the GIRaFH study. Subsequently, we assessed the contribution off the constructed guidelines to the quality of the GIRaFH study and their possible implications forr future research.

Methods s

Material l

Literature e

Thee MEDLINE database for the period of January 1966 through May 2003 was searched usingg the key terms methodology, study design, peer review, reporting, quality management, observationall study, retrospective study, validity, bias, and confounding. We found and evaluatedd major publications (and their references) on the quality assessment of clinical

research,, including papers on randomised controlled trials, pharmacological studies, meta analyses,, and observational studies.'9

Geneticc Identification of Risk Factors in Familial Hypercholesterolemia: Thee GIRaFH study

Heterozygouss familial hypercholesterolemia (FH) is a common (1:400) hereditary disorder off lipoprotein metabolism. Due to genetic defects in the low-density-lipoprotein receptor gene,, patients suffer from severely elevated LDL-cholesterol levels and, as a consequence, fromm early atherosclerosis and premature cardiovascular disease (CVD). Although FH is aa monogenic disorder, variation is observed in the severity and onset of cardiovascular symptoms.. The study objective was to estimate the contribution of genetic variations too the development of CVD in a large cohort of FH patients.

AA retrospective, multicenter cohort study was performed in 2400 FH patients from lipid clinicss of 27 hospitals throughout the Netherlands. These patients were randomly selected fromm the DNA-bank database of the Department of Vascular Medicine at the Academic Medicall Center in Amsterdam, which has been appointed as the officia! molecular diagnosticc center for nationwide FH screening in the Netherlands.

Phenotypicall data were acquired by reviewing medical records by a well-trained team off data collectors. First, strict in- and exclusion criteria were applied to ensure that only FHH patients were included in the study. Data were collected on demographics, classical riskk factors, medication use, physical examinations, laboratory parameters, and extensive informationn on CVD. All patients gave informed consent and the Ethics Institutional Revieww Board of each participating hospital approved the protocol.

Floww of information: The data-collection process

Too arrive at the present guidelines, we examined the flow of information in the data-collectionn process and designed strategies for accurate data-collection based on the literature andd our own experience. Figure 1 shows the flow of information for data gathered from patientt to medical record (a), and from medical record to database (b). The figure also displayss several proposed tools for consistent data collection (pilot study, case record form, handbook,, questionnaire and independent adjudication committee) and where they may playy a role, as discussed below.

a.. I n f o r m a t i o n f r o m patient t o medical record

AA medical record contains information supplied by the patient to the physician. This informationn is often not standardized or complete and is prone to subjectivity. For example, thee patient may recall information from his or her earlier medical history incorrectly, or may reportt symptoms incompletely or inaccurately (e.g. gastro-oesophageal reflux is reported

figuree 1 Data information flows

Pilott study

T T

uu u

Database e

z z

Dataa collector

Casee record form

Handbook k

Questionnaire e

Independentt adjudication committee

a.. Information from patient to medical record bb Information from medical record to database

ass angina). Furthermore, the physician may take an incomplete history or record information incorrectly.. In addition, it must be taken into account that certain information, for example dataa on potential confounders, may be lacking in older records due to advances in medical knowledge.. For instance, homocysteine has only recently been recognised as a risk factor forr CVD and may not be listed in earlier records.

Whenn researchers refer solely to a medical record for research data, the patient and physician aree usually not consulted. Therefore, errors occurring at the patient and physician levels are difficultt to avoid. To evaluate possible errors, questionnaires may be sent to a random selectionn of patients and checks may be performed on the information in the medical recordd versus that in the questionnaire. If important differences are identified, the researchers shouldd send questionnaires to all participating patients. To reduce possible errors, the data collectorr should verify any recorded information against the questionnaires in addition to originall source documents such as hospital discharge reports and other physician's notes.

b.. Information from medical record to database

Medicall record

Inn general, as in a randomised controlled trial, the data collected in a retrospective cohort studyy should be standardized for each patient. Data collection can best be standardized by usingg a case record form (CRF) that is easy to complete and effective for collecting the requiredd data. In the literature, we found useful publications on how to design an effective CRF.'0"" Another advantage of using CRFs rather than computerised data collection is that aa CRF is easier to authenticate for later referral/1

Thee availability of information from the medical record is important for answering the researchh question. We could not find recommendations about what constitutes adequate availability.. Therefore, we arbitrarily decided upon good availability of data as more than 80%% for clinical and laboratory parameters, and more than 95% for clinical outcomes. If thee availability of data is not sufficient, the investigator should send a supplementary questionnairee to all participating patients during definitive data collection.

Incorrectt transfer of data from the medical record to the CRF is a problem in large studies and whenn multiple data collectors are involved. To prevent this, random checks of the CRFs must bee performed prior to data entry, comparing the original records to the information listed in thee CRF. This can be achieved by applying the ' 100-20' rule: 100% of the data are checked in 20%% of the CRFs, and 20% of the most essential data are checked in 100% of the CRFs.12 Furthermore,, when data entry from the CRF to the database is done by para-medical personnel, inconsistenciess may be detected upon data entry and be reported in queries to the principal investigator.. In addition, consistency checks run on the database may reveal outlier results. Alll errors found must be corrected using the original medical records.

Dataa collectors

Forr a large study, a team of specially trained data collectors may be effective in dealing with aa large number of data in a relatively short period. Splitting such a team into smaller units forr multi-site data collection has been shown to be superior to using on-site data collectors.13 Inn order to avoid bias, information about clinical outcomes should be obtained in the same wayy for all patients. Using CRFs and a handbook of standards minimises information bias9. AA handbook of standards for completing each CRF item can be helpful in attaining maximal uniformityy of data records and in avoiding misinterpretation of data. For example, the handbookk can provide lists of stringent criteria for in- and exclusion characteristics of the patientss in addition to other clinical data. General instructions should include how to deal withh incomplete or missing data: for instance, so-called dummy data reduce the number of incompletee data such as dates. Using such a handbook trains the data collectors to critically examinee the patient information, the physician's notes and the physician's thought processes.

Inter-observerr variability poses a threat to the quality of the data collected, but a well-designedd CRF and handbook can reduce this. In addition, inter-observer discrepancies should bee evaluated during data-collection training. For example, each data collector gathers data fromm 10 medical records onto CRFs and the principal investigator does the same. The data enteredd into the two sets of CRFs are subsequently entered into the SPSS Data Entry program andd compared by computer. If necessary, further training sessions are performed. In this manner,, inter-observer variability can be reduced substantially.

Performingg a pilot study

Inn our opinion, a pilot study performed prior to initiating the study is essential for reconsideringg elements of the study design: the research question, the study population, thee in- and exclusion criteria, the CRF, the accuracy of the collected data, and the availability off information from the various data sources. During the pilot study, the entire data-collection processs is scrutinized, as shown in Figure 1.

Independentt adjudication committee

Consultingg an independent adjudication committee is essential when there are no consensus criteriaa for the definition of the disease and clinical outcome of interest. In cases where the clinicall outcome in question does not quite fulfil the criteria, the committee reviews all of thee availabfe data and makes a final decision regarding outcome status.

Results s

Guidelines s

Wee developed a set of guidelines which incorporated strategies for accurate data collection (seee Table 1). In the table, a and b refer to the data information flow as shown in Figure 1. Too illustrate the application of the proposed guidelines, we discuss three essential aspects off the GIRaFH study (in- and exclusion criteria, information on smoking, and information on cardiovascularr events) and how these data were retrieved from the medical records.

Inclusionn and exclusion criteria

DNAA samples of patients with clinically suspected FH are regularly sent to our DNA laboratory fromm lipid clinics throughout the Netherlands for analysis of the LDL receptor gene. For the GIRaFHH study, 4000 potential patients were randomly selected from this database. In- and exclusionn criteria were carefully defined in the handbook as an aid to the data collectors andd to avoid selection bias. It was important that the patients be similar in all important aspectss except for their genetic profile. Patients without FH and/or with secondary hypercholesterolemiaa were excluded. After these criteria were applied, a total of 2400 patientss were included in the study.

Smoking g

Smokingg is an important risk factor for cardiovascular disease. In the pilot study, smoking statuss could only be retrieved from 56% of the records. We then adapted the final CRF to containn more specific information on lifetime smoking status and the availability increased too 68%. In order to achieve maximal smoking status information questionnaires were sent

Tablee 1: Guidelines for data collection from medical records

Dataa collection process Risks s Strategiess for inconsistency reduction n

a.. Information from patient to medicall record

Incorrectt information supplied by thee patient and/or incorrectly recordedd information.

b.. Information from medical recordd to database Medicall record

Diagnosticc reports from the wrong patient t

Availabilityy of data from medical file

Completenesss of medical file

Verifyy information against original sourcee documents and multiple physicians'' notes

Sendd additional questionnaires to aa random selection of participating patientss and perform checks of medicall record information versus s questionnairee information

Checkk whether diagnostic reports correspondd to the patient

Checkk availability in a pilot study. Sendd additional questionnaires to alll participating patients if necessary

Dataa collectors should cooperate closelyy with local physicians and otherr staff, who should provide otherr available records or sources off information

Incorrectt interpretation of data in Data collectors must have a para-medicall record medical educational background

Consultt an independent advisory committeee on definitions of diseasee and clinical outcome

Performm an inter-observer study as partt of data collection training

Supplyy handbook of written guideliness to data collectors definingg consensus criteria for medicall conditions and other rules forr data collection

Non-standardizedd data collection Create a case record form (CRF)

Usee handbook of data collection guidelines s

Unauthenticatedd data collection Retain CRF as authenticated data

Savee completed and checked CRFs ass original data for later referral

Dataa collection process Risks Strategies for inconsistency reduction n

Dataa collectors

Uncertainn endpoints

Illegiblee physician's handwriting

Multi-centerr study: on-site data collection n

Biasedd data collectors

Uninformedd data collectors

Incorrectt transfer of data from medicall record to CRF

Formm an independent adjudication committee e

Askk different data collectors to judgee the same handwriting

Employy one team of data collectorss for different study sites

Performm an inter-observer study betweenn team members to control inter-observerr variability

Blindd data collectors

Organizee regular meetings betweenn data collectors to discuss dataa collected and interpretation thereof. .

Performm a random check of completedd CRFs against original medicall records

Performm a consistency check of finall database to reveal outlier results s

too all participating patients, which further increased the percentage to 88%.

Thee handbook contained tables useful for standardising recorded numbers of cigarettes. Forr example, if the doctor recorded that the patient 'only smoked at parties', the data collectorr standardized this into 0.5 cigarette a day. Items like this were discussed by members off the study team and standardized definitions were decided by consensus.

Cardiovascularr events

Onee of the goals of the pilot study was to assess whether the selected study population wouldd be representative of the FH patient population at large (FH patients who are seen in outpatientt clinics) and would therefore be appropriate for answering the research question. Onee of the important indicators was therefore the cardiovascular event rate. The pilot study populationn demonstrated an event rate during the observation period that was consistent withh the literature on cardiovascular disease in FH patients {32% of patients had clinically manifestt cardiovascular disease).M The final study data exhibited similar event rates.

Cardiovascularr events were scored using the handbook which listed recent consensus criteria forr these events. Independent sources such as the physician's notes on the patients'

symptoms,, the physician's notes on possible clinical diagnoses and the diagnostic reports weree reviewed by the data collectors. This information was combined to evaluate whether orr not the event fulfilled the criteria.

Inn case the events did not quite fulfil the criteria or if any suspect histories, symptoms or diagnosticc evaluations were discovered in the record, the case was presented to an independent adjudicationn committee consisting of a cardiologist, a neurologist and a vascular surgeon, usingg anonymous copies of the necessary documents from the medical record. In retrospect, wee proved to have interpreted nearly all events correctly prior to consulting the committee.

Discussion n

Byy the nature of its design, the GIRaFH study is prone to bias.9 However, by using guidelines forr consistent data collection we believe to have enhanced the quality of the research data.

Definingg clear in- and exclusion criteria is essential in any research project, neglecting to do soo may result in a heterogeneous patient population in which potential confounders abound. Inn a study using medical records as the primary data source, these criteria must be especially clearr to all data collectors and must be sought for in all source documents. A clear CRF and aa handbook for standardized data collection are essential for training the data collectors to applyy the criteria and to stimulate fastidious searching of the record. However, certain informationn is not always well-documented in patient records. In our experience, it is helpful too perform a pilot study. Particularly when a project is solely dependent on data from medicall records, a pilot study helps to assess the availability of the necessary information, thee feasibility of the project and the quality of the CRF.

Despitee the fact that smoking is an important predictor of cardiovascular disease, it is often poorlyy documented in medical records. Underreporting of smoking could lead to incorrect estimationn of the relative risk of a certain candidate gene associated with increased cardiovascularr risk. It was therefore important to assess the availability of the smoking statuss from the medical records. During the pilot study, the patients' smoking status was onlyy available in half of the records. Adaptation of the CRF and handbook according to recommendationss made after the pilot study as well as the use of questionnaires increased thee information on smoking status. However, we must take into account that some smokers failedd to return the questionnaire (non-respondent bias) which could still lead to an incorrect estimationn of the relative risk. Fortunately, the response rate was high (70%), therefore we considerr this type of bias to be minimal.

Inn this study, the genetic make up of FH patients with and without vascular events was compared.. It was therefore important to define which patients had experienced a vascular event.. We hoped that by consulting an independent adjudication committee, misinterpreting

eventss recorded in the medical records could be avoided. However, the number of misinterpretedd events proved to be insignificant. We believe that the meticulous definition andd application of cardiovascular event criteria led to this result. Consulting an independent committeee did not add to the quality of the GIRaFH study but an independent adjudication committeee may be important when decisive definitions of disease and clinical outcome are nott available. Interpretation and definition of myocardial ischemia are easier than, for example,, formulating a definition for pancreatitis.

Itt is widely recommended that research projects undergo quality review. However, as yet

t h o r oo a r o n n m m n r o h o n ^ i w p n u i H ^ l i n p t ; f n r n o r f r \ r m i n r t r o t r n < ; n P r t i \ / P r n h n r t c;tiirlip<L i K i n n _{v , . ^ . , ^^}

u, ~ , . „ ^ „ , . .f-, . _ . . . _ , . w . , ^ a " ' " i - 2 — ~ ~r- . _ . _ . . - ,

medicall records, although this type of design is increasingly used due to the availability of largee collections of patient materials. The recommendations in this paper are based on our ownn experience with the difficulties we encountered. The results do not give a definitive solutionn for minimising bias during data collection from medical records, but are suggestions forr quality management of data collection. We believe that attention paid to the methodology off such studies will stimulate adequate reporting. We expect that this will ultimately lead to thee creation of effective checklists that are easy to use by physician-scientists.

Acknowledgments s

Thee authors are grateful to the Netherlands Heart Foundation for their support (grant NHS 98/165).. John Kastelein is an established investigator of the Netherlands Heart Foundation (grantt D039/66510). The authors wish to thank Mrs. G.E.E. van Noppen for her valuable assistancee with manuscript preparation.

References s

1.. Moher D, Schulz KF, Altman DG. The CONSORTT statement: revised

recommendationss for improving the quality off reports of parallel-group randomised trials.. Lancet 2001; 357:1191-94. 2.. Spriet A, Dupin-Spriet Th.. Monitoring a

clinicall trial. In: Good practice of clinical drugg trails. Basel: Karger;1997:77-91. 3.. Grimes DA, Schulz KF. An overview of

clinicall research' the lay of the land Epidemiologyy series. Lancet 2002; 359:57-61. .

4.. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamsonn D, Rennie D, Moher D, Becker BJ,, Sipe T, Thacker SB. Meta-analysis of observationall studies in epidemiology (MOOSE).. JAMA 2000; 283:2008-12. 5.. DuRant RH. Checklist for the Evaluation of

Researchh Articles. J Adolesc Health 1994; 15:4-8. .

6.. Bogardus ST, Concato J, Feinstein AR. Clinicall Epidemiological Quality in Molecular Geneticc Research. The need for

methodologicall standards. JAMA 1999; 281:1919-26. .

7.. Moher D, Cook DJ, Eastwood S, Olkin I, Renniee D, Stroup DF. Improving the quality off reports of meta-analyses of randomised controlledd trials: the QUOROM statement. Lancett 1999; 354:1896-1900.

8.. Sackett DL. Bias in analytic research. JChronn Dis 1979; 32:51-63.

9.. Grimes DA, Schulz KF. Bias and causal associationss in observational research. Epidemiologyy series. Lancet 2002; 359:248-52. .

10.. Lawrence G. Case Record Form Design. In: Rondell RK, Varley SA, Webb C, Eds. Clinical Dataa Management. New York: John Wiley & Sons;1993:133-52. .

11.. Spriet A, Dupin-Spriet Th.. Designing case reportt forms. In: Good practice of clinical drugg trails. Basel: Karger;1997:64-76. 12.. Clarke PA. Data Validation. In: Rondel RK,

Varleyy SA, Webb C Eds. Clinical Data Management.. New York: John Wiley and Sons;1993:189-212. .

13.. Jasperse, DM, Ahmed SW. The Mid-Atlantic Oncologyy Program's Comparison of Two Dataa Collection Methods. Control Clin Trials 1989;; 10:282-89.

14.. Goldstein JL, Hobbs HH, Brown MS. Familial Hypercholesterolemia.. In: Scriver CR, Beaudett AL, Sly WS, Valle D Eds. The metabolicc basis of inherited disease. New York:: McGraw-Hill;2001:2863-2913.