There are children not receiving a single dose of any vaccine: from ‘data to
policy’ in immunisation and health systems. Data quality and socio-economic
determinants of unvaccination in low- and middle-income countries
Bosch-Capblanch, X.
Publication date
2012
Document Version
Final published version
Link to publication
Citation for published version (APA):
Bosch-Capblanch, X. (2012). There are children not receiving a single dose of any vaccine:
from ‘data to policy’ in immunisation and health systems. Data quality and socio-economic
determinants of unvaccination in low- and middle-income countries. Rozenberg Publishers.
General rights
It is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an open content license (like Creative Commons).
Disclaimer/Complaints regulations
If you believe that digital publication of certain material infringes any of your rights or (privacy) interests, please let the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the material inaccessible and/or remove it from the website. Please Ask the Library: https://uba.uva.nl/en/contact, or a letter to: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. You will be contacted as soon as possible.
Xa
vier
B
osc
h-Capblanc
h
IT
her
e ar
e C
hild
ren no
t r
ec
ei
ving a Single Dose of
An
y V
ac
cine
There are Children not receiving a Single
Dose of Any Vaccine: from ‘Data to Policy’
in Immunisation and Health System
Data Quality and Socio-Economic Determinants of
Unvaccination in Low- and Midde-Income Countries
THERE ARE CHILDREN NOT RECEIVING A SINGLE DOSE OF ANY VACCINE:
FROM ‘EVIDENCE TO POLICY’ IN IMMUNISATION AND HEALTH SYSTEMS
International Child Health Series 20 ISBN 978 90 361 0332 9 © Xavier Bosch‐Capblanch, 2012 All rights reserved. Save exceptions stated by the law, no part of this publication may be reproduced, stored in a retrieval system of any nature, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, included a complete or partial transcription, without the prior written permission of the authors, application for which should be addressed to author. Printed by Rozenberg Publishers Lindengracht 302 d+e 1015 NK Amsterdam The Netherlands Tel. (+) 31 (0) 20 625 54 29 info@rozenbergps.com
THERE ARE CHILDREN NOT RECEIVING A SINGLE DOSE OF ANY VACCINE: FROM ‘DATA TO POLICY’ IN IMMUNISATION AND HEALTH SYSTEMS (DATA QUALITY AND SOCIO‐ECONOMIC DETERMINANTS OF UNVACCINATION IN LOW‐ AND MIDDLE‐INCOME COUNTRIES)
ACADEMISCH PROEFSCHRIFT ter verkrijging van de graad van doctor aan de Universiteit van Amsterdam op gezag van de Rector Magnificus prof. dr. D.C. van den Boom ten overstaan van een door het college voor promoties ingestelde commissie, in het openbaar te verdedigen in Agnietenkapel op vrijdag 16 november 2012, te 10.00 uur
door Xavier Bosch‐Capblanch geboren te Barcelona, Spain
Promotiecommissie
Promotor:
Prof. dr. B.J.M. Brabin
Overige leden:
Prof. dr. R.J.P.M.Scholten Prof. dr. B.Eriksson dr. R. Pebody Prof. dr. H.S.A.Heymans dr. R. M. Boele van Hensbroek Faculteit der GeneeskundeEn homenatge a la mamà Amb el Toni a la memòria Per l’Alícia Com la Berhane
Table of contents
Abbreviations ... vii Chapter 1. Introduction and objectives ... 1 Chapter 2. Harmonisation of variables names prior to conducting statistical analyses with multiple datasets: an automated approach. Xavier Bosch‐Capblanch. BMC Medical Informatics and Decision Making 2011, 11:33. ... 7 Chapter 3. Accuracy and quality of immunization information systems in forty‐one low income countries. Xavier Bosch‐Capblanch, Olivier Ronveaux, Vicki Doyle, Valerie Remedios and Abdallah Bchir. Tropical Medicine and International Health 2009; 14(1): 2–10. ... 19 Chapter 4. There is no gold standard to consolidate discrepancies in vaccination coverage figures between data sources: are we globalising or trivialising access to data? Xavier Bosch‐Capblanch. Status: being submitted. ... 35 Chapter 5. Unvaccinated children in years of increasing coverage: how many and who are they? Evidence from 96 low‐ and middle‐income countries. Xavier Bosch‐ Capblanch, Kaushik Banerjee and Anthony Burton. Tropical Medicine and International Health 2012; 17(6): 697–710. ... 53 Chapter 6. Is it sex or gender that determines vaccination status in children? (I). Evidence from Demographic and Health surveys. Xavier Bosch‐Capblanch, X Bosch‐Capblanch, L Beck, C Schindler, P Namgyal, Sonja Merten, J Hombach, A Martin‐Hilber. Submitted. ... 75 Chapter 7. Do existing research summaries on health systems match immunisation managers’ needs in middle‐ and low‐income countries? Analysis of GAVI health systems strengthening support. Xavier Bosch‐Capblanch, Marion Kelly and Paul Garner. BMC Public Health 2011, 11:449. ... 95 Chapter 8. Discussion ... 105 Summary ... 111 Samenvatting ... 117 Acknowledgments and financial support ... 124 Curriculum Vitae ... 125
Abbreviations
BCG Bacille Calmette‐Guerin Vaccine DQA Data Quality Audit DTP Diphtheria‐Tetanus‐Pertussis DHS Demographic and Health Survey EPI Expanded Programme of Immunisation GAVI The GAVI Alliance LMIC Low‐ and middle‐Income Countries MCV Measles containing vaccine MDG Millennium Development Goals MICS Multi‐Indicator Cluster Survey OPV Oral Polio Vaccine SAGE Strategic Advisory Group of Experts SCIH Swiss Centre for International Health Swiss TPH Swiss Tropical and Public Health Institute UNICEF United Nations Children Fund WHO World Health OrganisationChapter 1. Introduction and objectives
Health systems
Health related Millennium Development Goals (MDG 4, 5 and 6) will not be achieved in many low‐ and middle‐income countries (LMIC) by 2015 [1]. Current trends show stagnation or draw backs of MDG monitoring indicators despite the existence of effective interventions to reduce childhood mortality, or to improve maternal health, and to reduce the burden of the major diseases affecting the populations living in those countries. Importantly, these trends have not been totally reversed by the increase in financing aid for health in the last two years [2]. Weak health systems have been widely recognised as barriers to the implementation and scaling up of effective interventions [3,4,5]. In order to boost the uptake of effective clinical and public health interventions, several health systems arrangements and strategies have been tried over recent decades. The early emphasis on Primary Health Care [6] in the 1970s and 1980s was followed by health sector reforms, as well as a shift of the focus from more holistic approaches, with prioritisation of the cost‐effectiveness of interventions [7]. However, the magnitude of major diseases affecting populations of LMIC, such as malaria, tuberculosis and HIV/AIDS, led to the establishment Global Health Initiatives (GHI) – large partnerships of public, private, multilateral agencies and civil society stakeholders. This was based on the assumption that unifying efforts for a single objective would be a more effective approach. In the immunisation area, The GAVI Alliance (GAVI) was created in the year 2000 with the mission of “saving children’s lives and protecting people’s health by increasing access to immunisation in poor countries” [8]. The growing evidence on the inability of health systems to cope with increasing pressures to deliver services and programmes for specific conditions, and the critics highlighting the vertical nature of GHI operations, among other factors, have led to the concept of health systems strengthening, although there is no agreed definition of the term [9]. This concept was subsequently adopted by GHIs as well. In parallel, health systems and health systems research have increased their space in the research and services domains. Health systems research is becoming a priority [10], and the challenges of carrying out this type of research have been emerging over recent years. For example, the first Global Symposium on Health Systems Research was held in Lausanne (Switzerland) in 2010, and the second in Beijing (China, 2012) [11]. More recently, it has been emphasised that health systems should not only deliver services and programmes, but should also do so in a way that all populations can benefit from them, as well as protecting users and the wider population from the financial risks arising from ill health, in the line of universal coverage [12].Childhood immunisation
Systematic childhood vaccination is one of the most cost‐effective health interventions [7], it is carried out in every country, and has saved millions of lives over decades. The globalimmunisation programme was created in 1974 under the form of the Expanded Programme of Immunisation (EPI) and was a part of the holistic and selective PHC concepts. The EPI introduced routine childhood vaccination against six diseases in health services worldwide (tuberculosis, diphtheria, tetanus, whopping cough, poliomyelitis and measles), as well as tetanus vaccination in pregnancy. An immunisation indicator (measles) is used to monitor the achievement of MDG 4, and it has a direct impact on the reduction of childhood mortality. Routine immunisation is probably the oldest, most implanted and most standard health intervention in the world and the best documented over many years, with data available from numerous countries since 1980. For these reasons it is a very suitable theme to examine determinants of utilisation, as well as access of childhood public health interventions. Despite the steady increase in immunisation coverage over the years [13,14], the stagnation of coverage in the 90s triggered the creation of GAVI in the year 2000 to support immunisation services in countries with a per capita GNI below 1,500 USD [15]. GAVI’s partners include the Bill and Melinda Gates Foundation, the World Health Organisation (WHO), the United Nations Children Fund (UNICEF), the World Bank, development and industrialised countries governments, and industry, civil society and research and technical health institutes [16]. GAVI offers different types of support and plays a critical role in the promotion of under‐used and new vaccines. However, GAVI was soon criticised of distorting, or weakening, health systems with its vertical approach to immunisation, and consequently in the year 2004 the GAVI Board approved a support line for countries on health systems strengthening [17]. The ‘Health systems Funding platform’ was created in in order to synergise GHI efforts to improve health systems (GAVI, WHO and the World Bank) [18]. Immunisation remains one of the cornerstones of child health with enormous achievements. Polio, for example, is close to eradication; and the availability of new vaccines, such as pneumococcal and rotavirus vaccines, have raised enormous hopes for further reducing childhood mortality. Immunisation initiatives have been galvanised by several stakeholders, including international agencies and governments, and initiatives such as the most recent ‘Decade of Vaccines’ initiative [19], which maintain immunisation at the top of the international health agenda and at the core of countries child health strategies. Nevertheless, old and new challenges in formulating policies and strategies to incorporate new vaccines, and financial and logistic constraints, produce major stresses in health systems, which are already weak in many LMIC. Furthermore, coverage rates in countries and regions of the world show that there are major inequities, between and within countries, in accessing immunisation services, with particularly vulnerable groups remaining unvaccinated. In the case of polio vaccine, eradication has been achieved almost worldwide; yet, four countries in the world are still reporting wild poliovirus transmission in 2012 (Afghanistan, Chad, Nigeria and Pakistan) [20]. Vaccination against major killers, such as pneumonia and rotavirus, is still in the phase of scaling up in most LMIC, despite having been introduced at the same time as in many high‐income countries. Strategies to overcome inequitable access to vaccination have been designed and implemented (e.g. Reaching Each District, RED [21]), but hard to reach populations still remain inaccessible due to geographic, financial and cultural barriers.
Equity in accessing vaccination services
The causes which can explain why there are still children unreached by vaccination are only partially understood. Several reasons can illustrate this fact, for example: global or even regional increases in coverage mask inequities between and within countries; administrative reporting systems from services cannot capture those children who precisely do not attend vaccination sessions; hard to reach populations are also hard to assess [22]. Immunisation programme performance is routinely monitored using mainly the third dose of diphtheria‐tetanus‐pertussis vaccine (DTP3), but it is much more difficult to document the number of children who have received not even a single dose of any vaccine, and these are those most at risk. This ‘last mile’ quantifying the numbers of children not receiving vaccinations is more difficult to achieve. Even when there is evidence on inequities, it is a long way to translate this evidence into actionable policies. This relates to challenges in generating and translating evidence into guidance in order to inform policies, and the difficulties of formulating health policies in complex systems, as is often the case in fragile states, with budgetary constraints and competing interests [23]. The Strategic Advisory Group of Experts (SAGE) is a group of worldwide immunisation experts [24] having an advisory role to WHO on global immunisation policies. The group meets in Geneva (Switzerland) twice a year. Cognisant of the existing inequities in immunisation, SAGE called for the production of more systematic and global evidence to describe the problem of unvaccination and to assess the socio‐economic and gender‐related determinants of unvaccinated children. This is needed in order to inform specific decisions, as well as global and national policies. These demands were channelled through the Immunisation, Vaccines and Biologicals (IVB) and the Initiative for Vaccines Research (IVR) of the WHO, respectively. Studies were carried out by a team led by the candidate over the last few years, at the Swiss Tropical and Public Health Institute (Basel, Switzerland).Aims and objectives
The overall aim of this dissertation is to bridge the evidence to policy gap in order to inform national and global immunisation policies on the status and determinants of unvaccinated children; i.e. those children not having received a single dose of routine vaccinations. The objectives are: 1. To develop an algorithm and software to harmonise data for analyses of different designs of national representative household surveys. 2. To compare the quality of individual subjects vaccination data from household surveys with data from vaccination administrative monitoring. 3. To describe socio‐demographic and gender‐related determinants of unvaccination in children. 4. To assess the evidence base of immunisation related to health system strengthening in funding proposals. These objectives were fulfilled by a series of analyses reported in the articles included in this dissertation. The first objective addresses the methodological challenge of harmonising largenumbers of datasets with different formats used in the analyses. The second objective refers to the types and quality of immunisation data from different sources. For the third objective, the analyses of determinants of unvaccination are presented. The fourth objective addresses the use of data and other evidence to inform policies. In many LMIC, national‐representative household surveys are conducted to produce estimates of demographic and health indicators. More than 200 surveys in around 100 countries have been included in these analyses. Databases of these surveys with individual subjects data are available and downloadable from different sites. The first challenge was to harmonise the structure, format and contents of the databases and variables of this large number of surveys. This was done by developing and algorithm to harmonise the names and values of the variables used (described in Chapter 2). Surveys are not the only source to assess immunisation indicators. Administrative service data generated at immunisation delivery sites is recorded, aggregated and sent to the central levels of the health systems in most countries. However, discrepancies between administrative and surveys data have been reported. In Chapter 3 we describe the quality of administrative immunisation data based on Data Quality Audits (DQA) conducted in 41 countries. The candidate carried out himself the field work in four DQAs (Angola, Democratic People’s Republic of Korea, Nigeria and Zambia). In Chapter 4 an analysis of these discrepancies is presented, challenging the assumption that survey data are the gold standard. Chapters 4 and 5 describe the determinants of unvaccination based on the analyses of national representative household surveys. These two studies were commissioned by the SAGE. The analyses were based on logistic regression techniques, as described in detail in the chapters, applied to a large number of datasets from LMIC, which were previously harmonised using the algorithms described in Chapter 2. These analyses described socio‐economic (Chapter 5) and gender‐related (Chapter 6) determinants of unvaccination. Finally, we conducted a desk review of all 44 countries proposals for health systems strengthening related to immunisation services submitted to GAVI. We described what types of requests for funding were included in these proposals, and the use of existing evidence to support those requests.
References
1 United Nations Statistics Division. Millennium Development Goals Indicators. http://mdgs.un.org/unsd/mdg/Default.aspx. 2 Hsu J, Pitt C, Greco G, Berman P, Mills A. Countdown to 2015: changes in official development assistance to maternal, newborn, and child health in 2009‐10, and assessment of progress since 2003. Lancet. 2012 Sep 29;380(9848):1157‐68. doi: 10.1016/S0140‐6736(12)61415‐9. Epub 2012 Sep 21. 3 Biesma RG, Brugha R, Harmer A, Walsh A, Spicer N, et al. The effects of global health initiatives on country health systems: a review of the evidence from HIV/AIDS control. Health Policy Plan 2009: 24: 239–252. 4 Travis P, Bennett S, Haines A, Pang T, Bhutta Z, et al. Overcoming health‐systems constraints to achieve the Millennium Development Goals. Lancet 2005; 364: 900–906. 5 Bhutta ZA, Chopra M, Axelson H, Berman P, Boerma T, et al. Countdown to 2015 decade report (2000– 10): taking stock of maternal, newborn, and child survival. Lancet 2010; 375: 2032–2044. 6 WHO‐UNICEF. Primary Health Care. Report of the International Conference on Primary Health Care. Alma‐ Ata (USSR). WHO, Geneva 1978. 7 The World Bank 1993. World development report 1993 : investing in health.8 The GAVI Alliance. GAVI’s mission. http://www.gavialliance.org/about/mission/ [Accessed October 2012]. 9 Sundewall J, Swanson RC, Betigeri A, Sanders D, Collins TE, et al. Health‐systems strengthening: current and future activities (comment). Lancet 2011; 377: 1222–1223.
10 The Mexico Statement on Health Research. Knowledge for better health: strengthening health systems. From the ministerial summit on health research; Mexico City; 16–20 November 2004.
11 Second Global Symposium on Health Systems Research. http://www.hsr‐symposium.org/ [Accessed October 2012]. 12 WHO. Health Systems Financing. The path to universal coverage. WHO, Geneva 2010. 13 WHO. The Reaching Every District Strategy. Fact sheet WHO⁄ xx 2006. WHO, Geneva. 14 WHO. WHO Vaccine‐preventable Diseases: Monitoring System. 2006 Global Summary. WHO, Geneva. WHO ‐ Immunisation, Vaccines and Biologicals. 15 GAVI. GAVI Alliance country eligibility policy. Version 1.0. 16 The GAVI Alliance. GAVI’s partnership model. http://www.gavialliance.org/about/gavis‐partnership‐ model/ [Accessed October 2012]. 17 Muraskin W. The Global Alliance for Vaccines and Immunization: is it a new model for effective public‐ private cooperation in international public health? Am J Public Health 2004; 94(11):1922‐1925. 18 The World Bank. Health systems funding platform. http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTHEALTHNUTRITIONANDPOPULATION/EXTHSD/0, ,contentMDK:22299073~pagePK:148956~piPK:216618~theSitePK:376793,00.html [Accessed October 2012].
19 Decade of Vaccines Collaboration. http://www.dovcollaboration.org/ [Accessed October 2012]. 20 Global Polio Eradication Initiative. http://www.polioeradication.org/ [Accessed October 2012]. 21 WHO. Implementing the Reaching Every District approach. A guide for district health management teams. WHO regional office for Africa 2008. 22 Setel PW, Macfarlane SB, Szreter S, Mikkelsen L, Jha P, Stout S et al. A scandal of invisibility: making everyone count by counting everyone. Lancet 2007; 370:1569–77. doi:10.1016/S0140‐6736(07)61307‐5 PMID:17992727. 23 Bosch‐Capblanch X, Lavis JN, Lewin S, Atun R, Røttingen J‐A, et al. Guidance for Evidence‐Informed Policies about Health Systems: Rationale for and Challenges of Guidance Development. PLoS Med 2012; 9(3): e1001185. doi:10.1371/journal.pmed.1001185. 24 WHO. Immunization, Vaccines and Biologicals. Strategic Advisory Group of Experts (SAGE) on Immunization. http://www.who.int/immunization/sage/en/ [Accessed October 2012].
Chapter 2. Harmonisation of
variables names prior to conducting
statistical analyses with multiple
datasets: an automated approach.
Xavier Bosch‐Capblanch. BMC Medical Informatics and Decision Making 2011, 11:33.
Chapter 3. Accuracy and quality of
immunization information systems
in forty‐one low income countries.
Xavier Bosch‐Capblanch, Olivier Ronveaux, Vicki Doyle, Valerie Remedios and Abdallah Bchir. Tropical Medicine and International Health 2009; 14(1): 2–10.Accuracy and quality of immunization information systems in forty‐one low income
countries
Xavier Bosch‐Capblanch1, Olivier Ronveaux2, Vicki Doyle1, Valerie Remedios3 and Abdallah Bchir4 1 Swiss Centre for International Health ⁄ Swiss Tropical Ins tute, Basel, Switzerland 2 Vaccine Assessment and Monitoring, Vaccines and Biologicals, WHO, Geneva, Switzerland 3 Euro Health Group, Copenhagen, Denmark 4 GAVI Alliance, Geneva, Switzerland
Summary
Objectives. To measure the accuracy and quality of immunization information systems in a range of low‐income countries eligible to receive GAVI support. Methods. The Data Quality Audit (DQA) uses a WHO validated, standard methodology to compare data collected from health unit (HU) records of immunizations administered with reports of immunizations at central level and to collect quality indicators of the reporting system. The verification factor (VF), as a measure of accuracy, expresses the proportion of immunizations reported at national level that can be tracked down to the HU. A VF of 80% or above entitles countries to receive additional GAVI financial support. Quality indicators are assigned points which were summed to obtain quality scores (QS) at national, district and HU levels. DQAs included here were conducted between 2002 and 2005 in 41 countries, encompassing 1082 primary healthcare units in 188 randomly selected districts. Results. Almost half of countries obtained a VF below 80% and only nine showed consistently high VF and QS scores. The most frequent weaknesses in the information systems were inconsistency of denominators used to estimate coverage, poor availability of guidelines (e.g. for late reporting), incorrect estimations of vaccine wastage and lack of feedback on immunization performance. In all six countries that failed a first DQA and undertook a second DQA, the VF and all QSs improved, not all of them statistically significantly. Conclusions. The DQA is a diagnostic tool to reveal a number of crucial problems that affect the quality of immunization data in all tiers of the health system. It identifies good performance at HU and district levels which can be used as examples of best practices. The DQA methodology brings data quality issues to the top of the agenda to improve the monitoring of immunization coverage. Keywords: immunization information, GAVI, Data Quality Audit, developing countriesIntroduction
Routine immunization is one of the most cost‐effective public health interventions (The World Bank 1993) to reduce child mortality (Jones et al. 2003). Global immunization coverage of systematic vaccines has been steadily increasing since the eighties (WHO 2006a,b). However, global figures mask great inequalities between geographical regions and population sectors (Pearson 2003; WHO 2006a,b). It has been estimated that almost 18 million infants have not received the first dose of Diphtheria‐Tetanus‐Pertussis vaccine (DTP), half of them living in Southeast Asia and one third of them in Africa (Anonymous 2006).The GAVI Alliance (GAVI), launched in the year 2000, is one of the global health partnerships that have emerged in recent years aiming at improving access to known effective health care interventions (Walt & Buse 2000). GAVI’s mission is to save children’s lives and to protect people’s health through the widespread use of vaccines (GAVI; http://www.vaccinealliance.org/General_ Information/About_alliance/index.php). It focuses on the 72 countries (in 2006) with a Gross National Income (GNI) per capita below 1000 USD (GAVI; http://www. vaccinealliance.org/Support_to_Country/Who_can_ Apply/index.php), where most of the unimmunized children live (WHO 2006a,b). GAVI’s support to countries includes immunization services (ISS), injection safety, new and underused vaccines, health systems strengthening and civil society organization support (GAVI; http://www.gavialliance.org/ support/what/index.php). ISS is provided in two phases: an investment phase (years one and two) and the reward phase (from year three onwards). During the latter, countries receive 20 USD per additional infant who has received DTP3 (third dose of DTP) as compared to baseline figures (GAVI; http://www.gavialliance.org/support/what/ iss/index.php). However, this reward is contingent to providing evidence that data reported by countries are reliable, as assessed with Data Quality Audits (DQA). The aim of any initiative to improve immunization is to increase coverage up to a level where all children are protected against the targeted diseases. However, it has been increasingly recognized that good quality information for decision‐making is essential to increase coverage (Bchir et al. 2006; Papania & Rodewald 2006). Surveillance, monitoring and evaluation are integral components of successful immunization systems (WHO ⁄ UNICEF 2005). The DQA is a survey methodology developed by WHO which estimates the robustness of immunization reporting systems. The main outcome of DQAs is the estimation of the Verification Factor (VF), which expresses the accuracy of the reporting system by estimating the proportion of DTP3 immunizations that can be traced through the reporting system, from the vaccine delivery points up to the national coverage estimates. GAVI partners agreed that countries with a VF of 80% or more would ‘pass’ and receive the reward while countries that ‘fail’ are required to produce their own plans to improve the reporting system and are encouraged to conduct a second DQA 2 years later (The LATH Consortium 2001). A previous paper (Ronveaux et al. 2005) reviewed the methodological issues of DQA and focused on the aggregated outcomes of the DQAs conducted up to 2003. In this report, we present individual countries’ performance of the DQA carried out in 41 countries up to 2005, and explore patterns of performance among countries. We also show the changes in the reporting system in those countries that failed the first DQA and undertook a second one. As DQA do not aim at estimating immunization coverage, we will not do any comparison with other methods to estimate immunization coverage.
Methods
The DQA is a standard methodology developed by the WHO; it is carried out by independent companies after an open tender process. Two external consultants travelled to each country and engaged with two staff members of the national immunization programmes to conduct the DQAs over a period of 2 weeks. In each country, a multistage sampling procedure was followed: first, four districts were randomly selected with probability proportional to the reported doses of DTP3 administered in the previous year; secondly, in each of the four districts, six Health Units (HU), where immunizations are administered, were randomly selected (total of 24 HUs per country). This weighted representative sampling was designed to fit with what could be reasonably achieved within the resources and timeframe of the DQA. Districts and HUs with unsolvable access problems which make them non‐eligible were excluded from the sampling process. Reasons for exclusion were mainly security situations or major geographical barriers that could not be overcome within the timeframe of a field visit. DQAs with a proportion of unreachable districts greater than 20% have been excluded from some of the analyses and indicated in the text. The DQAs have two outcome measures: the VF and the Quality Scores (QS). The period audited was the full calendar year previous to the date when the DQA took place. In each district, the VF is calculated by dividing the number of DTP3 vaccinations administered during the audited year as recounted in the HUs records filled at the very moment when children are vaccinated by the annual DTP3 vaccinations reported in the HUs reports found at the health district offices (the usual next tier in the reporting system). This quotient is adjusted for the weight of the six selected HUs in relation to the whole number of HUs in the district. This is finally extrapolated to the national level as the weighed average of districts VFs. The methods and mathematical expressions have been described in detail elsewhere (WHO 2003; Ronveaux et al. 2005). A VF less than 100% indicates that the reports at district level showed more DTP3 administrations than those that could be recounted at HU level (‘over‐reporting’); a VF over 100% suggests that not all DTP3 doses recounted could be traced in the reports at district level (‘under‐reporting’). The QSs were based on a series of questions and observations undertaken at each level of the immunization programme: national, district and HU. They covered topics such as recording and reporting of immunization data, keeping of vaccine ledgers and information system design. Each question correctly answered was assigned one point. An average QS ranging from 0 to 5 was obtained for the national level, for each one of the four districts and for each one of the 24 HUs (some questions for each level of the system differed). Finally, auditors provided feed‐back to immunization staff at all levels and suggested recommendations addressing the most relevant issues identified.Statistical analyses Summary QS for each level are presented as medians and inter‐quartile ranges estimated for each country, district and HU. Correlation between continuous variables was estimated using Spearman’s rank test since we could not assume that their errors followed a normal distribution. Differences between medians were tested using the Mann– Whitney test in SPSS 13.0 (SPSS Inc. 1989–2004).
Results
Forty‐seven DQAs were conducted between 2002 and 2005 in 41 countries: 30 African, 10 Asian and one Caribbean. Twenty‐one countries failed the DQA (VF less than 80%), and six of those conducted a second DQA (total 47 DQAs). The proportion of non‐eligible districts for sampling was higher than 20% in nine of the 41 DQAs: 55% in Yemen, 45% in Nepal, 43% in Myanmar, 41% in Congo, 34% in Afghanistan, 32% in Lesotho, 32% in Sudan’s second DQA, 25% in DR Congo and 22% in Mali. A total of 1082 HUs were surveyed in 188 districts in the 47 DQAs. Table 1 summarizes the country profiles and DQA framework. The VF (data accuracy) was below the threshold value (80%) in 46% of the DQAs (median of the VF 83%, interquartile range (IQR) 23%) (Figure 1). Excluding those DQAs with high proportion of unreachable districts: 50% had a VF below 80% and the median of the VF was 80% (IQR 33%). In Nigeria it was not possible to estimate the VF due to lack of data. Two DQAs showed VFs above 100%, indicating under‐reporting (the deviation from 100% was marginal: 100.2% and 106.4%). VF 95% confidence intervals (CI) were wide, especially in countries with low VFs, reflecting the great variability of the DTP3 recounted‐reported quotient among districts. DQAs with VFs above 95% showed very narrow CIs, suggesting homogeneity in the VFs among districts. There was a significant correlation between VFs and the widths of its CIs (rho = )0.679, P < 0.001). Table 2 shows a selection of the questions to assess the quality of the immunization reporting system in each tier, with the percentage of countries, districts and HUs that correctly answered them, excluding DQAs with a high proportion of unreachable districts. In theory, immunization reporting mechanisms can be integrated within the national health management information systems or can be set apart as a parallel vertical reporting system only for immunization. DQAs showed that reporting of immunization data from the HU to the district level was integrated in 61% of the DQAs; and from the district to the national level in 55% of DQAs. Computers to manage immunization data were used in all national immunization programme offices and in 41% of district offices. In almost three quarters of the DQAs, immunization data was used to provide feed‐back from the national to the district immunization offices; slightly more than half of districts provided feed‐back to the HUs under their catchment area. Immunization data was also compiled in some type of publication in 82% of nationalimmunization offices and 58% of districts. However, immunization monitoring charts or tables could only be seen in less than half of the national immunization offices, 59% of districts and in a smaller proportion of HUs. DTP1‐3 drop‐out rates were monitored in a lesser proportion at all three levels. A relatively high proportion of HUs had some immunization reports or primary recording forms available, but only two‐thirds had a complete set of reports from the previous year. The use of consistent denominators is essential to obtain accurate immunization coverage figures. Almost all national immunization programmes used different figures in different years, reflecting the change in population size. However, this was not the case in districts with 87% using the same figures in different years. In 82% of the DQAs, denominators complied with the WHO recommended definition. In only 14% of the DQAs it was found that districts were using consistent denominators to those assigned by the national immunization programmes to each district within a country. The presence of guidelines for different immunization related procedures was variable. At district level, guidelines seemed to be more available than at national level. Vaccine ledgers to manage vaccine stocks could be found in the majority of national immunization programmes, in district offices and HUs holding vaccine stocks; however, a smaller proportion were updated. Vaccine wastage calculations could be confirmed in almost one‐third of national programmes and district offices and in two‐thirds of HUs. The answers to these questions were used to estimate QS for each level of the immunization reporting system. The median QS at national level was 3.3 out of 5.0 (interquartile range 0.7), 3.3 out of 5.0 (inter‐quartile range 1.1) in the 152 districts and 3.1 out of 5.0 (inter‐quartile range 1.6) in the 912 HUs across all districts and countries. Correlation analyses Figure 2 is a scatter chart depicting one ‘bubble’ per DQA, with the X and Y axis showing the aggregated HU and district QSs respectively. The size of the ‘bubbles’ is proportional to the QS measured at national immunization headquarters. There was a significant correlation between QSs measured at HU and at district levels (rho = 0.865, P < 0.001). Larger ‘bubbles’ tended to be found towards the upper right corner of the chart suggesting a significant correlation of national QSs with district and HUs scores (rho = 0.525, P < 0.001 and rho = 0.4843, P = 0.002 respectively). Figure 2 identifies countries with consistent poor or the lower left corner with small size bubbles: showing poor good performances. Central African Republic, Haiti, Lao, QS at all levels. At the far right upper end, Tanzania, Madagascar, Mauritania, Mozambique and Nigeria are in Burkina Faso, Guinea (second DQA) and Kenya (second DQA) show the highest scores (DQAs with a high very poor national QSs (Ethiopia, Tajikistan and Yemen) proportion of unreachable districts, excluded).
We also explored to what extent there could be examples of good quality districts (good district QS) in the poorest performing countries (poor national QS). QSs at national level significantly correlated with those of the best performing district in each country (rho = 0.408, P = 0.004). Looking at pairs of national‐QS and best district QS in that country data, there were several cases of outstanding performance at district level in countries with very poor national QSs (Ethiopia, Tajikistan and Yemen) and also cases of consistent poor national QSs with even the best districts also poorly performing (Central African Republic, Haiti, Madagascar, Mauritania and Nigeria). The VF did not show any significant correlation with level significantly correlated with those of the best national QS (rho = 0.211, P = 0.202). On the contrary, performing district in each country (rho = 0.408, there were significant correlations with districts and HUs P = 0.004). Looking at pairs of national‐QS and best QS (rho = 0.703, P < 0.001 and rho = 0.726, P < 0.001 district QS in that country data, there were several cases of respectively) (DQAs with a high proportion of unreachable outstanding performance at district level in countries with districts, excluded). From the 21 countries that failed the first DQA, six countries undertook a second DQA, 2–3 years later: Burkina Faso, Cameroon, Guinea, Kenya, Madagascar and Sudan. Data from the second DQA in Sudan has to be interpreted with caution since 25% of districts were unreachable. Table 3 summarizes the changes in VFs and QSs between both DQAs in each country. VFs improved in all cases. 95% CI narrowed in all cases except Cameroon. However, first and second DQAs’ VFs overlapped in all countries except Madagascar, suggesting that the true values of the VFs may actually not differ. At national level, the median change of QS across the six countries was +0.7. Some examples of improvements included: five of the six countries could estimate vaccine wastage in the second DQA while none of them could in the first one; in the second DQA, four countries had guidelines for electronic data management and for reporting AEFI while only one and none had them in the first one, respectively. At district level, the median change of the QS was +1.1. In all six countries, districts showed better use of immunization performance monitoring tools (tables and charts showing coverage), better vaccine record keeping and had guidelines in place for late reporting. At HU level, the median change of the QS was +2.0. HU QS improved statistically significantly in all six countries (see Table 3). Quality items that improved in all cases included the management of vaccine ledgers, the availability of reports and tally sheets and the display of an updated chart or table showing immunization performance indicators.
Discussion
Data accuracy Sources of bias in the estimation of immunization coverage have been widely described elsewhere and include inconsistencies in the reporting systems (WHO 2006a,b), which DQAs detect through the VF. Poor information systems do not only fail to portray the real situation of immunization coverage but are themselves barriers for scaling‐up immunization (GAVI 2003a; Papania & Rodewald 2006). The VF expresses the deviation of the national numeration estimate from its sources at HUs, where immunizations take place and the primary data is recorded in the first instance. These deviations can be partially explained by some of the findings in the systems quality questions. For example, there were missing primary records and reports (how many of these ever existed?) or guidelines for late reporting were frequently not found (how is information received after the termination of the reporting period actually treated?). The same problem was found when vaccine wastage could not be calculated. These findings highlight basic problems in the production, storage and reporting of immunization data in countries with poor VFs. Not surprisingly, VF correlated well with QS at HU and district levels, which are the sources of primary immunization data. Guidelines and training manuals on immunization, which include monitoring and data management, are easily available (WHO 2004a) and extensive training has taken place in many countries (Mutabaruka et al. 2005). Why, then, do the basic administrative and reporting practices seem not to have been followed in those countries with poor DQA outcomes? Many determinants of performance at subnational and local levels have been described (Mays et al. 2006), including remuneration, working conditions and factors directly related to health workers performance (Rowe et al. 2005). Whether this is pointing at a lack of knowledge or a poor organizational environment is beyond what DQAs can answer. However, we think that training on immunization issues will need to take into account the basics of recording, reporting and data management practices and look in detail at the organizational environment needed to translate knowledge into effective, routine practice. Countries The best performing countries achieved excellent VFs and QSs. Central African Republic, low in all QS, had a good VF. At the other extreme, Haiti, Madagascar, Mauritania and Nigeria showed consistently poor performance at all levels of the immunization reporting system. Should poorly performing countries be penalized without additional funding under a performance‐based system, as it has been the case with GAVI’s rewards? (GAVI 2006). Could a system aiming at rewarding performance and ensuring transparency end up having adverse effects on those countries in most need of help? Would countries facing a performance‐based system feel tempted to generate some ‘creative’ reporting to increase rewards (Brugha et al. 2002) or to redirect their efforts to increase overall coverage rather than reducing in‐country inequities (Starling et al. 2002). The answers to these questions are not straightforward. First, there are multiple factors which determine immunization performance, including health systemand contextual factors; secondly, in real life situations it is hardly possible to have ‘control’ countries to establish sound comparisons in order to describe key determinants of success or failure. Our findings, though, identified several countries that showed consistent poor performance and that may call for special attention. Nigeria, for example, was the only country where the VF could not even be calculated due to the lack of data, it has one of highest numbers of non‐immunized children in the world (WHO 2006a,b) and had more than half the cases of polio in 2006 (Global Polio Eradication Initiative 2007). GAVI has wisely responded to those concerns by considering separate policies for ‘fragile states’ (Brugha et al. 2002). Furthermore, districts within countries showed very different performance levels in the DQAs outcomes, suggesting that, besides nation‐wide factors, there might be local determinants that may contribute to find very good performing districts in not so good performing countries, as seen in the cases of Ethiopia or Tajikistan. DQAs as inducers of change The DQAs are an assessment tool. However, one of the outcomes of DQAs is the issuing of recommendations to assist HUs, districts and national immunization programmes to improve their reporting systems (GAVI 2003b). Therefore, DQAs aim to induce change, as well. Neither the design of DQAs nor the number of countries that undertook two DQAs can generate enough evidence to attribute the observed improvements to the DQAs themselves. However, in those countries that undertook two DQAs, improvements in the VFs were consistent with improvements in the QS, and showed statistically significant changes in the QSs at HU level. These improvements could be due to a ‘learning effect’ of the DQA method by countries, although districts and HUs in both DQAs were randomly selected and repetitions are very unlikely. DQAs certainly were an opportunity to raise quality issues and increase awareness on the consequences of poor data quality for programme management. Indeed, there is some evidence that failure to ‘pass’ a DQA has led to specific efforts (e.g. investment) in reporting information systems in a number of countries (Guinea, Laos, Tanzania and Zambia) (Abt Associates Inc 2007). The DQAs have a number of limitations (Ronveaux et al. 2005), some of them analysed in detail (Woodard et al. 2007); namely the wide CI of the VF, more imprecise at the medium and low ranges of the VF, the lack of verification of immunizations actually administered to children and the number of non‐eligible districts in a few countries.
Conclusion
DQA is a systematic methodology to describe in depth data quality issues and to provide recommendations to address them. DQAs can reveal a number of crucial problems that affect the quality of immunization data and provide countries with an opportunity to identify the weakest parts in the collection, transmission and use of information. Basic recording and reporting practices at the periphery of the system, alongside design aspects (e.g. denominators), have been identified as key factors that need to be tackled. DQAs also provide insights from all tiers of the health system, identifying good practices in some HUs and districts even in countriespoorly performing as a whole. Those HUs and districts can become drivers to improve reporting mechanisms in the countries. DQAs have been adapted into a self‐assessment tool (WHO 2004b) and can be simplified to assess specific aspects of the information system. In whatever form, DQAs bring data quality issues to the top front of the agenda to improve the monitoring of immunization coverage. Furthermore, the DQA methodology could be considered to address data quality issues across the spectrum of national disease control programmes (The Global Fund 2007) so data quality remains a priority to help improve planning and service delivery based on accurate coverage estimates.
Acknowledgements
DQAs were conducted with financing of the Vaccine Fund and the analysis of this set of data by WHO (Vaccine Assessment and Monitoring), number HQ ⁄ 05 ⁄ 051359. We thank Lorelei Silvester (LATH) for the administrative support; Ian Hastings, Brian Faragher (Liverpool School of Tropical Medicine) and Amanda Ross (Swiss Tropical Institute) for their contributions to the statistical methods. Charles Collins, Maria Paz Loscertales, Rete Trap and more specially Birna Trap, made suggestions about the manuscript at several stages. We also thank the staff of the immunization programmes at national, district and health unit levels for their open and intense collaboration during the implementation of the DQAs.
References
Abt Associates Inc (2007) Evaluation of the First Five Years of GAVI Immunisation Services Support Funding. Abt Associates Inc, Bethesda, MD. Anonymous (2006) Challenges in global immunization and the global immunization vision and strategy 2006–2015. Weekly Epidemiological Record 81, 190–194. Bchir A, Bhutta Z, Binka F et al. (2006) Better health statistics are possible. Lancet 367, 190–193. Brugha R, Starling M & Walt G (2002) GAVI, the first steps: lessons for the Global Fund. Lancet 359, 435–438. GAVI (2003a) Achieving our Immunisation Goal. Final report. McKinsey‐GAVI. GAVI (2003b) How to Prepare for a Data Quality Audit. Briefing Paper. Available from: http://gavi.elca‐services.com/resources/ DQABriefPaper02.pdf. GAVI (2006) Improving GAVI’s Engagement and Effectiveness in Fragile States. GAVI Alliance Board Meeting, 29 November 2006 (for decision). Global Polio Eradication Initiative (2007). Wild Poliovirus Weekly Update. Available from: http://www.polioeradication.org/casecount.asp (accessed 18 December 2007). Jones G, Steketee RW, Black RE, Bhutta ZA, Morris SS & the Bellagio Child Survival Study Group (2003) How many deaths can we prevent this year? Lancet 362, 65–71. Mays GP, McHugh MC, Shim K et al. (2006) Institutional and economic determinants of Public Health System performance. American Journal of Public Health 96, 523–531. Mutabaruka E, Nshimirimana D, Goilav C & Meheus A (2005) EPI Training Needs Assessment in 12 African Countries (2002– 2004) WHO‐Afro. Available from: http://www.afro.who.int/ ddc/vpd/tfi2005/tna_2002_2004.pdf. Papania M & Rodewald L (2006) For better immunisation coverage, measure coverage better (Comment). Lancet 367, 965– 966. Pearson M (2003) Improving the Health of the Nigerian People. DFID Health Systems Resource Centre, London, UK. Ronveaux O, Rickert D, Hadler S et al. (2005) The immunisation data quality audit: verifying the quality and consistency of immunisation monitoring systems. Bulletin of the World Health Organization 83, 503–510. Rowe AK, de Savigny D, Lanata CF & Victoria CG (2005) How can we achieve and maintain high‐quality performance of health workers in low‐resource settings? Lancet 366, 1026– 1035. Starling M, Brugha R & Walt G (2002) New Products into Old Systems. The Global Alliance for Vaccines and Immunization (GAVI) from a Country Perspective. Save the Children UK, London. The Global Fund (2007) Data Quality Assessment Tool. Guidelines for Implementation by an Auditing Team The Global Fund, Geneva. The LATH Consortium (2001) Immunisation Data Quality Audit. Evaluation Report. Final report. LATH, Liverpool, UK. The World Bank (1993) World Development Report 1993: Investing in Health. Oxford University Press, Oxford. Walt G & Buse K (2000) Editorial: partnership and fragmentation in international health: threat or opportunity? Tropical Medicine and International Health 5, 467–471. WHO (2003) . The Immunisation Data Quality Audit (DQA) Procedure. Vaccines and Biologicals, Geneva. WHO (2004a) Immunisation in Practice. A Practical Resource Guide for Health Workers. 2004 Update. WHO, Geneva. WHO (2004b) The Immunization Data Quality Self‐assessment (DQS) Tool. WHO Document WHO ⁄ IVB ⁄ 05.04. WHO, Geneva. WHO (2006a) The Reaching Every District Strategy. Fact sheet WHO ⁄ xx2006. WHO, Geneva.WHO (2006b) WHO Vaccine‐preventable Diseases: Monitoring System. 2006 Global Summary. WHO, Geneva. WHO ‐Immunisation, Vaccines and Biologicals. X. Bosch‐Capblanch et al. Accuracy of the immunization information systems WHO⁄ UNICEF (2005) GIVS – Global Immunization Vision and Strategy 2006–2015. WHO. Available from: http://www.who.int/vaccines‐documents/DocsPDF05/GIVS_Final_EN.pdf. Woodard S, Archer L, Zell E, Ronveaux O & Birmingham M (2007) Design and simulation study of the immunisation Data Quality Audit. Annals of Epidemiology 17, 628–633.
Tables and Figures
Table 1. Country profiles and DQA framework.
Region Country Country
code Number of districts Under 1s in audit year x1000 GNI (*) per capita (USD) Number of DQA Year audited Health units visited
Africa Burkina Faso BFA 53 504 211 2 2001, 2004 48
Burundi BDI 17 260 91 1 2002 24 Cameroon CMR 144 663 579 2 2001, 2003 45 Central African Republic CAF 22 134 281 1 2003 24 Chad TCD 53 302 437 1 2004 24 Congo COG 27 148 780 1 2004 24 Congo DR COD 481 2,245 111 1 2003 24 Côte d'Ivoire CUV 46 674 594 1 2001 24 Eritrea ERI 6 107 210 1 2003 24 Ethiopia ETH 71 2,352 99 1 2001 23 Ghana GHA 120 756 256 1 2001 21 Guinea GIN 38 329 345 2 2001, 2003 44 Kenya KEN 85 1,158 353 2 2001, 2003 48 Lesotho LSO 19 48 736 1 2003 24 Liberia LBR 18 118 121 1 2004 24 Madagascar MDG 111 599 252 2 2002, 2004 48 Mali MLI 58 421 241 1 2001 24 Mauritania MRT 53 113 390 1 2003 23 Mozambique MOZ 12 689 198 1 2001 14 Niger NER 42 550 158 1 2002 24 Nigeria NGA NA 5,054 345 1 2002 24 Rwanda RWA 39 338 195 1 2001 22 Senegal SEN 50 429 447 1 2002 24 Sierra Leone SLE 14 219 193 1 2003 24 Sudan SDN 129 1,001 365 2 2001, 2003 46 Tanzania TZA 135 1,377 271 1 2001 24 Togo TGO 35 199 295 1 2003 21 Uganda UGA 64 1,022 224 1 2001 24 Zambia ZMB 72 425 319 1 2002 24 Zimbabwe ZWE 59 365 387 1 2003 24
Asia Afghanistan AFG 32 943 141 1 2002 24
Bangladesh BGD 64 3,202 389 1 2001 24 Cambodia KHM 73 412 256 1 2002 24 Korea DPR PRK 206 420 579 1 2003 24 Lao PDR LAO 18 159 315 1 2002 24 Myanmar MMR 320 1,350 191 1 2003 24 Nepal NPL 75 737 221 1 2002(‡) 24 Pakistan PAK 115 5,262 498 1 2002 24 Tajikistan TJK 62 161 158 1 2001 19 Yemen YEM 286 599 498 1 2002 20
Caribbean Haiti HTI 11 286 457 1 2001 16
TOTALS 41 Countries 3,335 36,130 ‐ 47 ‐ 1,082 MEAN PER COUNTRY 83 881 322 ‐ ‐ 26 *Gross National Income per capita in US Dollars (UN; http://unstats.un.org/unsd/snaama/dnllist.asp). _Audit year 2001–2002. The number of HUs approached in three countries was less than 80% of 24, as stipulated in the DQA guidelines (Mozambique, Tajikistan and Haiti). Data on regions and countries is from UN (UN; http://unstats.un.org/unsd/methods/m49/m49regin.htm, UN; http://unstats.un.org/unsd/methods/m49/m49alpha.htm#ftna).
Table 2. Performance in a selection of quality questions at the three levels. Quality question % of the 38 DQAs % of the 152 districts % of 912 HUs Integration of immunisation reporting systems from
HUs to district level 61% NA( ) NA
Integration of immunisation reporting systems from
district to national level 55% NA NA
Use of computers to manage immunisation data 100% 41% NA Feed‐back on immunisation to lower level 71% 53% NA Publication with immunisation data 82% 58% NA Existence of chart or table showing immunisation performance indicators 45% 59% 53% Monitoring DPT1‐3 drop out rate 35%(*) 46%(†) 55%(‡) Availability of current tally sheets for DPT NA NA 82% Availability of reports NA NA 65% Use of different denominators according to year to estimate DTP3 coverage 97% 87% NA Denominators for DTP3 defined according to WHO
definitions 82% NA NA
Denominators used at national and district levels
coincide 14%(*) NA NA
Existence of data reporting guidelines 74% 89% NA Existence of guidelines to deal with late reporting 13% 50% NA Existence of guidelines to report AEFI(§) 32% 54% 83% Existence of vaccine ledgers NA 88% 85% Vaccines ledgers are up to date for DTP 79% 72% 65%(‡) Vaccines ledgers are up to date for TT(**) 84% 75% 49% Correct estimation of vaccine wastage 32% 31% 68% AEFI, adverse events following immunization; NA, not assessed at that level; TT, tetanus toxoid vaccine. *In 23 DQAs; _in 92 districts; _in 552 HUs.
Table 3. Compared performance of countries that undertook two DQAs.
Countries Year Verification factor Quality Scores
DQAs 95% CI National Distrital HU
2001 58% 19% 96% 3.2 3.3 2.5 Burkina Faso 2004 96% 81% 111% 3.9 4.3 4.4 Change +38% +0.7 +1.0 +1.9 (*) 2001 48% 15% 81% 3.6 2.9 2.1 Cameroon 2003 89% 53% 125% 4.3 4.4 4.1 Change +41% +0.7 +1.5 +2.0 (*) 2001 57% 1% 113% 3.0 3.3 3.5 Guinea 2003 95% 92% 99% 3.4 4.2 4.5 Change +38% +0.4 +0.9 +1.0 (*) 2001 50% 8% 91% 3.4 3.1 2.3 Kenya 2003 85% 68% 103% 4.0 4.1 4.3 Change +35% +0.6 +1.0 +2.0 (*) 2002 58% 42% 75% 2.4 2.7 2.3 Madagascar 2004 100% 83% 117% 3.5 4.4 4.0 Change +42% +1.1 +1.7 +1.7 (*) 2001 69% 18% 121% 2.6 2.7 2.1 Sudan 2003 96% 89% 103% 4.5 3.9 4.1 Change +27% +1.9 +1.2 +2.0 (*) CI, confidence interval. *P < 0.001 comparing the median QS of the 24 HUs in both years.
Figure 1. Verification factors (VF) in the 47 DQAs. CI, confidence intervals; VF, Verification Factor; Solid squares, African countries and Haiti; empty squares, Asian countries and Yemen. Figure 2. Scatter chart for the quality scores (QS) at the three levels. Good correlation between QS is shown by many ‘bubbles’ lying relatively close to the diagonal of the chart and their size growing from the lower‐left up to the upper‐right corners.
Chapter 4. There is no gold standard
to consolidate discrepancies in
vaccination coverage figures
between data sources: are we
globalising or trivialising access to
data?
Xavier Bosch‐Capblanch. Status: being submitted.Title
There is no universal ‘gold standard’ to consolidate discrepancies between data sources in vaccination coverage figures: are we globalising or trivialising access to data?
Xavier Bosch‐Capblanch1,2*, David W Brown3, Kaushik Banerjee4, Abdallah Bchir5, Tony Burton4 1 Swiss Tropical and Public Health Institute, Basel. 2 University of Basel, Basel. 3 United Nations Children Fund, New York. 4 Immunisation, Vaccines and Biologicals, World Health Organisation, Geneva. 5 The GAVI Alliance, Geneva. * E‐mail: x.bosch@unibas.ch
