Making Decisions on Data Quality
The landscape has become somewhat crowded with tools for assessing health data quality in low- and medium-resource countries. Just recently, the United States President’s Emergency Plan for AIDS Relief (PEPFAR) and the Centers for Disease Prevention and Control (CDC) published excellent tools to strengthen data for measuring outcomes for people on treatment for HIV. What should countries and health planners turn to for assessing their data when so many good options are available?
A new publication, Comparative Analysis of Data Quality Assessment (DQA) Tools, published by MEASURE Evaluation, which is funded by the United States Agency for International Development (USAID) and PEPFAR, outlines the similarities and differences among these new tools—comparing their objectives and the scope of their methods. The publication is intended to aid countries to understand the intended purpose of these tools and to guide them in selecting the appropriate one for their context.
“The comparison of methods was in response to these new tools,” says David Boone, PhD, an epidemiologist and data quality activity manager for MEASURE Evaluation. “USAID wanted to know which one to use and when—and this helps to answer that question.”
Boone is a proponent of Country-Led, Holistic Data Quality Assurance, an approach embodied by a suite of tools and guidelines that contributes to USAID’s vision of improving the evidence base for public health monitoring, evaluation, and planning, by improving the quality of routine health data. The holistic approach is centered on the Data Quality Review (DQR) methodology and toolkit which provides countries a systematic and routine means of ensuring a minimum standard of quality for health data and identifies areas in need of strengthening. The DQR assesses data quality across program areas in a routine and standardized manner, coordinated by a government-led central technical working group. This approach improves efficiency and reduces burden on the health workforce by meeting the needs of stakeholders in one routine, carefully planned and coordinated assessment which obviates the need for ad hoc and overlapping program-specific assessments.
The guidance is the product of collaboration with international health program experts from leading donors and technical assistance agencies, such as the World Health Organization (WHO), USAID, Gavi, the Vaccine Alliance, and the Global Fund to Fight AIDS, Tuberculosis and Malaria (Global Fund). MEASURE Evaluation assisted in the development of the guidance and tested approaches to improve country ownership and leadership of data quality assurance.
MEASURE Evaluation recently published guidance on instituting a multi-stakeholder central-level coordinating mechanism for data quality assurance centered on the DQR methodology, Country-Led, Holistic Data Quality Assurance: Institutionalizing Data Quality through a National Technical Working Group and the Data Quality Review. The guidance provides instructions for establishing a technical working group (TWG); using the DQR for collecting and preparing data for analysis, conducting data verifications, analyzing and interpreting results; and best practices for countries on how and when to apply the methods.
“We feel the holistic approach is the best way to improve quality, coordination, and efficiency in data quality assurance,” he says. “Organizations or programs needing more in-depth information on data quality can still conduct program-specific assessments using their choice of tool, but this should be in collaboration with and coordinated by the central TWG to ensure country ownership and capacity to assess and improve data quality. We have made progress in reducing fragmentation in routine health information systems, thereby improving coherence and quality in the data. We want to avoid vertical approaches if possible so as not to lose these gains.”
“The DQR for country-led data quality assurance sets out a way for countries—not outside donors—to lead and manage data quality with their own partners and ministries calling the shots,” says Boone. “We were trying to show if you could put a TWG in place and work collaboratively, this approach would help countries improve data quality and and meet the needs of various stakeholders. The DQR is the centerpiece of this approach.”
Boone says that WHO gets the lion’s share of the credit for developing and promoting the holistic approach—an approach he advocates as well. “With good information up front and an understanding of the strengths and limitations of the data, health planners can make good decisions about service provision and resource allocation,” Boone says. “This is important—you wouldn’t bet big on something you weren’t sure of.”
