Health information systems for program evaluation: How to connect the dots?

By Manish Kumar and Heidi W. Reynolds

Program evaluation and health information systems (HIS) are interconnected. It therefore is wise that those setting up HIS and people charged with conducting evaluations of health programs should be in touch with one another, as each of their respective roles impacts the other and efficiencies can be gained through synchronicity.

Functional HIS can enable collection, analysis, and use of high-quality, timely data to strengthen health service delivery and good policy decisions. It can enable policy makers, managers, and researchers to monitor the quality of programs being implemented, assess program effectiveness and efficiency, and analyze health trends among the population.[1]

Evaluation design is meant to produce information on the extent to which programs are performing well (and why), data on distribution of resources and equity of health services, and, therefore, the evidence needed to improve health outcomes. The data needed for such an evaluation design can be obtained by many means, but it is efficient if they can be gathered through the HIS itself. Intrinsic elements of HIS data have the potential to inform evaluations that can measure people’s exposure to programs, program outcomes of interest, and may allow investigation of variables regarding access or non-exposure to programs for establishing control or comparison groups.

Use of HIS data for program evaluation is an important and added value proposition to its role of monitoring program implementation, vital events, emerging health issues and disease outbreaks, and population health outcomes. In other words, use of HIS data for evaluation is efficient and increases the return on HIS investment as data become “reusable.”

Building on our experience with MEASURE Evaluation and in assessments of HIS, we strongly believe that deliberations focused on connecting HIS and evaluation are essential. These discussions should explicate the nuances of using HIS for evaluating programs—including how to determine when HIS data are appropriate data sources and when they are not.

These discussions should be based on what a country needs to know about its health services performance. This will establish what needs to be evaluated. A country also needs to know what data its health information system ought to be gathering to satisfy itself that the health needs of the population are being met.

And so, it becomes a sort of symbiosis: Efforts to define priority evaluation questions will reveal if an existing HIS can serve those questions or if the HIS design might need modification to capture more data. Conversely, when efforts are directed at improving HIS, this will signal issues that need to be evaluated—because countries ought to be evaluating performance against their own priorities. Attention to the needs of the HIS informs the evaluation community and the needs of that community should inform HIS design so that it produces data needed to answer evaluation questions.

Over the last 10 years, investments and interventions focused on HIS strengthening have grown, with an aim to improve health system performance and health outcomes.[2] The Health Data Collaborative is a multi-stakeholder global initiative working toward building strong national and subnational data systems and providing guidance on digital health investments that will allow integrated monitoring, review, and assessment of program performance.[3]

The results of investments and lessons learned in HIS initiatives present an opportunity for discussions to determine priority program evaluation questions in countries to further inform HIS system design. As HIS functionality improves, program and policy evaluation will become increasingly possible. It has the potential to reduce the cost of evaluations. And this will improve the return on investment for HIS strengthening activities. We’d call it a win-win.