Qualitative Comparative Analysis in Case Management Systems: Worker Support Is a Predictor of Better Outcomes

By Zulfiya Charyeva, MEASURE Evaluation

For the past eight years, I have focused on research and evaluation of programs that provide services to orphans and vulnerable children (OVC) and their families. These programs provide services to improve overall health and well-being of their beneficiaries. Almost by definition, these kinds of programs are multifaceted and causal pathways are hard to determine.

As with many other programs worldwide, OVC programs rely on the local workforce, such as community health workers, case workers, and health volunteers. What I found perplexing is why certain case workers achieve better results than others. Obviously, personal characteristics can partially explain differences. However, I am more interested in understanding differences at the level of organizations: specifically, what can organizations do to better equip their workers to improve beneficiary outcomes? I chose to answer this question by mapping causal pathways for a complex intervention—a solution that might work for other programs, too.

My project, MEASURE Evaluation, funded by the United States Agency for International Development (USAID) and the United States President’s Emergency Plan for AIDS Relief (PEPFAR), conducted a study of COVida—a USAID-funded, national OVC program in Mozambique. COVida supports roughly 300,000 OVC and their caregivers each year to access high-quality services. We knew that some community-based organizations (CBOs) in the COVida program had better HIV outcomes than others. How, we wondered, could COVida best influence those factors so that other CBOs also would enjoy improved performance? We also wanted to produce evidence-informed, actionable recommendations for programs and donors in Mozambique on how to shift their strategies and, ultimately, their resource allocations, to optimally balance service quality and cost.

Our objective was to identify what combinations of modifiable case management attributes would lead to a defined outcome: improved knowledge of HIV status among beneficiaries. Incorporating collaborating, learning, and adapting (CLA) into the study, we worked with USAID and COVida to define the study research questions and methods. We chose to use qualitative comparative analysis (QCA) to unpack the complex effects of modifiable attributes of case management—such as caseload, training of case workers, and supervision structure. We thought QCA would be a good choice because it would provide a richer contextual picture of what led to changes in knowledge of HIV status. Linear and logistic regression models that we have used before are adequate for identifying one factor at a time, but less effective for our interest in how factors combine to create an effect—in other words, when and where do different combinations of modifiable factors become important in reaching key outcomes. We chose six CBOs in three provinces and analyzed data from surveys with 70 case workers and their supervisors, plus COVida routine data.

Here’s what we learned:

1. Numerous factors improve knowledge of HIV status. No single attribute can explain changes in our outcome of interest. Instead, it is a combination of factors.

2. A few factors are chief contributors to the outcome:

• Case worker experience or training
• Case worker support through high-quality team meetings, one-on-one supervision, and a low number of case workers per supervisor
• An appropriate number of cases per case worker and fewer complex cases (i.e., HIV-positive beneficiaries)
• Transportation and phone access for case workers

We found six “pathways” that led to a change in the outcome of interest—each pathway with a different combination of case management attributes that contributed to the change. In identifying these pathways, the study showed ways to employ CLA to change processes to improve case management. All paths shared one condition: how cases are assigned. A formal process that considered complexity, caseload, experience, skills, and proximity in assigning cases was at least a partial factor in every instance of a high percentage change in HIV status knowledge.

3. Alternative paths can achieve the outcome. Figure 1, below, illustrates the six pathways we identified. Each pathway is independent of the others and each one leads to an increase in HIV status knowledge—but all of them are examples of employing CLA to learn from experience. Depending on the available resources, a CBO may focus on any one of the six.

For example, if a CBO cannot recruit experienced case workers—attributes in four of the six pathways—it can pursue one of the other two pathways. One path could be to address case worker retention and improving the quality of team meetings, for example. Or, an alternate path could be to assign fewer complex cases to case workers, reimbursing for out-of-pocket expenses, and allowing more time per case.  

Figure 1. Pathways to improve HIV status knowledge

In summary, in order to improve knowledge of HIV status, we recommend that programs embrace CLA and try these tactics for improved performance:

• Implement a formal process to assign cases, considering case complexity and existing caseload to reduce overwork.
• Provide case workers with at least two types of support, such as weekly care team meetings, weekly supervisor meetings to address challenges and develop case management plans or establishing low supervision ratios so that managers are more available.
• Hire experienced case workers.
• Provide all case workers with follow-up training so that they have the tools to address challenging cases.
• Expand the financial resources offered to case workers, such as increasing stipends, implementing bonuses, and reimbursing them for work-related expenses.

As part of the CLA process, COVida adapted its programming by providing new instructions to CBOs and much closer supervision of the process of assigning cases. The updated process considers caseload, work experience, skills, case complexity, and worker proximity to a case. The process is meant to prepare case workers for effective case management and to ensure they are not overburdened and that they have enough time to address the needs of each beneficiary.

Prior to the QCA study COVida had already introduced changes to use monitoring tools managed by case workers themselves, upgraded the chief case worker qualifications for management, and a bonus system to incentivize performance, among other changes.

Our findings from the QCA study strengthened the project’s arguments for having introduced these changes and provided evidence that making these changes was the right path.

For more information, see the full study report and brief and access a recorded webinar.

Reprinted with permission from the USAID Learning Lab.