http://orcid.org/0000-0002-9192-5695
Figures
Abstract
Background
Social accountability approaches are increasingly being employed in low-resource settings to improve government services. In line with the continuous quality improvement (CQI) philosophy that quality is the product of a linked chain, collaborative social accountability approaches like the Community Score Card (CSC) aim to empower clients and frontline service providers to transform their own lives and hold public officials to account for state obligations. Despite being a critical focus of collaborative social accountability approaches, to our knowledge, a quantitative survey of health workers to understand the impact of these approaches on their self-reported responsibilities and service provision has not been conducted. To fill this gap, we carried out a quantitative survey with health workers to assess the CSC’s impact on health worker-reported service responsibilities and provision and complement women’s self-reports.
Methods
We evaluated the effect of the CSC on reproductive health-related outcomes using a cluster-randomized design in Ntcheu district, Malawi. We matched 10 pairs of health facilities and surrounding catchment communities; one from each pair was randomly assigned to the intervention and control arms. The intervention communities and health workers each completed 3–4 cycles of the CSC process by endline. We then surveyed all health workers in the 20 intervention and comparison sites at endline (n = 412) to estimate the intervention’s impact.
Results
Significantly (p < .05) more health workers in the CSC intervention areas compared to control areas reported responsibility for antenatal care, comprehensive antenatal care counseling, recording of the number of pregnant and postpartum women seen each month, and the average age of their last family planning client was younger. In addition, marginally significantly (p < .10) more health workers in treatment versus control areas report visiting women at their home at least once during their pregnancy. However, health worker-reported responsibility for HIV testing was significantly lower in intervention areas than in control.
Conclusions
The CSC aims to empower health workers to collaborate with the community and rest of the health system to identify and overcome the diverse and context-specific range of performance barriers they face. In doing so, it aims to support them to demand and ensure quality care for themselves from the health system so they can, in turn, deliver quality services to clients. Our results contribute to the evidence that the CSC may hold promise at improving service provision. While there is increasing evidence that collaborative social accountability approaches like the CSC are effective means to improving reproductive health-related service provision and outcomes in low-resource settings, additional research is needed.
Citation: Gullo S, Galavotti C, Sebert Kuhlmann A, Msiska T, Hastings P, Marti CN (2020) Effects of the Community Score Card approach on reproductive health service-related outcomes in Malawi. PLoS ONE 15(5): e0232868. https://doi.org/10.1371/journal.pone.0232868
Editor: Florian Fischer, Ravensburg-Weingarten University of Applied Sciences, GERMANY
Received: October 31, 2018; Accepted: April 23, 2020; Published: May 19, 2020
Copyright: © 2020 Gullo et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: The Sall Family Foundation funded this work through a grant to CARE USA. The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The Funder provided support in the form of research materials and salaries for authors SG, CG and TM, but did not have any additional role in the study design, data collection and analysis, decisi’on to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section. Far Harbor, LLC, a public health research firm, was contracted by CARE USA to conduct data analysis and assist in the preparation of the manuscript.
Competing interests: The authors have declared no competing interests. The relationship with Far Harbor, LLC, the firm that conducted data analysis and assisted with the preparation of the manuscript, does not alter our adherence to PLOS ONE policies on sharing data and materials.
Introduction
Many healthcare workers in developing countries face challenging conditions, including inadequate and delayed salaries, competency gaps, heavy workloads, ambiguous responsibilities, lack of opportunities for growth, poor living and difficult working conditions, lack of autonomy, barriers to inter-professional collaboration and poor treatment by colleagues and patients [1,2,3,4]. These conditions not only result in burnout, stress, and poor retention for providers, but they also result in poorer quality care [1,3]. The connection between poor working conditions for providers and providers in turn providing poor quality of care to clients is in line with continuous quality improvement (CQI) philosophy. This philosophy suggests quality is a product of a linked chain in which each person is a customer of the people in the process preceding theirs and posits that for an organization to function well and provide quality care to its external customers it has to take care of the quality needs of its workers or internal customers [4,5,6,7]. A study from Ghana exploring the quality chain found that health workers perceived they do not receive ‘people-centered care’ from their employers, despite being asked to provide it to clients (i.e., the values they are being asked to hold for external clients are not being held for them by the health system). This hypocrisy weakened the message from the health system and in some cases lead to health worker apathy and less motivation to respond to external clients’ needs [7].
Social accountability is understood as “ongoing and collective effort to hold public officials to account for the provision of public goods which are existing state obligations” [8], including the delivery of quality services. While some social accountability approaches are adversarial others take a collaborative approach [9], recognizing that in many cases frontline health workers in developing countries are also disempowered and face challenging conditions. Collaborative social accountability approaches that improve dialogue between health workers and communities can lead to alliances to negotiate with higher level authorities for improvements [10]. Interestingly, a review of provider responsiveness to social accountability initiatives found health provider receptivity to citizen demands is mediated by the extent to which the social accountability initiatives provide personal and professional support to providers [11]. In addition, a review of evidence on interventions promoting external participation and accountability in low- and middle-income countries suggests that collaborative approaches to service provider engagement in transparency and accountability processes appear more effective than confrontational approaches and that many such interventions experience challenges stemming from lack of positive engagement with supply-side actors [12]. The collaborative approach to social accountability is line with the CQI philosophy that quality is a product of linked chain.
One such collaborative social accountability approach, CARE’s Community Score Card (CSC), aims to build “collaborative capacity” between service users and providers, empowering community members and service providers to identify obstacles and solutions in efforts to improve service utilization and quality [13]. Within the health sector, the CSC especially targets the frontline health workers who are the backbone of the health system but often feel disengaged and powerless in the decision-making process. Furthermore, the CSC involves district-level government officials and other power holders to help spur institutional and systemic change that can improve the working environment for health workers. By taking this approach, the CSC aims to improve two links in the quality chain (i.e., the external client and frontline provider links) and strengthen the relationship between the two. This is unlike many social accountability approaches that focus only on the external link in the quality chain.
In the health sector, there is promising evidence that collaborative social accountability approaches—like the CSC and the similar Citizen Voice and Action (CVA) approach—can improve relationships and spaces for negotiation between the community and health providers, and improve community empowerment, provider responsiveness, and ultimately enhance the availability, access, quality and uptake of services [12,13,14,15,16,17,18,19,20].
While it is important to understand how collaborative social accountability approaches impact client-reported health behaviors and service delivery experiences, to fully grasp the impact of these approaches it is critical to understand how these approaches impact health worker-reported service provision-related behaviors and experiences. This is important not only because frontline health workers play a critical role in the functioning of these social accountability approaches, but also because these approaches may lead to changes in service delivery and health worker behaviors that can be best (and is some cases only) ascertained from health worker self-reports. While several studies examining health sector collaborative social accountability approaches have used community and client-reported qualitative and quantitative data as well as process data (e.g., Score Card data) to understand the impact of these approaches, there has been less published on the impact of these approaches using health worker-reported service provision-related data [12,13,14,15,16,17,18,19,20]. To our knowledge, a quantitative survey of health workers to better understand a collaborative social accountability approach’s impact on their self-reported responsibilities and service provision as well as governance-related outcomes has not been carried out.
This paper presents the findings from a quantitative survey with health workers, exploring the CSC’s impact on health worker responsibilities and service provision in Malawi. This evidence is part of a cluster-randomized controlled evaluation examining the effect of the CSC on reproductive health-related outcomes in Malawi. In addition to the health worker survey, the evaluation also included Score Card data as well as a quantitative survey with women [17,19]. The health worker survey was used to bring the health workers’ voice to the evaluation of the CSC, which is often missing from evaluations of collaborative social accountability interventions. We sought to bring this voice through rigorous data collection, just as was done with the clients, and to triangulate this data with the women’s survey and Score Card data. The women’s survey showed that the CSC intervention had the following effect: significantly increased community health worker (CHW) visits to women during pregnancy and the postnatal period and women’s satisfaction with services, compared with control areas; had a significant effect on the use of modern contraception, with an estimated 57% greater use in the intervention versus control condition at endline; and improved spaces for negotiation and dialogue between women and health workers [17,19]. Analysis of the Score Card data, which included both community- and provider-rated indicators on areas prioritized to drive improvement efforts, showed that all 13 indicators (e.g., relationship between providers and communities, commitment of service providers, and level of youth and male involvement in reproductive health issues) improved from first to last scoring, with several improving significantly [17]. Complementing these findings, here we will present results from the health worker survey, sharing the CSC impact story using frontline health worker self-reported data.
Methods
Study setting
Much of Malawi’s population still lives in rural areas and relies on subsistence agriculture. The country’s health indicators remain relatively poor despite recent improvements. Ntcheu district lies near the border with Mozambique about half-way along the road between the country’s biggest cities of Lilongwe and Blantyre. Ntcheu has three hospitals and 33 health facilities administered by either the Malawi Ministry of Health (MOH) or the Christian Health Association of Malawi (CHAM). In 2011, 26 health facilities offered Prevention of Mother to Child Transmission of HIV (PMTCT) services, 6 provided basic emergency obstetric care (bEmOC), and 22 offered youth-friendly services. Family planning services were offered at all the MOH health facilities and some CHAM facilities.
Intervention theory of change and description
The theory of change underlying the CSC intervention (Fig 1) suggests that bringing together community members, health workers, and local officials to a) identify barriers and facilitators of service use and delivery, b) develop and implement local solutions to overcome these barriers, and c) jointly monitor improvements will result in governance outcomes. These governance outcomes include empowered women and communities, empowered health workers, and new and expanded spaces for inclusive, effective dialogue and negotiation between the two. These governance outcomes interact and lead to a) improved health behaviors, increased service utilization and satisfaction with services among users, b) systems and institutional changes, and c) higher quality and more equitable service delivery by providers. Ultimately, these changes should decrease maternal and neonatal mortality in communities. As the theory of change hinges on equal participation and actions from both the community-side and health-delivery-side, women’s and health workers’ quantitative surveys as well as process data were carried out to examine the theory of change. The results from the women’s survey and process data have been published [17,19], and this paper presents the health worker survey service delivery-related results.
The CSC intervention consists of five phases (Fig 2). The first phase of the CSC intervention involves planning and preparation. This stage involves identifying the sectoral and geographic scope of the initiative, understanding the context and barriers both service providers and users face, training facilitators, and securing cooperation and buy-in from all participating parties, including government officials as they are key partners in facilitating and responding to issues surfaced through the process. In Phase 2, issue generation, focus group discussions are carried out with community members (separated into groups such as men, women, youth, etc.) to identify and prioritize issues they are facing in accessing and utilizing services. Identified issues are organized into themes and a measurable indicator is developed for each theme (e.g., the relationship between providers and the community). The indicators are then verified and scored from 0 to 100 by the community, generating a Score Card. The community also indicates reasons for why a particular score was given and creates suggestions for improvement. The same process of issue generation and indicator development is conducted with service providers in Phase 3; through focus group discussions, service providers identify issues they are facing in delivering quality services, develop and score indicators, give reasons for the scores, and make suggestions for improvement. Phase 3 can occur either after or concurrently with Phase 2. Phase 3 is critical as it ensures that providers, who also often face significant challenges and are often disempowered have an opportunity to surface what is preventing them from delivering quality services. During Phase 4, the interface meeting, community members, service providers, local government officials and other power holders come together to discuss the community and service provider Score Cards, issues and priorities. This dialogue gives way to locally identified solutions to overcome barriers to service utilization and provision and an action plan for improvement. The action plan consists of items that need to be carried out by the community, service providers, local government and other power holders to improve service utilization and quality. Finally, Phase 5 involves action plan implementation, monitoring, and evaluation in which community members, service providers, government staff and additional power holders all have a role to play in implementing solutions as well as reviewing and monitoring progress on indicators. This cycle is repeated (minus the initial planning and preparation stage) every six months: communities and service providers reconvene to discuss issues (and generate new ones, if needed), re-score the indicators and discuss reasons for changes, and then meet in an interface meeting to review their respective Score Cards, in an on-going cycle of problem identification, solution generation, implementation of improvements, and mutual accountability. This process aims to foster conditions to build stronger relationships between community members and providers. It also aims to shift power to community members and frontline service providers so they can become self-learning, self-adaptive and self-leading in the co-creation of health and well-being.
The CSC intervention evaluated here focused on maternal and reproductive health-related outcomes such as family planning (FP), antenatal care (ANC) and postpartum care service utilization, and use of the health facility for labor and delivery. CARE Malawi in partnership with the local government District Health Management Team (DHMT) facilitated the CSC processes with service providers, including both facility-based service providers and CHWs, such as Health Surveillance Assistants (HSAs), and community members in the 10 intervention sites. All of the health facilities and workers that participated in the process were supported by the government of Malawi and administered either by the MOH or CHAM. Half of the intervention sites completed 4 cycles of the CSC process by the start of the endline data collection while the other half had completed 3 cycles. The Score Card indicators and results from the Score Card analysis are presented in Gullo et. al 2017 [17].
Study design
For our cluster-randomized design, we selected study sites based on health facility and surrounding catchment area, with 33 health facilities and surrounding catchment areas in the initial population. We excluded 13 of the 33 health facilities either because they did not provide PMTCT services or they did not have one or more of the required matching criteria. From the remaining health facilities, we created matched pairs based on four characteristics: presence of bEmOC services, facility administration (MOH or CHAM), proximity to the Mozambique border (which affects the population using the facility as well as the ability of health workers to provide services), and population size of the catchment area. This selection process resulted in ten matched pairs of health facilities. We randomly assigned one health facility from each pair to the treatment condition and the other to the control condition (Fig 3). The study was reviewed and approved by Malawi’s National Health Science Research Committee as a program evaluation.
Footnote: HF: health facility; PMTCT: Prevention of Mother to Child Transmission of HIV; bEmOC: basic emergency obstetric care.
Data collection
We collected baseline data (n = 280) between November and December 2012 and endline data (n = 412) between November and December 2014. We attempted to survey to all health workers in the catchment areas of the twenty participating health facilities at both time points. An independent data collection team from the Centre for Development Management carried out the data collection. The health facility in-charge was the entry point at each health facility. Service providers were requested to participate in the interviews at least 48 hours in advance. A majority of the providers were interviewed at the health facility and the remaining in the health facility catchment areas. All participants provided consent prior to the start of the survey. The survey was quantitative and covered the following areas: participant identification, background and demographic characteristics, service responsibilities and provision, intervention exposure and participation, and a range of governance outcome-related topics (e.g., social cohesion). The questionnaire included a majority close-ended questions and only a small number of open-ended questions. All the data were self-reported. This paper focuses on the CSC’s impact on service responsibilities and provision.
Measures
To assess service responsibilities, respondents indicated whether they had responsibility for each of the ten core health service domains: ANC, FP counseling, FP provision, HIV/PMTCT counseling, HIV testing, labor and delivery, postpartum follow-up, breastfeeding counseling, record keeping/tracking/monitoring, and supervision of other providers. Respondents were also asked if they performed record keeping functions in practice.
To evaluate service provision, respondents were asked questions related to working location and detailed questions on their provision of ANC and FP services. To assess working location, participants were asked the proportion of the working hours they spent at the health facility. ANC service provision questions for providers included the following: number of pregnant women and new mothers seen in the last month, proportion of pregnant women visited at home at least once during their pregnancy, number of times pregnant women were visited at home during their pregnancy, number of pregnant women counseled in the last month, and whether the most recent woman they provided services to received comprehensive ANC counseling. Comprehensive ANC counseling indicated that each of the six ANC topics had been discussed with all women seen in the past month: importance of getting check-ups during pregnancy; danger signs during pregnancy and childbirth; planning for delivery; getting tested for HIV; exclusive breastfeeding once the baby is born; and planning for contraception to avoid or delay another pregnancy after the baby is born.
FP service provision questions included the following: the number of women counseled about FP during the last month, details on who was provided FP services (i.e., the proportion of clients counseled on FP in the last month who were young women or adolescents as well as the average age, marital status and whether the client’s partner was present during the most recent FP consultation), and details on the most recent counseling session (i.e., whether counseling occurred in a private place, whether the woman was asked how long she wanted to wait before having a child/another child, whether HIV prevention was discussed, and whether comprehensive FP counseling was provided). Comprehensive FP counseling involved reporting all five recommended FP counseling practices during the most recent FP consultation: counseling the woman in a private place; asking how long she wanted to wait before having a child/another child; clarifying that it was her choice alone whether to use FP; discussing HIV prevention; and encouraging her to ask further questions.
Service provider self-reported ANC and FP service responsibility and provision items reported on here were developed to complement a more comprehensive set of questions around service provision and utilization included in the women’s survey and presented in Gullo et al. 2017 [17]. With the service provider survey we hoped to gain insights on the CSC’s impact on health worker responsibilities (e.g., did the CSC result in providers perceiving greater responsibilities?), working location (e.g., did the CSC result in more health workers providing services in the community?), clients served (e.g., did the CSC increase the number of adolescents receiving FP or men going with their partners for FP?) and additional details of FP and ANC service provision.
Analysis
Respondents classified their role as one of four types of health workers. At endline, 2.4% of respondents identified as doctor/clinical officer, 10.9% as nurse/midwife, 29.4% as patient attendant, and 45.4% as HSAs. In addition, 11.9% of respondents selected ‘other’ with a write-in response. Analytic models presented here are limited to the 232 nurse/midwife and HSA respondents at endline. The other categories were not directly responsible for patient care: patient attendants primarily focus on patient support and facility upkeep (though many reported some patient care activities); doctors/clinical officers largely work in an administrative capacity and represent <3% of the endline sample. Write-in job titles were related to either administrative roles or to non-health care roles (e.g., security guards). Analysis focused on ANC and FP-related services. The other health services could not be evaluated because too few health workers (<70%) identified them as primary job responsibilities, there was a ceiling effect at baseline which left little room for change, or it was mostly the responsibility of just nurses/midwives of whom there were too few at endline (n = 45) to analyze separately.
We limited our analyses presented here to the endline-only data based on two considerations. First, health workers provided survey data anonymously which precluded the possibility of linking records from the same health workers across time. We expected that many of the same individuals were surveyed at both time points based on the 100% sampling plan, creating unaccountable dependencies between the baseline and endline data. Second, the total sample size increased unexpectedly from baseline to endline. Follow-up correspondence with project staff indicated that a greater awareness of the project at endline may have contributed to an increased participation rate; thus, the baseline and endline samples were nonequivalent.
Prior to analyzing treatment effects, treatment condition was regressed on demographic characteristics in individual logistic regression models that adjusted for catchment area clustering; the global null hypothesis (i.e., a model containing a single demographic variable is equivalent to an intercept-only model) was used to evaluate the possibility of non-equivalence across conditions. Treatment effects for all outcomes were estimated using both regression models to estimate intention to treat (ITT) and instrumental variable (IV) models to estimate local average treatment effects (LATE) [21]. In the ITT analysis, treatment was based on the condition to which the participant was assigned regardless of their participation in the CSC. The LATE estimates represent the treatment effect as if there had been perfect compliance with the random treatment assignment and is useful in adjusting for noncompliance in real-world data. Prior to fitting the models, we evaluated the extent of non-compliance and determined that the data contain both forms of non-compliance: treatment-area cases that did not participate in the CSC (34.4%) as well as control-area cases that did participate in the CSC (8.9%). All models were fit using the Stata 14.2 svy command in which catchment area was the primary sampling unit (i.e., cluster); standard errors were thus estimated using robust variance estimates. All models included the following covariates: age, religion (Catholic v. not Catholic), sex, ethnicity (Ngoni v. not Ngoni), location where the health worker lives (at the facility, in the community, or does not live in the community), electricity in home, and the time required to reach a main road (less than 30 minutes, 30–59 minutes, 1–2 hours, more than 2 hours).
Results
Sample characteristics
Socio-demographic and household characteristics of HSAs and nurses/midwives are outlined in Table 1. A majority of the nurses/midwives are 40 years or older and are mostly female. However, the majority is also of non-Ngoni ethnicity and live in the facilities where they work with a metal roof and electricity. The majority of nurses/midwives have been in their current position for more than 2 years. The majority of HSAs are 31–40 years-old, mostly male, and of Ngoni ethnicity. They tend to live in one of the communities where they work, in homes with a metal roof but no electricity. Like the nurses/midwives, nearly all the HSAs have been in their current position for more than 2 years. The only significant difference in socio-demographic and household characteristics of HSAs and nurses/midwives between the treatment and control areas was that fewer HSAs and nurses/midwives in the treatment areas lived in the communities in which they worked (F[2, 18] = 3.94, p =. 038). The similarities between the treatment and control areas suggest that our cluster randomization process worked well in generating equivalence in individuals across conditions though, while not accounting for significant effects, nurses/midwives appeared to exhibit greater variability than HSAs.
Service responsibilities
HSA and nurse/midwife reported responsibilities across both treatment and control areas are outlined in Table 2. Health workers self-report being responsible for a wide range of sexual, reproductive, and maternal health services. HSAs nearly universally report being responsible for counseling about FP and breastfeeding across both treatment and control areas; approximately three-quarters report being responsible for providing FP and ANC, for HIV/PMTCT counseling, and for monitoring pregnant and postpartum women. Less than half of HSAs report being responsible for postpartum follow-up; however, approximately one-third report some supervisory responsibility of other providers.
A large majority of nurses/midwives across both treatment and control areas report being responsible for all the sexual, reproductive, and maternal health services assessed. Providing FP and HIV testing are the services for which the fewest number of nurses/midwives claimed responsibility, yet more than half did provide these services. Interestingly, 75% of HSAs across both treatment and control areas report being responsible for FP provision while 56% of nurses/midwives report having this responsibility. In addition, a little more than 77% of HSAs and 94% of nurses/midwives report recording the number of pregnant women or new mothers they see each month.
Service provision
HSA and nurse/midwife reported service provision across both treatment and control areas is outlined is Table 3. A little over 40% of HSAs and 97% of nurses/midwives from both treatment and control areas report spending half or more of their working hours at the health facility. Both HSAs and nurses/midwives report seeing a large number of pregnant women as well as counseling a large number for ANC during the past month. Fifty-five percent of HSAs report visiting all pregnant women in their catchment area at home during the last month whereas only 12% of nurses/midwives do, which likely reflects the HSAs’ role in the community. Eighty-seven percent of HSAs and 30% of nurses/midwives report visiting pregnant women at home in their catchment area 2 or more times during their pregnancy. A majority of HSAs and nurses/midwives report asking pregnant women if they plan to avoid or delay another pregnancy after their baby is born. A little over half of HSAs (51.8%) and a majority of nurses/midwives (75.0%) across both treatment and control areas report providing comprehensive ANC counseling to all of their clients in the past month.
Both HSAs and nurses/midwives in treatment and control areas report counseling a large number of women on FP each month. Just over 31% of HSAs and 62.5% of nurses/midwives report that half or more of the clients they saw in the last month were young women or adolescents. Furthermore, 21.6% of HSAs and 24.3% of nurses/midwives reported the last client they counseled on FP was 15–19 years old. A majority of both HSAs and nurses/midwives reported the last woman they provided FP services to was married, however, a minority of both reported the last woman they provided services to brought her partner. A majority of both HSAs and nurses/midwives reported that the last woman they counseled was counseled in a private place, was asked how long she wanted to wait before having a child/another child, and was counseled on HIV prevention. Over half of HSAs (56.8%) and 67.5% of nurses/midwives in both treatment and control areas report providing comprehensive FP counseling.
ITT and LATE
The ITT and LATE estimates of the CSC’s effect are outlined in Table 4. Both ITT and LATE estimates indicate that at endline a significantly greater proportion of HSAs and nurses/midwives in the treatment areas report being responsible for providing ANC than those in the control areas (p = .004 and p = .005, respectively). However, both ITT and LATE estimates indicate HSA and nurse/midwife reported responsibility for HIV testing was lower in treatment than in control areas (p = .041 and p = .038, respectively). Both ITT and LATE estimates indicate that a marginally significant (i.e., p < .10) proportion of HSAs and nurses/midwives in the treatment areas compared to those in the control areas reported visiting pregnant women in their catchment areas at least once during their pregnancy. ITT and LATE estimates both indicate a significantly greater proportion of HSAs and nurses/midwives in the treatment areas reported providing comprehensive ANC (p = .062 and p = .011, respectively) than in the control areas. Similarly, a greater proportion of HSAs in the treatment areas also reported recording the number of pregnant and postpartum women that they see each month than HSAs in the control areas (p < .001). ITT and LATE estimates of the CSC’s effect indicate that women receiving FP counseling are younger in the treatment areas than in the control areas (p = .026 and p = .023, respectively).
Discussion
In order to more fully understand the CSC’s impact on health worker responsibilities and service provision we carried out a quantitative survey with health workers in Malawi. We found that significantly more health workers in the CSC intervention communities compared to the control communities self-report the following at endline: responsibility for ANC, comprehensive counseling during ANC, recording of the number of pregnant and postpartum women seen each month, and the average age of their last FP client served was younger. In addition, marginally significantly more health workers in treatment versus control areas report visiting women at their home at least once during their pregnancy. To our knowledge, this is the first quantitative survey of health workers as part of collaborative social accountability approach evaluation efforts. Ensuring health workers and women both had voice in the CSC’s evaluation was important given the approach’s aim is to empower both, recognizing that quality is the product of a linked chain [4,5,6,7].
Significantly more health workers in the CSC intervention communities than control reported responsibility for ANC. Score Card data may be able to shed some light on this finding. Through the CSC process, providers and community members identified that availability and accessibility of maternal and newborn health (MNH) and FP information and services were barriers to reproductive health service utilization. In digging into these issues to come up with solutions, it was discovered that not all HSAs or community members knew that MNH care is a responsibility of HSAs. Therefore, actions were taken to train and support HSAs in delivering MNH. Analysis of the Score Card data showed that providers and community members reported improvements in both availability and accessibility of MNH and FP information and health services, with the availability and accessibility of information improving significantly [17]. A study in Ethiopia found role ambiguity to be a major stressor for CHWs [3]. The CSC aims to surface and overcome this type of provider-side stressor and obstacle alongside community-side challenges in efforts to improve service utilization and delivery.
Marginally significantly more health workers in treatment versus control areas report visiting women at their home at least once during their pregnancy. These findings are important because they align with the Malawi MOH expectations that HSAs should provide their services through door-to-door visits [22]. They also converge with our evaluation findings from the women’s survey data showing that the CSC increased the proportion of women who reported receiving a home visit during pregnancy [17]. The Score Card data and actions resulting from the CSC process shed light on this finding. The CSC process helped to facilitate support for HSAs from both the health system and community (e.g., capacity building from the district government, improvements in supervision, improvements in the relationship with the community, and community mobilization of resources to build HSA houses and clinics) [17]. While the CSC increased women receiving home visits during pregnancy, it also supported HSAs to do so. As George 2003 [10] suggests, accountability approaches that support interactions between communities and services can benefit both sides.
The CSC was also associated with more comprehensive counseling during ANC visits and increased monitoring and record-keeping. After years of primarily encouraging pregnant women in lower-income countries to obtain ANC and increase the number of visits they receive during pregnancy, recent efforts have focused on how to improve and measure the quality of those ANC visits [23]. Though comprehensive counseling is not the only indicator of ANC quality, it is an important indicator—especially in a low-resource setting like Malawi where ANC serves to educate women and build a relationship between her, her family, and the health system. In addition, most HSAs and nurses/midwives in both the treatment and control areas reported being responsible for monitoring and tracking pregnant women, but health workers in the treatment areas reported higher levels of actually doing so. Being able to track and monitor pregnant women in their catchment area is an important step in being able to follow-up with them, make referrals, and deliver quality care. Record-keeping by frontline health workers also helps from a health systems perspective because it provides a mechanism to monitor demand and capacity, and then to make adjustments as necessary. For this reason, data tracking by the health workforce is a key component of the WHO’s Global Strategy on Human Resources for Health: Workforce 2030 initiative [24]. These improvements in counseling and tracking converge with previous findings from our evaluation that the CSC significantly improved providers commitment as well as improved women’s satisfaction with services in the treatment areas [17].
The average age of the last FP client served was younger in the intervention communities versus control. The association we found between the treatment areas and slightly younger age for the last woman provided FP counseling may indicate a shift in the characteristics of women accessing FP and/or for whom providers feel that FP counseling is appropriate. Through the CSC process, increasing involvement of youth in reproductive health was an area identified by service providers and community members as important for accelerating reproductive health progress. In addition, it was an area that community members and service providers perceived improved significantly due to the CSC process [17]. Evidence suggests that provider bias poses a barrier to adolescents accessing and utilizing FP information and services [25,26,27,28]. The CSC may make health workers more amenable to providing FP counseling and services to adolescents as their knowledge of patients’ rights increases and as they see increased community investment in and commitment to reproductive health for youth. This is in line with research that suggests that social accountability approaches that emphasize dialogue and negotiation can help to mitigate social biases or prejudices among health professionals and improve their receptivity to clients’ needs [10]. Increasing adolescent and young women’s access to FP is a key priority of the Family Planning 2020 Initiative to which Malawi has committed itself [29]. Social accountability approaches that attack and overcome the demand and supply-side issues may hold promise in advancing towards this goal.
While the CSC did improve some aspects of service provision, there are some notable areas where the CSC appeared to have a negative effect or no effect at all. Interestingly, the CSC was associated with lower health worker-reported responsibility for HIV testing. We do not know how this occurred; however, it could be a result of HSA role and responsibility clarification that occurred as a result of the CSC process. Self-reported job responsibilities and service provision tend to match with MOH expectations for HSAs [22] across both the treatment and control areas, however, there are two notable exceptions. First is the 40% of HSAs who, even at endline, report spending half or more of their working hours in a health facility instead of in the community. The Malawi MOH established the HSA role as a community health worker whose main responsibilities involve visiting families in their homes to deliver promotive and preventive health services [30]. It appears that a large percentage of HSAs do not follow this community-based guideline. In addition, less than half of HSAs report being responsible for postpartum follow-up despite it being an official part of their job responsibilities and critical to accelerating maternal health efforts in the country. Perhaps these issues require greater focus in the CSC process in order for providers and community members to identify the obstacles and solutions to remedy these issues. Or perhaps the reason for the CSC not positively affecting these issues is that they require central level health system responsiveness, which the CSC and similar CVA have had limited success in garnering [13,16,17,18]. Despite finding that many HSAs report spending a significant proportion of their working hours in a health facility and a lower than expected percentage report responsibility for postpartum care, we found the CSC intervention significantly increased HSA visits to women during pregnancy and during the postnatal period compared to control [17].
Limitations
Several limitations may have affected our ability to identify significant differences in the treatment areas. First, we elected to accelerate the endline data collection by a full year because other organizations began planning maternal child health-related interventions and services in the control areas which would have contaminated our evaluation design. However, systemic changes facilitated by the CSC process may require a longer time period to operate in order to observe anticipated effects on service delivery outcomes. Second, we were unable to use the baseline data as planned to assess the impact of the intervention on service delivery outcomes over time. This limited us to estimating a treatment effect via LATE models at endline-only, a method which cannot rule out group differences in existence prior to intervention; thus, effects should be interpreted conservatively. Third, while the models adjusted for non-independence of observations attributable to catchment areas and a variety of person-level characteristics, we did not possess catchment-level characteristics that could be used to estimate contextual effects. Fourth, as the survey is with health workers about their responsibilities and service provision, it is subject to recall bias and social desirability bias. As the analysis was conducted after the intervention, there is some possibility that healthcare providers exposed to the intervention are aware of what is expected which could increase a positive response bias. There were hopes to use facility records to triangulate the findings; however, the high number of missing records and poor quality of recordkeeping made them unusable. Fifth, while carrying out a quantitative survey with health workers allowed a much broader sample, we were unable to gain the depth in understanding that a qualitative approach would have allowed. Sixth, the ‘Cashgate’ scandal froze donor funds to Malawi for a while during the intervention period [31] and strained the country’s public services including the health system. Finally, randomization was conducted at the health center catchment area and not the health worker level; nevertheless, health workers only differed in the proportions living in the community across treatment conditions.
Conclusions
The CSC aims to empower health workers to collaborate with the community and rest of the health system to identify and overcome the diverse and context-specific range of performance barriers they face. In doing so, it supports them to demand and ensure quality care for themselves from the health system so they can, in turn, deliver quality services to clients. Here we present the CSC’s impact on health workers’ self-reported service responsibilities and provision in Malawi. These results, taken with the women’s survey and Score Card data results [17], suggest the CSC may hold promise at improving service provision. While there is increasing evidence that collaborative social accountability approaches like the CSC are effective means to improve reproductive health-related service provision and outcomes in low-resource settings, additional research is needed. Research, including qualitative and realist evaluations, is needed to unpack the black box of how these approaches work as well as understand stakeholders’ (including frontline service providers’) experiences of and satisfaction with these approaches.
Acknowledgments
We want to acknowledge and thank the CARE Malawi team who ensured that the intervention was implemented as planned and who oversaw all aspects of program operations, Palikena Kaude, District Team Leader, Kriss Chinkhota, Project Coordinator, and Eliza Mhango, Project Coordinator. We also wish to thank the Ntcheu District Health Management team for the generous staff support that made this program possible, and the CARE Malawi Country Director, Michael Rewald, and the Assistant Country Director, Rose F. Tchwenko, who provided support throughout. We also want to acknowledge Carolyn Grant and Anne Laterra for their insights, ideas and hard work to support the project, and Noor Tirmizi for his help with the research design. Thanks too to the health workers and community members who gave their time to participate in the survey as well as to carry forward the ideas and actions generated during the CSC process.
References
- 1. Munabi-Babigumira S, Glenton C, Lewin S, Fretheim A, Nabudere H. Factors that influence the provision of intrapartum and postnatal care by skilled birth attendants in low- and middle-income countries: a qualitative evidence synthesis. Cochrane Database Syst Rev [Internet]. 2017 Nov 17;(2). Available from: http://doi.wiley.com/10.1002/14651858
- 2. El Koussa M, Atun R, Bowser D, Kruk ME. Factors influencing physicians’ choice of workplace: systematic review of drivers of attrition and policy interventions to address them. J Glob Health [Internet]. 2016 Dec;6(2). Available from: http://www.jogh.org/documents/issue201602/jogh-06-020403.pdf
- 3. Selamu M, Thornicroft G, Fekadu A, Hanlon C. Conceptualisation of job-related wellbeing, stress and burnout among healthcare workers in rural Ethiopia: a qualitative study. BMC Health Serv Res [Internet]. 2017 Dec 19;17(1):412. Available from: http://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-017-2370-5 pmid:28629360
- 4. Agyepong IA, Anafi P, Asiamah E, Ansah EK, Ashon DA, Narh-Dometey C. Health worker (internal customer) satisfaction and motivation in the public sector in Ghana. Int J Health Plann Manage [Internet]. 2004 Oct;19(4):319–36. Available from: http://doi.wiley.com/10.1002/hpm.770 pmid:15688876
- 5.
Mclaughlin C, Kaluzny A. Continuous quality improvement in health care: Theory, implementation, and applications. Aspen Publishers. Gaithersburg, MD; 1995.
- 6. Agyepong IA, Sollecito WA, Adjei S, Veney JE. Continuous Quality Improvement in Public Health in Ghana. Qual Manag Health Care [Internet]. 2001;9(4):1–10. Available from: http://content.wkhealth.com/linkback/openurl?sid=WKPTLP:landingpage&an=00019514-200109040-00002 pmid:11499347
- 7. Aberese-Ako M, van Dijk H, Gerrits T, Arhinful DK, Agyepong IA. “Your health our concern, our health whose concern?”: perceptions of injustice in organizational relationships and processes and frontline health worker motivation in Ghana. Health Policy Plan [Internet]. 2014 Sep 1;29(suppl 2):ii15–28. Available from: https://academic.oup.com/heapol/article-lookup/doi/10.1093/heapol/czu068
- 8. Houtzager PP, Joshi A. Introduction: Contours of a Research Project and Early Findings. IDS Bull [Internet]. 2007 Dec;38(6):1–9. Available from: http://doi.wiley.com/10.1111/j.1759-5436.2007.tb00413.x
- 9.
O’Meally SC. Mapping context for social accountability: a resource paper [Internet]. Washington, DC; 2013. Available from: http://siteresources.worldbank.org/EXTSOCIALDEVELOPMENT/Resources/244362-1193949504055/Context_and_SAcc_RESOURCE_PAPER.pdf
- 10. George A. Using accountability to improve reproductive health care. Reprod Health Matters [Internet]. 2003 May;11(21):161–70. Available from: http://www.ncbi.nlm.nih.gov/pubmed/12800713 pmid:12800713
- 11. Lodenstein E, Dieleman M, Gerretsen B, Broerse JEW. Health provider responsiveness to social accountability initiatives in low- and middle-income countries: a realist review. Health Policy Plan [Internet]. 2017 Feb;32(1):125–40. Available from: https://academic.oup.com/heapol/article-lookup/doi/10.1093/heapol/czw089 pmid:27375128
- 12.
Waddington H, Sonnenfeld A, Finetti J, Gaarder M, Stevenson J. Does incorporating participation and accountability improve development outcomes? Meta-analysis and framework synthesis [Internet]. US; 2019 May. Available from: https://www.3ieimpact.org/sites/default/files/2019-05/SR43-PITA-report.pdf
- 13. Gullo S, Galavotti C, Altman L. A review of CARE’s Community Score Card experience and evidence. Health Policy Plan [Internet]. 2016 Dec;31(10):1467–78. Available from: https://academic.oup.com/heapol/article-lookup/doi/10.1093/heapol/czw064 pmid:27190223
- 14. Edward A, Osei-Bonsu K, Branchini C, Yarghal TS, Arwal SH, Naeem AJ. Enhancing governance and health system accountability for people centered healthcare: an exploratory study of community scorecards in Afghanistan. BMC Health Serv Res [Internet]. 2015 Dec 31;15(1):299. Available from: http://dx.doi.org/10.1186/s12913-015-0946-5
- 15. Ho LS, Labrecque G, Batonon I, Salsi V, Ratnayake R. Effects of a community scorecard on improving the local health system in Eastern Democratic Republic of Congo: qualitative evidence using the most significant change technique. Confl Health [Internet]. 2015 Dec 3;9(1):27. Available from: http://dx.doi.org/10.1186/s13031-015-0055-4
- 16.
Wales J, Wild L. CARE’s experience with community score cards: what works and why? [Internet]. London; 2015. Available from: http://insights.careinternational.org.uk/publications/care-s-experience-with-community-score-cards-what-works-and-why-synthesis-report%5Cnhttp://insights.careinternational.org.uk/media/k2/attachments/CARE-experience-with-CSCs-synthesis-report.pdf
- 17. Gullo S, Galavotti C, Sebert Kuhlmann A, Msiska T, Hastings P, Marti CN. Effects of a social accountability approach, CARE’s Community Score Card, on reproductive health-related outcomes in Malawi: A cluster-randomized controlled evaluation. Ciccozzi M, editor. PLoS One [Internet]. 2017 Feb 10;12(2):e0171316. Available from: pmid:28187159
- 18. Schaaf M, Topp SM, Ngulube M. From favours to entitlements: community voice and action and health service quality in Zambia. Health Policy Plan [Internet]. 2017 Jul;32(6):847–59. Available from: https://academic.oup.com/heapol/article-lookup/doi/10.1093/heapol/czx024 pmid:28369410
- 19. Gullo S, Kuhlmann AS, Galavotti C, Msiska T, Nathan Marti C, Hastings P. Creating spaces for dialogue: a cluster-randomized evaluation of CARE’s Community Score Card on health governance outcomes. BMC Health Serv Res [Internet]. 2018 Dec 14;18(1):858. Available from: https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-018-3651-3 pmid:30428881
- 20. Community Matter PTY LTD. Citizen Voice and Action for Government Accountability and Improved Services: Maternal, Newborn, Infant and Child Health Services [Internet]. 2018. Available from: https://gpsaknowledge.org/wp-content/uploads/GPSA-Indonesia-CVA-Project-Final-Evaluation-Report-2018.pdf
- 21. Angrist JD. Instrumental variables methods in experimental criminological research: what, why and how. J Exp Criminol [Internet]. 2006 Apr 27;2(1):23–44. Available from: http://link.springer.com/10.1007/s11292-005-5126-x
- 22.
Government of Malawi M of H. Malawi Health Sector Strategic Plan 2011–2016 [Internet]. Lilongwe; 2011. Available from: http://www.nationalplanningcycles.org/sites/default/files/country_docs/Malawi/2_malawi_hssp_2011_-2016_final_document_1.pdf
- 23. Hodgins S, D’Agostino A. The quality–coverage gap in antenatal care: toward better measurement of effective coverage. Glob Heal Sci Pract. 2014; pmid:25276575
- 24.
World Health Organization. Global strategy on human resources for health: Workforce 2030 [Internet]. Geneva; 2016. Available from: http://who.int/hrh/resources/global_strategy_workforce2030_14_print.pdf?ua=1
- 25. Warenius LU, Faxelid EA, Chishimba PN, Musandu JO, Ong’any AA, Nissen EBM. Nurse-Midwives’ Attitudes towards Adolescent Sexual and Reproductive Health Needs in Kenya and Zambia. Reprod Health Matters [Internet]. 2006 Jan 19;14(27):119–28. Available from: https://www.tandfonline.com/doi/full/10.1016/S0968-8080%2806%2927242-2 pmid:16713886
- 26. Wood K, Jewkes R. Blood Blockages and Scolding Nurses: Barriers to Adolescent Contraceptive Use in South Africa. Reprod Health Matters [Internet]. 2006 Jan 19;14(27):109–18. Available from: https://www.tandfonline.com/doi/full/10.1016/S0968-8080%2806%2927231-8 pmid:16713885
- 27. Calhoun LM, Speizer IS, Rimal R, Sripad P, Chatterjee N, Achyut P, et al. Provider imposed restrictions to clients’ access to family planning in urban Uttar Pradesh, India: a mixed methods study. BMC Health Serv Res [Internet]. 2013 Dec 23;13(1):532. Available from: https://bmchealthservres.biomedcentral.com/articles/10.1186/1472-6963-13-532
- 28. Tilahun M, Mengistie B, Egata G, Reda AA. Health workers’ attitudes toward sexual and reproductive health services for unmarried adolescents in Ethiopia. Reprod Health. 2012;
- 29.
UKAID, Bill & Melinda Gates Foundation. London summit on family planning: summaries of commitments [Internet]. 2013. Available from: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/67328/london-summit-family-planning-commitments.pdf
- 30.
Kadzandir JM CW. Role of HSAs in delivery of health services and immunizations in Malawi [Internet]. Lilongwe; 2001. Available from: http://www.eldis.org/go/home&id=56114&type=Document#.WJC3P38TqUk
- 31.
J.O’S. The $32m Heist: Malawi’s “Cashgate” Scandal [Internet]. 2014 [cited 2019 Dec 6]. Available from: http://www.economist.com/blogs/baobab/2014/02/malawi-s-cashgate-scandal