Methodological and Policy Limitations of Quantifying the Saving of Lives: A Case Study of the Global Fund’s Approach.


Summary Points

·         A recent trend in global health has been a growing emphasis on assessing the effectiveness and impact of specific health interventions.

·         For example, it has been estimated that 8.7 million lives were saved between 2002 and mid-2012 by “Global Fund–supported programmes” (as distinct from The Global Fund alone) through antiretroviral therapy (ART); directly observed tuberculosis treatment, short course (DOTS); and distribution of insecticide-treated mosquito nets (ITNs).

·         This paper assesses the methods used by The Global Fund to quantify “lives saved,” highlights the uncertainty associated with the figures calculated, and suggests that the methods are likely to overestimate the number of “lives saved.”

·         The paper also discusses how the attribution of “lives saved” to specific programmes or actors might negatively affect the overall governance and management of health systems, and how a narrow focus on just ART, DOTS, and ITNs could neglect other interventions and reinforce vertical programmes.

·         Furthermore, the attribution of “lives saved” to Global Fund–supported programmes is potentially misleading, because such programmes include an unstated degree of financial support from recipient governments and other donors.

Discussion

This paper argues that the number of “lives saved” that are attributed to Global Fund–supported programmes is not as certain as has been suggested by The Global Fund, and is likely to be an overestimate. Furthermore, estimating the “lives saved” by Global Fund–supported programmes is confusing and potentially misleading, because such programmes include a considerable but unstated amount of financial support from other sources. Finally, a number of potentially negative policy effects are associated with the selective impact estimation of downstream clinical interventions.

While this paper focuses on The Global Fund, the issues raised here apply to other global health partnerships and international donor agencies that are increasingly under pressure to quantify the health impact of their investments. The methods for estimating and attributing “lives saved,” and the consequences of doing so, should be questioned and subjected to critical debate.

In the case of The Global Fund, for a start, greater clarity and explanation about the assumptions and generalisations of the methods are required; this should include publication of uncertainty ranges and of disaggregated estimates of “lives saved” for each of the three interventions and for each year. The Global Fund should also conduct and publish sensitivity analyses, particularly in relation to treatment effectiveness, and publish estimates of “lives saved” through DOTS based on alternative counterfactual scenarios.

If the health impact of ART, DOTS, and ITNs is to be estimated in the form of “lives saved,” we argue that this should not be done as an exercise focused on individual external agencies, but rather on the collective contributions of governments and development partners within countries. This would confer a number of benefits. First, the monitoring of service delivery outputs and the estimation of their health impact would be linked to an assessment of the performance of national health systems (a more appropriate unit for assessment) and the degree to which development partners are working in harmonisation with each other and in alignment with ministries of health and their national plans and priorities. This would help shift more attention towards the strengthening of integrated national plans and information systems.

Second, holistic assessments of service delivery results and health improvement at the country level would allow for a context-based analysis of performance, including assessments of efficiency and equity. This would be aided by cross-country comparisons that would reveal variations in effectiveness (and efficiency) of ART, DOTS, and ITNs that arise from differences in, amongst other things, access to health care, quality of care and treatment adherence, and population coverage of nonclinical determinants of health such as access to clean water and nutrition. By describing this variation, policy attention can be directed not just at the delivery of selected clinical interventions, but also at the social, economic, and environmental conditions that influence the degree to which those interventions are effective. This stands in marked contrast to a modelling approach that assumes standardised levels of effectiveness across countries or regions.

Third, estimates of “lives saved” at the country level might be more valid and less uncertain because they would be derived from more appropriate and country-specific modelling assumptions, and because it would motivate countries to improve the quality of their data. In addition, it could stimulate other actors within countries, such as parliamentary health committees, universities, and local nongovernmental organizations, to develop the capacity to scrutinise the performance of the health system. While many countries produce annual health reports, health needs assessments, and national health plans, which provide some description of progress in the health sector, they are often incomplete or weak. Subnational analyses are frequently absent or superficial; and the fragmented and piecemeal nature of reporting systems, encouraged by vertical and donor-driven DAH, still undermines the development of coherent planning, budgeting, management, and information systems.

While an estimate of “lives saved” by ART, DOTS, and ITNs at country level would still be limited by its narrow focus on three interventions, it would provide a platform for monitoring and evaluating other aspects of HIV, TB, and malaria programmes and be more easily incorporated into a national system of data collection and evaluation that takes into account a wider package of health systems inputs, processes, and outputs, enabling policy makers and planners to consider the importance of investments that do not have a measurable or immediate mortality impact.

If individual external agencies need to estimate their specific contribution to “lives saved,” this could be done more simply by apportioning a share of a country’s estimated number of lives saved on the basis of their proportional financial contribution to THE or total HIV/AIDS, TB, and malaria programme financing. This would provide a more meaningful assessment of the contribution of individual agencies, avoid double-counting in reported estimates of “lives saved” by external agencies, and incentivise external agencies to promote coherent national health planning and reporting.

Many of these recommendations  are applicable to external agencies in general. However, since 2012, The Global Fund has been providing more active support for detailed national evaluations of programme performance and impact, and more accurate measures of disease incidence, prevalence, mortality, and morbidity in 20 to 25 “high-impact” countries. This provides it with an opportunity to shift emphasis away from estimating “lives saved” by individual interventions and donor-supported programmes, towards an assessment of health systems performance and impact that incorporates all major actors, programmes, and interventions, and a fuller assessment of the contribution of social, economic, and other upstream determinants of health.

Source:PLOS

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.