Abstract

Objectives International donors financing the delivery of antiretroviral treatment (ART) in developing countries have recently emphasized their commitment to rigorous evaluation of ART impact on population health. In the same time frame but different contexts, they have announced that they will shift funding from vertically-structured (i.e., disease-specific) interventions to horizontallystructured interventions (i.e., staff, systems and infrastructure that can deliver care for many diseases). We analyze likely effects of the latter shift on the feasibility of impact evaluation. Methods We examine the effect of the shift in intervention strategy on (i) outcome measurement, (ii) cost measurement, (iii) study-design options, and the (iv) technical and (v) political feasibility of program evaluation. Results As intervention structure changes from vertical to horizontal, outcome and cost measurement are likely to become more difficult (because the number of relevant outcomes and costs increases and the sources holding data on these measures become more diverse); study design options become more limited (because it is often impossible to identify a rigorously defined counterfactual in horizontal interventions); the technical feasibility of interventions is reduced (because lag times between intervention and impact increase in length and effect mediating and modifying factors increase in number); and political feasibility of evaluation is decreased (because national policymakers may be reluctant to support the evaluation). Conclusions In the choice of intervention strategy, policymakers need to consider the effect of intervention strategy on impact evaluation. Methodological studies are needed to identify the best approaches to evaluate the population health impact of horizontal interventions.