• Lalit Dandona, Magdalena Z Raban, and Rakhi Dandona.
• Public Health Foundation of India, ISID Campus, 4 Institutional Area, Vasant Kunj, New Delhi 110070, India. latit.dandona@phfi.org
• Natl Med J India. 2011 Sep 1; 24 (5): 263-8.

AbstractBACKGROUND. Analysis of the scope and quality of evaluations of health system/policy interventions done in India is not available. Such analysis can help in conducting more useful evaluations. METHODS. We accessed evaluation reports of health system/ policy interventions aimed at improving population health in India, reported during 2001-08, which were available in the public domain through extensive internet searches. We developed and used a classification system for the type of evaluation, commissioning agency, health system/policy area covered and methodology used, and a method for assessing the quality of evaluation reports. RESULTS. Of the 219 total evaluation reports in the public domain, 6% assessed needs, 22% process, 42% outcome and 30% impact. Seventy-six per cent evaluations were commissioned by international agencies. Among health system components, services were the focus of evaluation in 74.9% of reports, with human resources, financing, drugs/products, information system and governance having little representation. Only 21% of evaluation reports were rated as good quality. Among evaluations based mainly on health system data, 42% were poor quality compared with 20% that were based on population data. Seventy-two per cent of the outcome/impact evaluations presented only basic tabulations and just 12% attempted multivariate analysis. Eighty-two per cent of the outcome/impact evaluations had no controls, among which 42% were poor quality versus 17% poor quality among outcome/impact evaluations with controls. Among the 54% evaluations in which the intervention implementer was involved, only 1% reported negative conclusion about the intervention compared with 37% among evaluations in which the implementer was not involved. CONCLUSION. This analysis of health system/policy intervention evaluation reports from India identifies specific areas that need improvement. We recommend that Indian agencies should commission more evaluations as international agencies currently dominate, involvement of intervention implementer in the evaluation needs scrutiny as potential bias is suggested by our analysis, and health system components other than services need more attention. Outcome/impact evaluations need to incorporate controls in design and multivariate techniques more often in their analysis to achieve higher quality robust evaluations in India.Copyright 2011, NMJI.

12 keywords...

hide…