Evaluation and program planning
-
Eviction from housing is associated with several negative outcomes, further exacerbated among high-need populations requiring financial and supportive services to maintain housing stability. This study investigated risk and protective factors-both characteristics and precipitating events of tenant eviction-informing permanent supportive housing (PSH) programs' efforts to identify tenants at risk and intervene. ⋯ PSH providers may use these correlates of eviction to identify Veterans in need of an intervention to prevent eviction. Future work should focus on operationalizing these findings and identifying appropriate interventions.
-
Theories of program and theories of evaluation form the foundation of program evaluation theories. Theories of program reflect assumptions on how to conceptualize an intervention program for evaluation purposes, while theories of evaluation reflect assumptions on how to design useful evaluation. These two types of theories are related, but often discussed separately. ⋯ Pragmatic synthesis is being developed to serve these moderate- complexity programs. These three theoretical perspectives have their own strengths and challenges. Knowledge on these three perspectives and their evaluation implications can provide a better guide for designing fruitful evaluations, improving the quality of evaluation practice, informing potential areas for developing cutting-edge evaluation approaches, and contributing to advancing program evaluation toward a mature applied science.
-
This paper describes a framework that can help refine program theory through data explorations and stakeholder dialogue. The framework incorporates the following steps: a recognition that program implementation might need to be multi-phased for a number of interventions, the need to take stock of program theory, the application of pattern recognition methods to help identify heterogeneous program mechanisms, and stakeholder dialogue to refine the program. As part of the data exploration, a method known as developmental trajectories is implemented to learn about heterogeneous trajectories of outcomes in longitudinal evaluations. ⋯ The framework is highlighted with data collected in an evaluation of an alcohol risk-reduction program delivered in a college fraternity setting. The framework discussed in the paper is informed by a realist focus on "what works for whom under what contexts." The utility of the framework in contributing to a dialogue on heterogeneous mechanism and subsequent implementation is described. The connection of the ideas in paper to a 'learning through principled discovery' approach is also described.
-
Many widely-used impact evaluation designs, including randomized control trials (RCTs) and quasi-experimental designs (QEDs), frequently fail to detect what are often quite serious unintended consequences of development programs. This seems surprising as experienced planners and evaluators are well aware that unintended consequences frequently occur. ⋯ We present a numbers of cases to illustrate how infusing RCTs with a mixed-methods approach (sometimes called an "RCT+" design) can strengthen the credibility of these designs and can also capture important unintended consequences. We provide a Mixed Methods Evaluation Framework that identifies 9 ways in which UCs can occur, and we apply this framework to two of the case studies.
-
Historically, effectiveness of community collaborative prevention efforts has been mixed. Consequently, research has been undertaken to better understand the factors that support their effectiveness; theory and some related empirical research suggests that the provision of technical assistance is one important supporting factor. The current study examines one aspect of technical assistance that may be important in supporting coalition effectiveness, the collaborative relationship between the technical assistance provider and site lead implementer. ⋯ Results suggest that community collaborative prevention work should consider the collaborative nature of the technical assistance provider - prevention community team relationship when designing and conducting technical assistance activities, and it may be important to continually assess these dynamics to support high quality implementation.