The Strategy Design and Evaluation Initiative is concerned with whether, how, when and why public health strategies deployed to alleviate complex problems achieve greater and lesser success. Governments use strategies for a variety of problems including, for example: tobacco, cannabis, poverty, sexual violence, gambling, drug overdoses, and housing. What does and does not get included in these strategies and how they are implemented will have a substantial effect on success in alleviating social problems. Yet, the science of strategy design is painfully underdeveloped. Conceptual models of strategies suggest the need for multi-component, multi-level interventions supported by system enablers such as capacity building, advocacy and learning systems and brought together by a shared moral imperative. However, there is almost no systematic empirical study of the actual design of the many social strategies that are being implemented and of the effect of different designs on achieving strategy goals under a variety of circumstances.
The Initiative has three prongs:
- Research on Strategy Design and Evaluation: Conducting research to understand what are the principles of good strategy design and testing these principles empirically to understand what works in what context.
- Supporting Strategy Design and Evaluation: The Initiative provides support to governments and NGOs in designing and evaluating complex strategies. We use our expert knowledge to help to improve individual strategies, and at the same time garner new knowledge to inform our program of research.
- Knowledge Translation and Community Building
Ontario Alcohol Strategy
We worked with key stakeholders providing input to the Government of Ontario’s forthcoming Alcohol Strategy to develop an Intervention Path Logic Model, which is a visual representation of a comprehensive and integrated strategy to ameliorate the burden of alcohol related harms in Ontario. The model was submitted to the Government of Ontario on behalf of a diverse group of stakeholders.
Municipal Drug Strategies of Ontario
The SDEI completed a two year contract to develop an Evaluation Framework that can be used and adapted to evaluate the over 30 municipal drug strategies. The framework provides a modular approach to guide strategies to use different evaluation approaches and activities to answer different types of evaluation questions as evaluation needs and resources available for evaluation activities change. The framework also provides an exemplar logic model for a “typical” drug strategy, exemplar evaluation questions, indicators, and a resource list of methods that can be used to gather data. An emphasis is put on measuring different types of behaviour change among strategy partners to measure changes in outcomes over time.
The full framework can be viewed here: MDSCNO Evaluation Framework [PDF]
SSHRC Insight Grant
We have been awarded an SSHRC Insight Grant to conduct an in-depth study of 12 strategies to study their design, perceived success and perceptions of what elements of design contribute to success and what is missing in order to achieve greater success. Findings will be synthesized across strategies in a series of open access manuscripts. A publicly available toolkit will be developed and purposefully disseminated to knowledge users- those who are in the business of designing social strategies- in particular in government as well as NGOs. The toolkit will summarize what we have learned about how strategy design affects perceived outcomes, the relative importance of these design elements, and how contexts affect the relative importance of particular design elements for achieving perceived outcomes.
Tobacco Control Strategy
Through the Ontario Tobacco Research Unit we conduct ongoing monitoring and evaluation of the Smoke-Free Ontario Strategy. We have played key roles in developing tobacco endgame strategies at both the federal and provincial levels
Evaluation of HKPR Opioid Strategy
We have an agreement with Haliburton, Kawartha, Pine Ridge District Health Unit to evaluate their developing Opioid Strategy. This work is underway.
Accountability for Horizontal Initiatives
We are conducting research about approaches to accountability for initiatives, including their evaluation.
- The Public Health Agency of Canada has provided a Health System Impact Fellowship placement for a PhD student. This creates the basis for a broad collaboration with PHAC on strategy design and evaluation, starting with the emerging tuberculosis eradication strategy.
- We are working with the Public Health Agency of Canada, the McConnell Foundation and with other groups to conceptualize projects related to strategy design and evaluation -A CIHR Operating Grant is being considered to conduct a formative evaluation of local opioid response strategies, with a focus on comparative approaches to naloxone distribution.
- A substantial proposal has been submitted to CIHR to conduct research on the impact of health strategy design on achieving strategy outcomes.
- An LOI is being considered by Health Canada, to conduct a developmental evaluation of Municipal Opioid Strategies across Ontario. The evaluation would encourage innovation and cross learning across communities, with the goal of understanding best practices and scaling up in similar contexts across Canada.
Knowledge Translation and Community Building
We have identified the need to build a public health strategy design community in order to build interest amongst partners, learn from those involved with strategies, and to build an audience to uptake the results of the research and evaluation work.
Robert Schwartz leads the Strategy Design and Evaluation Initiative. He is Professor at the Institute of Health Policy, Management and Evaluation at the Dalla Lana School of Public Health, University of Toronto. Dr. Schwartz is also the Executive Director of the Ontario Tobacco Research Unit. Rob has made important contributions to the literature on evaluating complex strategies and has applied this knowledge in evaluating the Smoke-Free Ontario Strategy. His formal training is in political science specializing in public policy and administration.
Emily Taylor, a seasoned Research Officer with the Strategy Design and Evaluation Initiative has in depth knowledge of strategy design principles and is involved in several projects related to strategies spanning from drug strategies to tuberculosis strategies. She has superior qualitative research and evaluation skills, and research coordination experience.
Joslyn Trowbridge is a PhD Candidate in the Social and Behavioural Health Sciences Division at the Dalla Lana School of Public Health. She holds a Master of Public Policy from the Hertie School of Governance in Berlin and a Joint Honours Bachelor of Arts from McGill University in Montreal. She has experience in developing and implementing provincial health system improvement strategies for the Ontario Ministry of Health and Long-Term Care and regional and municipal governments. Her research investigates the use of theoretical approaches in health behaviour change strategies.
To date, the following are involved in existing and proposed projects:
- Dr. Michael Chaiton
- Dr. Erica DiRuggiero
- Dr. Mark Dobrow
- Dr. Diane Finegood
- Dr. John Garcia
- Dr. Pamela Kaufman
- Dr. Pamela Leece
- Dr. Greg Marchildon
- Dr. Shawn O’Connor
- Dr. Alex Price
- Dr. Carol Strike
- Dr. Terry Sullivan
- Dr. Dan Werb
- K. Forss, M. Marra, R. Schwartz (Eds.), Evaluating the complex: Attribution, contribution, and beyond, Vol. 18, Transaction Publishers, New Brunswick (2011)
- OTRU. Smoke-Free Ontario Strategy Monitoring Report. (2017). http://otru.org/2016-smoke-free-ontario-strategy-monitoring-report/
- Alcohol Strategy Intervention Path Logic Model (pdf document attached)
- R. Schwartz, G Pais. Challenges and Approaches to Evaluating Comprehensive Complex Tobacco Control Strategies, Vol. 24(3). (2009)
- L. Johnson, C. Matteson, D. Finegood. Systems Science and Obesity Policy: A Novel Framework for Analyzing and Re-thinking Population Level Planning. Am J Public Health, 104(7) (2014).
- M. Barber, A. Moffit, P. Kihn. Deliverology 101: A Field Guide for Educational Leaders. London: Sage
Strategies are a multicomponent set of policy and program interventions, including system enablers, employed toward the achievement of social goals. As such, our interest is not in single policies, but in strategies that include several policy and programmatic interventions. Importantly, our use of the word ‘strategy’ differs from the use of the term in management science where ‘strategy’ is “about where an organization is headed and how it intends to get there”.