CEP attending Our Bright Future Annual Seminar

CEP HELPING LEAD A SESSION ON FINDINGS FROM THE OUR BRIGHT FUTURE MID-TERM EVALUATION REPORT AT ANNUAL SEMINAR on 19 June 2019

CEP’s Owen White is attending this year’s annual Share Learn Improve seminar in Belfast of the Our Bright Future (OBF) programme as part of CEP’s role in the evaluation of the OBF programme.  The seminar brings together all of the youth and environmental projects to discuss and share experiences and good practice in management and delivery.

As part of this seminar, Owen will be contributing to an interactive working session to engage participating projects in the findings of the Mid-Term Programme Evaluation Report, and identify specific actions and next steps for projects and the OBF Programme team.

The evaluation team is led by ERS with CEP and is carrying out the evaluation of the programme between 2016 - 2021. The Programme Evaluation seeks to identify, analyse and assess:

• the collective impact of the portfolio of 31 projects;

• the added value of the Programme i.e. what value has been derived from a programme with the Share Learn Improve, Policy and Youth Function and from the development of an Our Bright Future network/movement;

• whether the Programme has achieved its long-term ambitions; and

• best practice and lessons learnt, and provide evidence and guidance to support Programme learning and development.

Please contact Owen White (Technical Director) for more information.

Evaluating policy interventions: What role for Theory of Change?

Evaluating policy interventions: What role for Theory of Change?

Blog post by Sian Morse-Jones

What is a Theory of Change (ToC)?  Different terminologies/approaches exist. From an evaluation perspective a ToC commonly articulates how an intervention (e.g. policy, programme, project) is expected to lead to an ultimate goal(s) by showing what needs to happen, in what order and in what way. It establishes the ‘how’ and ‘why’ activities lead to outputs, outcomes and ultimately goals/impacts, explaining the assumptions underpinning this. Usually presented in a diagram or map, a ToC allows big picture thinking, and can help to contextualise where an intervention sits alongside other influences, depicting how external factors may also influence the goal. 

Example Theory of Change diagram for Countryside Stewardship Facilitation Fund (CEP, 2015, Report to Defra)

In evaluation, having a clear understanding of an intervention’s ToC is incredibly useful for a variety of reasons. It can clarify the causal relationships between different activities, outputs, outcomes and goals and highlight the chief assumptions which underpin why these contribute to specific goals. This can provide pertinent information to inform the evaluation framework, questions, criteria, evidence and needs.

A frequent challenge in the evaluation of policy interventions is that impacts/goals are often long term, for example, the biodiversity benefits from habitat improvement or creation may take years to materialise, or as in the case of emergencies such as flooding, may not manifest within the timeframe of the evaluation. The ToC is a powerful tool in such contexts because it provides a theoretical basis for evaluating these in terms of activities, outputs or outcomes which are measurable.

Evaluation of an intervention also provides the opportunity to further test and explore the validity of the ToC, enabling further refinements. Because the ToC tells us that achieving goals rely on specific assumptions, it can be very helpful to explore and consider whether assumptions are correct, why, and in what circumstances? Similarly, if assumptions are not correct, why are they not? What’s not working? In this way, a ToC-based evaluation can help to surface valuable lessons to benefit the design of an intervention, as well as informing policy and practice.

CEP has  much experience in using ToC to evaluate policy interventions – applications include:

• Our work on Our Bright Futures  to evaluate how, and to what extent, a programme and portfolio of projects aimed at young people, has led to progressive change in outcomes for the young people, the environment, their communities and the economy, as well as the long-term influence and legacy.

• In the monitoring and evaluation of Nature Improvement Areas for Defra, and

• In evaluating a project on community engagement on flood risks for Natural Resources Wales.

In CEP’s experience, understanding the theory behind ‘how’ and ‘why’ an intervention will have an impact in the real world is a vital part not just of designing effective interventions, but also in evaluation. Yet it is surprisingly rare for policy interventions to be well articulated in a ToC, often because an explicit policy cycle / clear role for how evaluation can be used in future policy is lacking[1].  Getting the ToC right at the outset may not only result in a stronger intervention, but also a more robust and efficient evaluation process, drawing out key lessons for the future direction of policy and implementation.

Dr Sian Morse-Jones, Senior Consultant, CEP, 22 October 2018

[1] For more details see CEP’s meta-evaluation of 10 years’ of our projects: Learning the Lessons for Evaluating Complexity across the Nexus.

CEP attending Our Bright Future Annual Seminar

CEP HELPING LEAD A SESSION ON EVALUATING OUTCOMES FROM THE OUR BRIGHT FUTURE PROGRAMME AT ANNUAL SEMINAR on 20 and 21 June 2018

Owen White is attending this year’s annual Share Learn Improve seminar in Cardiff  of the Our Bright Future (OBF) programme as part of CEP’s role in the evaluation of the OBF Programme.  The seminar brings together all of the projects to discuss and share experiences and good practice in management and delivery. 

As part of this seminar, Owen will be contributing to a presentation on the measurement and evaluation of social and environmental impacts. He will also co-facilitate a working session in which participants will explore assumptions related to the achievement of impacts from project activities and share knowledge and experience in the monitoring and evaluation of impacts.

The evaluation team is led by ERS with CEP and is carrying out the evaluation of the programme between 2016 - 2021. The Programme Evaluation seeks to identify, analyse and assess:

• the collective impact of the portfolio of 31 projects;
• The added value of the Programme i.e. what value has been derived from a programme with the Share Learn Improve, Policy and Youth Function and from the development of an Our Bright Future network/movement;
• Whether the Programme has achieved its long-term ambitions; and
• Best practice and lessons learnt, and provide evidence and guidance to support Programme learning and development.

Please contact Owen White (Technical Director) for more information.

CEP to evaluate Future Farming Scotland programme

CEP commissioned to evaluate Soil Association Scotland's Future Farming Scotland programme

CEP has been commissioned to evaluate Soil Association Scotland's Future Farming Scotland Programme. The programme aims to promote skills development and knowledge transfer in the agricultural sector in Scotland supporting the delivery of various competitiveness and environmental outcomes. CEP will be evaluating both process and outcome aspects of the programme by surveying beneficiaries and preparing case studies of certain interventions. The project builds on CEP's evaluation and assessment expertise in Scotland covering various aspects of land based natural capital. 

CEP's Dr Peter Phillips and Izzy Cotton will be working on this project.

CEP to contribute to ‘Our Bright Future’ evaluation learning events

CEP attends evaluation learning events for the Big Lottery funded Our Bright Future (OBF) programme, leading a session on how to evaluate policy campaigns

Dr Clare Twigger-Ross and Owen White are attending evaluation learning events in London (6th February 2018) and Bristol (13th March 2018).  These events are part of a series that are intended to help OBF projects design, deliver and evaluate policy and advocacy campaigns as part of their overall delivery.  As part of these events Clare and Owen will be leading a practical working session on the monitoring and evaluation of policy campaigns, with the aim of improving participants’ knowledge and capacity.

These events are being delivered as part of the wider project to evaluate the Our Bright Future programme.  The Programme Evaluation seeks to identify, analyse and assess:

• The collective impact of the portfolio of 31 projects;
• The added value of the Programme i.e. what value has been derived from a programme with the Share Learn Improve, Policy and Youth Function and from the development of an Our Bright Future network/movement;
• Whether the Programme has achieved its long-term ambitions; and
• Best practice and lessons learnt, and provide evidence and guidance to support Programme learning and development.

For more information on this project please contact Clare Twigger-Ross or Owen White.

CEP presenting at the European Environmental Evaluators Network Forum

Dr Clare Twigger-Ross is presenting at the European Environmental Evaluators Network Forum in Edinburgh, 23 -24th November 

The 2017 Forum will ask whether evaluation is evolving in line with the societal and environmental challenges policy is trying to address. Europe, in order to achieve its goal of “living well, within the limits of our planet”, will need to rely inter alia on innovation to make the transition to sustainability. Many forms of innovation will be required to turn Europe into a resource-efficient, green and low-carbon economy.  

Clare is presenting  work carried out by a team at CEP (Dr Bill Sheate, Dr Clare Twigger-Ross, Owen White, Rolands Sadauskis, Paula Orr, Liza Papadopoulou and Ric Eales):  "Learning Lessons from evaluations across the nexus: a meta-evaluation", which examined CEP evaluations in the context of complexity and impact, developing insights and key questions for future evaluations.   This is being presented as part of a session from CECAN on Innovations in instrument evaluation - integrating complexity into Environmental policy evaluation: Insights from CECAN on Thursday afternoon.   The project report and summary can be found here.