When to incorporate evaluation into your program

What is evaluation?

The Canadian Evaluation Society provides a synthesized definition of evaluation:
"Evaluation is the systematic assessment of the design, implementation and results of an initiative for the purposes of learning or decision-making."

Why is evaluation important?

Evaluation plays a central role through the lifecycle of public programs. Designing and applying an evaluation framework informs planning, strengthens performance, and supports investment decisions. This function becomes increasingly critical during periods of economic uncertainty and fiscal constraint.

When in the program lifecycle should we apply evaluation?

Evaluation supports learning and evidence-based management at multiple stages through a program lifecycle:

1. Design Stage
At the beginning of a program lifecycle, during the Design Stage, program evaluation is used to set up the program for success in achieving measurable results. It involves clarifying and confirming objectives and target outcomes for a program. It directs the program structure and implementation to align with intended outcomes. This stage may involve testing the program theory, developing a logic model (diagramming how activities will lead to outputs and outcomes), and deciding on performance indicators and data collection strategies.

2. Implementation Stage
While the program is being implemented, evaluation examines how the program is being delivered in order to optimize performance. Key roles include determining whether activities are being actioned as intended; identifying operational issues or implementation barriers; reviewing efficiency, reach and service delivery quality; and obtaining feedback for course correction.

3. Outcome Stage
Once the program has been running for an amount of time by which it is expected to have produced some outcomes, evaluation focuses on assessing the effectiveness of the program and whether it represents a sound investment. At this stage, evaluation determines to what extent the program is achieving intended outcomes; assesses value for money; and compares results to targets or benchmarks.

4. Renewal or Transition Stage
In many cases a program will be ongoing indefinitely. In other cases, such as a pilot initiative, the program will reach a renewal, transition or termination stage. At this point, evaluation analysis informs strategic decisions about the future of the program, ensuring public resources are allocated in a way that demonstrates value and impact. Evaluation provides recommendations with respect to renewing, scaling, redesigning or sunsetting the program. It zooms in on lessons learned to inform future decisions or initiatives, and provides evidence for funding approvals and/or policy decisions.

The following graphic summarizes the role of evaluation through the program evaluation life cycle.

An example...

Over 2012-13 the United Kingdom rolled out the Universal Credit program (UC), which consolidated six legacy, income-based benefits into a single monthly payment. The system includes elements for housing, children, disabilities, and caring responsibilities (the six legacy benefits included the Jobseeker’s Allowance, Employment and Support Allowance, Income Support, Working Tax Credit, Child Tax Credit, and Housing Benefit). Claimants sign an agreement setting out work-search expectations, and failure to meet conditions may lead to sanctions.

The purpose of the UC was to simplify and reform the benefit system to make it fairer, more affordable and better able to tackle poverty, worklessness and welfare dependency.

(Source: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/48897/universal-credit-full-document.pdf)