Define
the purpose of the intervention or policy being evaluated
|
Asking the
right question
|
Can you identify the question
being asked of the programme, and is it defined
appropriately for the context of the programme and its
evaluation?
|
Identifying
mechanisms
|
Have the
mechanisms for change as a result of the programme been
identified or hypothesized and
do they take account of the dynamics of the complexity of the
programme and its evaluation?
|
Defining
expected outcomes
|
Has the
evaluation defined what “success” looks like for key
stakeholders including program recipients? Have the outcomes of
the programme been defined, and are they relevant to the question
being asked? Do the outcomes include the OECD criteria of
relevance, coherence, effectiveness, efficiency, sustainability
and impact? Do the outcomes encompass other potentially relevant
criteria, such as equity?
|
Measure
and understand process and outcome
|
Collecting
data
|
Is the
programme description sufficient and does it include what
supports and structures are needed? Have appropriate data to
describe the programme and its outcomes been collected and are
there biases in the data collection methods?
|
Analysing
data
|
Have
the data
analysis methods been appropriately described and applied?
|
Appraising
the data
|
Are the
evaluation methods appropriate for the question and the context
of the evaluation?
|
Has the relationship between
the purpose of the intervention and the data needed for an
evaluation been presented?
|
Have the
resource and cost implications of implementing the intervention
and cost-effectiveness of the intervention been described?
|
Have other
aspects of study design been described, in terms of context,
sampling, comparison groups, and data collection time-points?
|
Have the values of the
programme, such as empathy been described and included in the
evaluation?
|
Have the OECD criteria of
relevance, coherence, effectiveness, efficiency, sustainability
and impact been fully covered?
|
Interpreting the results
|
Did the
evaluation help answer the question posed at the start? Did the
results cover the interests of the stakeholders?
|
Are the
results relevant and coherent and likely to be sustainable?
|
Are there
any ethical concerns about the evaluation?
|
Can and
should the results of the study influence practice in the
population described, or in different contexts?
|
Synthesising the information
|
How do the
results fit into the body of evidence previously collected?
|
Learning
from the evaluation
|
Implementing
the results
|
Has there
been appropriate reflection on the results of the study and the
context in which the evaluation took place?
|
Have the results been
appropriately communicated by the evaluator to relevant
stakeholders?
|
Is the
evaluation likely
to be relevant
to a similar programme delivered in a different context?
|
Exploring
inequity
|
Has the evaluation considered
the socio-cultural diversity of the population?
|
Has the evaluation considered
whether the programme has had an impact on equity?
|
Reconsidering the purpose of
the evaluation
|
In the
light of the findings, should the purpose of the evaluation have
been defined differently?
|
Cycle
of learning through feedback
|
Adapting
to changes during the evaluation
|
Have any
changes during the evaluation been described and accounted for?
Has the opportunity been taken to adapt to changes in
circumstances over time or feedback during the programme?
|
Have the
authors considered what differences they would make to the
evaluation if repeating it?
|
