Critical appraisal
Topic outline
-
'Critical Appraisal is the process of carefully and systematically examining research to judge its trustworthiness, and its value and relevance in a particular context.' This quote comes from the Critical Appraisal Skills Programme (CASP). As they go on to say, 'Critical appraisal skills are important as they enable you to assess systematically the trustworthiness, relevance and results of published papers. Where an article is published, or who wrote it should not be an indication of its trustworthiness and relevance.'
This does not mean criticising what is published, but assessing the value of what is published.
There are a number of tools available for the critical appraisal of published papers, and we will demonstrate them as they are applied to various aspects of clinical and public health practice as well as in the broad field of evaluation.
The art of critical appraisal can also be very valuable as a teaching method to identify the strengths, weaknesses and applicability of various research designs.
We suggest that you explore the various tools shown in the sections below. These tools are all slightly different as they can be applied in different settings. In many ways it does not matter which tool you use - the important thing is to take a structured approach to critical appraisal, and use whichever tool you feel is relevant.
How to navigate the course
Each section comprises a set of resources that we think you will find interesting - click on the collections of resources in each section. There is a forum in each topic for reflection, and there is a quiz at the end to check your understanding of the concepts.
We encourage you to reflect on what you have learned or comment on the course. When you click on the hyperlink in each topic labelled reflection, you will be able to add a new topic or respond to a previous one. You may want to share your learning from this and other readings, comment on the topics from your own experience, comment on others' posts, or provide feedback on how we can improve the content and presentation.
In the final section you will see that you can gain a Certificate of Completion - the requirements for this are to access the resources and post a reflection in each section.
This work is licensed under a Creative Commons Attribution 4.0 International License. -
Although there are many different types of research and ways of publishing the results, there is a common theme in many types of research performed in Public Health. For each, there is a need to explore the 'anatomy' of the research - the population being studied, the exposure to a causative factor or an intervention, and the outcome being studied. We need to assess the features of each of these.
The GATE framework is a very useful way of depicting this (You can read more detail of the GATE framework here).
This picture shows the anatomy of a research paper, and thus what to look out for in a critical appraisal of the paper. Each aspect needs to be explored:
- the population - how it was selected;
- the intervention or possible exposure or causative factor - how they were chosen and measured and if this was biased;
- the comparison group - was there one and if so was it appropriate
- the outcome - was this appropriate, how was it measured and if this was biased
- the study time frame
There is much more to consider in appraising a publication, but it all hangs on the basic anatomy of the research performed. The sections below each provide much more detail on how to appraise a published paper, but they will each amplify and extend the basic steps in the GATE framework
-
A very good way to start to work on critical appraisal is to go through the various checklists on the CASP website. You will see the following checklists:
-
View Make forum posts: 1
Reflect on which of the CASP tools you feel would be most relevant to you, and why
-
The Newcastle Critical Appraisal Worksheet was developed as a simple 11 question checklist that could be applied to most types of study. As you see from the CASP checklists in the previous section, many people prefer to use a checklist that relates to a particular type of study. Have a look at this one, and see what you think!
-
The attached worksheet is in the form of a presentation, below you will see it in the form of a checklist to fill in.
-
-
Critical appraisal tools were initially devised for the practice of Evidence Based Medicine, so most of the checklists have a clinical science basis. Here is one that can be applied to Public Health.
-
This checklist is presented as a worksheet so that you can download and complete it if you wish. It is taken from the published paper: Critical appraisal for public health: A new checklist.
-
This file applies the public health checklist to the paper Impact on carbon emissions of online study for a cohort of overseas students: A retrospective cohort study and offers suggested answers
-
View Make forum posts: 1
Reflect on whether you agree on the answers we have offered in applying the checklist to the paper on carbon emissions
-
-
In order to explore this checklist which has been developed to be applied to the evaluation of interventions, it is worth noting that it has been derived from a framework for evidence based evaluations.
The figure shows this framework for performing and appraising evaluations. Three practical steps are shown as ellipses in the foreground, and the constructs in the background are the underpinnings of these steps.
The critical appraisal tool below is based on this framework
-
Here we show how we applied the worksheet to a published evaluation of an online public health capacity building programme
-
View Make forum posts: 1
Reflect on whether you agree with the appraisal of the evaluation paper which we have offered in the worked example
-