The best clinical practice requires one to appraise and use the research evidence available while making clinical decisions. The skill to appraising scientific evidence will become easy if one takes a methodical approach.
In this article, I will provide you with steps to follow while appraising a scientific article. It is key to know the structure of a research article before you attempt to appraise the evidence.
Most research articles will include the following sections:
- Introduction / Background (leading to the need for research)
- Methods / Methodology
- Results / Findings
Consider asking the following questions while appraising a research article. This is not an exhaustive list of questions to ask while appraising evidence but can be used direct your assessment.
- Does the title cover the specific theme or the main aim of the research?
- Is the title relevant to the aims of the research?
- Does the title attract the readers while capturing the important theme of the study?
- Does the title include the ‘PICO’ in a randomised controlled study?
- Population / Problem / Patients (with disease in question)
- Intervention / Exposure / Prognostic / Risk factor (under investigation)
- Comparison / Control group intervention
- Are the main features of all sections of the article mentioned briefly and clearly in the abstract?
- Background: Is the ‘need for research’ mentioned?
- Aims: Is the ‘main aim’ of the study mentioned?
- Methods: Study design, randomisation/allocation methods, statistical analyses?
- Results: Participant characteristics, results of the main hypothesis?
- Conclusions: Description/explanation of the main results?
- Discussion: Implication of the findings? How do these findings relate to the existing scientific literature?
- Does the information packed in the abstract correspond to what is in the manuscript?
- Does the introduction/background section clearly state what already is known about this topic?
- Is the need for research clearly outlined?
- Is the main aim of this study clearly stated?
- Does the need for research from the background section and the main hypothesis (aim) of this study relate conceptually?
- Subject Selection Process: If the subjects are selected wrongly, the findings from the study risks having limited relevance and applicability to the population other than one sample.
- Was the subject selection process described clearly?
- Are the inclusion and exclusion criteria listed?
- Was the population from which the subjects were selected defined clearly?
- What is the subjects’ likelihood of truly representing the population that they were selected from?
- What proportion of selected subjects completed the study?
- Are the details of excluded subjects, missed cases, withdrawals, no-shows, lost to follow-up presented?
- Can we compare the study sample to similar studies?
- Can we generalise the study sample to other population?
- Flow Diagram:
- Is there a study flow diagram listing subjects included, excluded, missed, withdrawn, lost to follow-up and who successfully completed the study (in both experimental & control groups).
- Did the study have sources of sampling bias while selecting subjects by following methods?
- Convenience sampling (may favour/disfavour certain subjects)
- Volunteer / Self-selection sampling (subjects may have similar characteristics)
- Stratified sampling (may allow selection bias with unclear strata)
- Random sampling (may result in unbalanced groups)
- Other sampling methods: systematic, clustered, quota, snowball etc.,
- Assessing Validity:
- Did the study use measures/methods with adequate validity?
- Face validity: Did the study use a measure/method that appears appropriate for the construct it is intending to assess?
- Construct validity:
- Did the study use a measure/method that estimates the theoretical construct adequately?
- Does the measure/method used in the study correlate with other measures/methods that estimate the same theoretical construct adequately?
- Content validity: Did the study use a measure/method that fully represents the construct it is intending to assess?
- Did the study use measures/methods with adequate validity?
- Assessing Reliability:
- Did the study use measures (eg., bone mineral density by DEXA) with adequate reliability?
- Did the study use methods (eg., using two radiographers for diagnosis) to increase the reliability of subjective assessments?
- Did the study use methods (eg., taking serial MRIs) to increase the reliability of objective measurements (i.e., to reduce systematic/measurement errors)?
- Does this study require a statistical analysis? (Literature review vs Method study vs Empirical experiment)
- Adequate statistics were done?
- Did the authors report sample size calculation?
- Did they report criteria for allocation into groups?
- Did they use a clear unit of measurement?
- Do the data/results address the research question?
- Presenting data:
- Are tables and figures understandable without reading the text?
- Is there a repetition of content between tables/figures and text?
- Is there adherence to common conventions on units, symbols, format, rounding etc.,?
- Ensuring validity of data:
- Did authors define statistical significance clearly?
- Were the effect size, practically important differences explained?
- Are the data/measurement categories used for analyses clearly defined and are they valid/reliable?
- Does the conclusion section describe/explain/interpret the results succinctly?
- Does the conclusion answer the main research question (aim) of this study?
- Are the main findings re-stated briefly?
- Are the results interpreted and compared with current literature?
- Do the findings contribute to the existing body of knowledge?
- Are the implications of the research mentioned?
- Are the limitations of the study stated?
- Find out if the limitations of the study contain serious errors? eg., wrong study design.
- Are the future research directions suggested?
- Did the authors use relevant articles for reference?
- Did the authors use recent articles for reference?
- Did the authors cite the referred articles correctly?
- Can you find the cited articles in the online databases?
- Did the authors fail to cite seminal articles for their area of research?