An experimental study of determinants of group judgments in clinical guideline development.
Raine, Rosalind, Sanderson, Colin, Hutchings, Andrew et al. · Lancet (London, England) · 2004 · DOI
Quick Summary
This study examined how doctors develop medical guidelines by testing whether expert opinions match what research evidence actually shows. Researchers asked different groups of doctors to rate whether certain treatments (like therapy or antidepressants) were appropriate for conditions including chronic fatigue syndrome. They found that doctors' judgments agreed with published research only about half the time, and that having a literature review and certain group compositions helped improve accuracy.
Why It Matters
For ME/CFS patients, this study is crucial because it reveals that clinical guidelines—which shape treatment recommendations and access to care—may not consistently reflect actual research evidence. Understanding how expert consensus is formed helps patients and advocates recognize when guidelines might need updating based on new evidence, and highlights the importance of ensuring that robust research data influences clinical practice recommendations.
Observed Findings
Only 51% of 192 consensus statements agreed with published research evidence.
GP-only groups showed higher agreement with research evidence compared to mixed GP-specialist groups.
Provision of a literature review significantly increased likelihood of agreement with research evidence.
Clinicians' prior beliefs and clinical experience modified judgments irrespective of actual research data.
Resource availability assumptions had no impact on appropriateness ratings.
Inferred Conclusions
Guidelines cannot be based on empirical data alone; expert judgment inevitably shapes outcomes.
The nominal group technique effectively elicits and structures expert judgment but does not guarantee alignment with research evidence.
When consensus statements diverge from published literature, the reasons for disagreement should be made explicit in guidelines.
Clinician beliefs and experience are powerful but sometimes conflicting influences on guideline development.
Remaining Questions
Why do GP-only groups show better concordance with evidence than mixed groups, and does this reflect different training or different access to evidence?
What This Study Does Not Prove
This study does not demonstrate which treatments are actually effective for ME/CFS, nor does it evaluate treatment outcomes. It also does not prove that guideline development *should* be based entirely on evidence alone, only that current consensus methods often diverge from published literature. The findings reflect the guideline development process itself, not the validity of the treatments being rated.
About the PEM badge: “PEM required” means post-exertional malaise was an explicit required diagnostic criterion for participant inclusion in this study — not that PEM was studied, observed, or discussed. Studies using criteria that do not require PEM (e.g. Fukuda, Oxford) are tagged “PEM not required”. How the atlas works →
Contribute
Private, reviewed by a human. Not a public comment thread.