E3 PreliminaryModerate confidencePEM not requiredMethods-PaperPeer-reviewedReviewed
Standard · 3 min
BRIEF REPORT: Assessing Adherence and Competence in Delivering Telehealth Group Cognitive Behavioral Stress Management.
May, Marcella, Ream, Molly, Milrad, Sara F et al. · International journal of stress management · 2024 · DOI
Quick Summary
This study developed a way to check whether therapists are properly delivering a stress management program called CBSM (Cognitive Behavioral Stress Management) when it's delivered online through video. The researchers watched 146 recorded therapy sessions with ME/CFS patients and their partners to see if therapists were following the program correctly and doing it well. They found that their checklist system worked fairly well, meaning different raters could agree on whether therapists were doing their job properly.
Why It Matters
Treatment fidelity assessment is essential for ensuring that CBSM interventions for ME/CFS are delivered consistently and effectively across different settings. This study establishes a reliable tool for monitoring whether therapists are implementing the intervention as designed, which helps researchers identify which specific components of CBSM actually drive improvements in patient outcomes. Having validated quality-control methods strengthens the evidence base for telehealth interventions, making it easier to scale effective treatments for ME/CFS.
Observed Findings
Twenty of 23 fidelity items (adherence and competence combined) achieved moderate or better interrater reliability (κw > .600).
Overall average interrater reliability across all 23 items was moderate (κw = .751).
Three items (approximately 13% of the coding system) fell below the κw > .600 threshold and would benefit from clearer definition and additional rater training.
The RAND Corporation Fidelity Coding Guide was successfully modified from audiorecorded individual therapy format to videoconference-delivered group CBSM.
Inferred Conclusions
A reliable fidelity assessment tool can be developed for remotely-delivered group cognitive behavioral interventions by carefully adapting existing frameworks and providing standardized rater training.
The adapted CBSM Coding Guide can now be used to validate consistent intervention delivery and to link specific therapist adherence and competence variables to patient treatment outcomes.
The RAND Corporation approach is adaptable across different remotely-delivered cognitive behavioral group therapies, supporting broader dissemination of evidence-based interventions.
Remaining Questions
Which of the 23 fidelity items (adherence or competence factors) most strongly predict positive treatment outcomes in ME/CFS patients?
How does therapist adherence and competence in remote CBSM delivery compare to in-person delivery?
What This Study Does Not Prove
This study does not demonstrate that CBSM itself is effective for ME/CFS—it only measures whether the program is being delivered correctly. The study also does not establish which specific therapeutic components actually improve patient outcomes, nor does it compare the efficacy of videoconference-delivered CBSM to in-person delivery or to other treatments. Fidelity measurement alone does not prove causation between adherence/competence and patient benefit.
About the PEM badge: “PEM required” means post-exertional malaise was an explicit required diagnostic criterion for participant inclusion in this study — not that PEM was studied, observed, or discussed. Studies using criteria that do not require PEM (e.g. Fukuda, Oxford) are tagged “PEM not required”. How the atlas works →
Contribute
Private, reviewed by a human. Not a public comment thread.
Can the three items with lower reliability be successfully operationalized with additional training, or should they be modified or removed from the coding system?
Does higher overall fidelity in CBSM delivery correlate with better clinical outcomes for ME/CFS patients and their partners?