Assessing Adherence and Competence in Delivering Telehealth Group Cognitive Behavioral Stress Management

Treatment fidelity, or consistent therapist adherence and competence in intervention delivery, is an important component of intervention design and dissemination. Yet few systems have been developed to assess fidelity in telepsychology and related remotely delivered intervention venues. We sought to modify and evaluate a fidelity coding approach for a videoconference-delivered group cognitive behavioral stress management (CBSM) intervention. Although validated rating systems for treatment fidelity exist, they typically focus on individual therapy and are inappropriate for the structure of group stress management interventions. We modified the RAND Corporation Fidelity Coding Guide, which was initially designed for audio-recorded group cognitive behavioral therapy, for applicability to videotaped sessions of CBSM. Two raters applied the resulting CBSM Coding Guide to 146 session recordings taken from a CBSM trial for persons with chronic fatigue syndrome and their partners, with 40% of sessions double-coded. Reliability for eight adherence items and 15 competence items was moderate overall (average κ_w =.751) with 20 of the 23 items demonstrating at least moderate interrater reliability (i.e., κ_w >.600). Three items that did not meet this cutoff would benefit from more extensive operationalization and rater training. Our adapted system provides a means of assessing adherence and competence variables to validate the delivery of CBSM and of associating these with positive treatment outcomes to isolate the “active ingredients” of CBSM. Further, the successful modification and application of the RAND Corporation Fidelity Coding Guide support the adaptation of this system for a diverse range of remotely delivered cognitive behavioral group therapies.