Intra-rater and inter-rater reliability of the Empathic Communication Coding System

Br J Occup Ther. 2025 Mar 24;88(7):420-428. doi: 10.1177/03080226251314367. eCollection 2025 Jul.

Abstract

Objects: The Empathic Communication Coding System (ECCS) is a promising measure of such behaviors, yet its reliability remains largely unestablished. The aim of the study was to examine the intra-rater and inter-rater reliability of the ECCS.

Methods: Transcripts of occupational therapy audio recordings were used to examine the reliabilities. Two occupational therapists were trained to evaluate the data for inter-rater reliability, and one of them evaluated the data twice for intra-rater reliability. Transcripts of 20 patients treated by 18 therapists were evaluated for intra-rater reliability. Transcripts of 25 patients treated by 22 occupational therapy interns were evaluated for inter-rater reliability. Adult rehabilitated patients with basic verbal communication ability were recruited. The percentage of agreement and Cohen's κ were calculated as the indices of reliability.

Results: The results showed acceptable intra-rater reliability but poor inter-rater reliability of the ECCS.

Conclusions: The ECCS demonstrates potential as a reliable tool for assessing empathic behaviors over time when evaluated by the same raters.

Keywords: Assessment; Empathic Communication Coding System; empathy; occupational therapy; reliability.