Inter-rater reliability of the dental practicality index and the American Association of Endodontists Endodontic case difficulty assessment form

Abstract

The Dental Practicality Index (DPI) and the American Association of Endodontists Endodontic Case Difficulty Assessment (AAECDA) are potentially useful for triaging cases in academic dental institutions. However, the information on the reliability of these methods was limited. This study aims to evaluate the inter-rater reliability of DPI and AAECDA for assessing teeth requiring endodontic treatment. Cases potentially requiring endodontic treatment were selected with purposive sampling to construct 25 case scenarios. All final year dental students (cohort 2021–22) were trained in using DPI and AAECDA. Ten randomly selected students rated all the cases with DPI and AAECDA. The raters were asked to answer multiple-choice questions about the appropriate treatment and treatment provider for each case. The item-by-item inter-rater reliability was estimated with Gwet’s agreement coefficient (AC2). The overall interrater reliability was analysed with a two-way random rater effects ANOVA model to compute the intraclass correlation coefficient (ICC). The association of clinical decision and the assessment categorisation was analysed with generalised estimating equations. The inter-rater reliability of DPI was generally very good (AC2 = 0.81–1.00), except ‘Context’ which was good (AC2 = 0.718, 95% CI = 0.575–0.861). The inter-rater reliability of AAECDA was generally very good (AC2 = 0.81–1.00) and good (AC2 = 0.61–0.80), except ‘Radiographic appearance of canal(s)' which was fair (AC2 = 0.424, 95% CI = 0.263–0.585). Moderate overall inter-rater reliability of AAECDA (ICC = 0.53, 95% CI = 0.38–0.70) and DPI (ICC = 0.618, 95% CI = 0.48–0.77) was observed. Referral to an endodontist was positively associated with AAECDA score (OR = 1.323, 95% CI = 1.145–1.529, p < 0.001). Decision of tooth extraction was positively associated with DPI score (OR = 1.983, 95% CI = 1.539–2.555, p < 0.001). DPI and AAECDA are methods with moderate inter-rater reliability in cases potentially requiring endodontic treatment. DPI and AAECDA can be used in academic dental settings to guide inexperienced clinicians in making treatment and referral decisions. However, the inter-rater reliability of several items in both methods can be improved by having clearly defined and mutually exclusive selections