How do you assess inter-rater reliability during CDI audits?

• A. Use a single auditor to review all records.
• B. Calculate time to complete audits and judge consistency.
• C. Use automated scoring only.
• D. Have multiple auditors review the same records and calculate agreement metrics (Cohen's kappa, percent agreement).

Answer: (D)

Inter-rater reliability looks at how consistently different auditors apply the same criteria when reviewing records. To assess this in CDI audits, have multiple auditors independently review the same set of records and compare their judgments using agreement metrics, such as Cohen's kappa or percent agreement. This approach directly measures how much auditors agree beyond chance (kappa) and the raw level of agreement (percent). The other options miss the point: using a single auditor doesn’t reveal agreement between raters; measuring time to complete audits gauges efficiency, not consistency; relying on automated scoring ignores human judgment variability. Calibration with a clear rubric beforehand helps ensure meaningful reliability results.