Automated Question Type Coding of Forensic Interviews and Trial Testimony in Child Sexual Abuse Cases

Date

2024

Journal Title

Journal ISSN

Volume Title

Publisher

Law & Human Behavior

Abstract

Question type classification is widely used as a measure of interview quality. However, question type coding is a time-consuming process when performed by manual coders. Reliable automated question type coding approaches would facilitate the assessment of the quality of forensic interviews and court testimony involving victims of child abuse. We examined whether a large language model (RoBERTa) trained on questions (N = 351,920) asked in forensic interviews (n = 1,435) and trial testimony (n = 416) involving 3- to 17-year-old alleged victims of child sexual abuse could distinguish among 1) invitations, 2) wh- questions, 3) option-posing questions and 4) non-questions. The model achieved high reliability (95% agreement; K = .93). In order to determine if disagreements were due to machine or manual errors, we re-coded inconsistencies between the machine and manual codes. Manual coders erred more often than the machine, particularly by overlooking invitations and non-questions. Correcting errors in the manual codes further increased the model’s reliability (98% agreement, K = .97). Automated question type coding can provide a time-efficient and highly accurate alternative to manual coding. We have made the trained model publicly available for use by researchers and practitioners.

Description

Keywords

child forensic interview, question types, interview quality, question type coding, automation

Citation

Szojka, Z.A., Yashraj, S., & Lyon, T.D. (in press). Automated question type coding of forensic interviews and trial testimony in child sexual abuse cases. Law & Human Behavior.

DOI