Publications

When deploying ASR in high-stakes settings, character error rate (CER) alone often cannot indicate whether to exclude spurious output on non-speech input, request a repeat utterance, or manually verify recognition content. We propose a layered evaluation that splits errors into text on non-speech input, empty output on speech, and incorrect content on speech, each mapped to one of these follow-up responses. We evaluate on advance care planning dialogue by comparing a cloud and an edge deployment pipeline: when CER is similar on clinic-style sentences, short answers diverge toward content errors versus empty outputs; with other settings held fixed, changing only the language setting also changes medical-term rewrites of negated short answers and lexical coverage. Thus similar CER need not imply the same follow-up handling.

@inproceedings{chou2026drpf,
  title     = {Deployment Risk Probe Framework ({DRPF}): Complementing {CER} with Actionable Failure Modes},
  author    = {Chou, Ming-Kun and Lo, Yu-Tai and Lu, Wen-Hsiang},
  booktitle = {Proceedings of the 38th Conference on Computational Linguistics and Speech Processing (ROCLING 2026)},
  year      = {2026}
}