Speech Recognition Technology Introduces Errors in Physician Notes

Manual review of notes transcribed by speech recognition (SR) technology must be manually reviewed for errors, according to a study published online in JAMA Network Open.

Researchers from Massachusetts and Colorado, performed a cross-sectional study to identify and analyze error rates that appeared in SR-assisted documentation of office notes, discharge summaries, and operative notes.

The researchers collected a stratified random sample of 83 office notes, 75 discharge summaries, and 59 operative notes dictated by 144 physicians using Dragon Medical 360 | eScription (Nuance). Errors were annotated in the SR engine–generated document (SR), the medical transcriptionist–edited document (MT), and the physician’s signed note (SN). Each document was compared with a criterion standard created from the original audio recordings and medical record review.

Main outcomes and measures include:

• Error rate
• Mean errors per document
• Error frequency by general type (such as deletion), semantic type (such as medication), and clinical significance;
• Variations by physician characteristics, note type, and institution

The results showed that among the 217 notes, there were 144 unique dictating physicians: 44 females (30.6 percent) and 10 unknown sex (6.9 percent). Mean (SD) physician age was 52 (12.5) years.

The specialty information was available for 121 physicians (84 percent); 35 specialties were represented:

• 45 surgeons (37.2 percent)
• 30 internists (24.8 percent)
• 46 others (38 percent)

The error rate in SR notes was 7.4 errors per 100 words (7.4 percent), decreasing to 0.4 percent after transcriptionist review and 0.3 percent in SNs. Overall, 96.3 percent of SR notes, 58.1 percent of MT notes, and 42.4 percent of SNs contained errors. The most common errors were deletions (34.7 percent), followed by insertions (27 percent).

Discharge summaries had higher mean SR error rates than other types (8.9 percent versus 6.6 percent).

The surgeon SR notes had lower mean error rates than other physicians (6.0 percent versus 8.1 percent). One of the two institutions that participated in the study had a higher mean SR error rate (7.6 percent versus 6.6) but lower mean MT and SN error rates (0.3 percent versus 0.7 and 0.2 percent versus 0.6).

The authors concluded that with 7 in 100 words in SR-generated documents containing errors; many involving clinical information, manual review is needed to correct the errors.