The problem bigger than speech recognition in medical records

January 25, 2020
Harjinder Sandhu

Throughout my career, one question I kept asking myself was why speech recognition and machine learning weren’t being better applied to medical dictation. The answer was more complex than I initially thought, especially as speech recognition gained traction. Speech recognition creates efficiencies for medical offices that were spending millions of dollars on high turnover scribes and human transcribers. Many healthcare organizations adopted this productivity enhancement tool, saving them time and money. 

With the passage of time, though, I realized what the problem was. The problem was actually twofold: getting doctors away from the time-intensive dictation or typing process, and determining a way for doctors to benefit from this digitized medical data. 

EMRs are helpful in creating legible and standardized notes that can be viewed from a variety of settings. However this is not why they were created. EMRs were created to make billing easier. The way EMR notes are structured, the data can’t easily be used for clinical decision support during the visit, nor for searching after. Narrative data is difficult to parse, even with machine learning. When I was the Chief Technologist at Nuance, I focused on turning these notes into discrete data for the EMR. When I left in early 2011, I felt there was still an unfinished promise of what the technology could do for doctors and their medical records, and it became my driving force. In continuing to explore this issue, the biggest challenge was data creation in medical records, and making that data useful. That meant capturing medical information from a clinical encounter and putting it in a discrete form to be used as decision support. 

Given all the individual record fields, dictation – even with speech recognition – is an exercise in frustration. With traditional dictation, doctors must learn how to use the system. Those not dictating notes awkwardly type them into the computer, with their backs to the patients or looking back and forth between the computer and the patient. They attempt to have a natural conversation and relationship, hunting and pecking to enter information into the right computer fields. EMRs make doctors the world’s most expensive data entry clerks. It’s no surprise that physicians feel burned out by the data entry or transcriptions piling up.

Until recently, no one has been successful in adequately capturing the doctor/patient conversation using speech recognition. What if there were a platform that not only did that, but didn’t require the physician to type or dictate anything? The information gleaned from the doctor/patient encounter would be captured in a discrete fashion, in the right medical record fields. This would create the framework and groundwork to allow decision science applications to leverage the data and provide more value than current EMRs provide.

With Saykara, that is now a reality. We created machine learning-based tools using speech recognition, that turns narrative data into discrete data. And it’s now available to healthcare organizations across the country. Speech recognition is the foundation, and machine learning then turns the doctor/patient conversation into usable data. Using an iPhone app, the Saykara platform allows doctors to have a natural face-to-face conversation with the patient, while Saykara streams the conversation. In lieu of traditional dictation, the app uses speech recognition to transcribe the medical visit conversation into the proper fields. A human reviewer ensures its accuracy.

In medicine, speech recognition evolved more rapidly than in other industries, those using the technology sporadically. Speech recognition drives high value for physicians, and machine learning is poised to do the same in interpreting conversations for use in the medical records. Once it gains a foothold in medicine, it will evolve quickly, as the value proposition is so high.

To get to a higher level of automation and accuracy, machine learning relies on a great deal of data. The only way to do this is to create a hybrid system, leveraging machine learning as it exists today, augmented by humans on the back end, to ensure what’s reported in the medical records is accurate. This allows the system to learn over time. The vision is arriving at the point of offering a completely autonomous system. We are getting closer.

Fine tuning this technology will take some time, as interpreting conversations is much harder than just speech recognition. That’s why we created our own speech recognition program. There was no foundation to work from. It didn’t exist. We created a specific way of approaching and training the data, to address the way we think about the problem. This is what makes our machine learning application novel.

We know we’re on to something because of our high adoption rate. Physicians who try Saykara recognize that it’s better than anything they’ve ever used. They’re comforted by the quality of the records returned for their signature. Because of the data augmentation process, it’s not up to the physician to learn how to say things to the system, but rather how the system learns from the physician. Physicians spend very little time correcting the records.

Saykara began commercializing the services in summer, 2018 with a mix of large health systems rolling it out, and smaller specialty groups. If you’re interested in finding out more about how Saykara is different, and how we can help your organization, contact us for more information and a trial. BEST CONTACT INFO HERE.


The problem bigger than speech recognition in medical records