Training AI Systems to Ask Clinical Questions

Training AI Systems to Ask Clinical Questions 

Physicians often query a patient’s electronic health record for information that helps them make treatment decisions, but the cumbersome nature of these records hampers the process. Research has shown that even when a doctor has been trained to use an electronic health record (EHR), finding an answer to just one question can take, on average, more than eight minutes. This is where AI can be of great value in obtaining the needed information in the database thereby saving health professionals massive amounts of time. Today we will take a look at such AI systems and how they are trained. 

How Can AI Be Trained to Find Patient Information? 

Researchers have begun developing machine-learning models that can streamline the process by automatically finding information physicians need in an EHR. However, training effective models requires huge datasets of relevant medical questions, which are often hard to come by due to privacy restrictions. Existing models struggle to generate authentic questions — those that would be asked by a human doctor — and are often unable to successfully find correct answers.

To overcome this data shortage, researchers at MIT partnered with medical experts to study the questions physicians ask when reviewing EHRs. Then, they built a publicly available dataset of more than 2,000 clinically relevant questions written by these medical experts.When they used their dataset to train a machine-learning model to generate clinical questions, they found that the model asked high-quality and authentic questions, as compared to real questions from medical experts, more than 60 percent of the time.

With this dataset, they plan to generate vast numbers of authentic medical questions and then use those questions to train a machine-learning model which would help doctors find sought-after information in a patient’s record more efficiently.

Shortage of Quality Data

The few large datasets of clinical questions the researchers were able to find had a host of issues. Some were composed of medical questions asked by patients on web forums, which are a far cry from physician questions. Other datasets contained questions produced from templates, so they are mostly identical in structure, making many questions unrealistic. 

To build their dataset, the MIT researchers worked with practicing physicians and medical students in their last year of training. They gave these medical experts more than 100 EHR discharge summaries and told them to read through a summary and ask any questions they might have. The researchers didn’t put any restrictions on question types or structures in an effort to gather natural questions. They also asked the medical experts to identify the “trigger text” in the EHR that led them to ask each question. Once they had compiled their dataset of questions and accompanying trigger text, they used it to train machine-learning models to ask new questions based on the trigger text.

What are the Practical Applications? 

While this is a pretty advanced AI system, don’t expect it to replace human doctors any time soon. In fact, the researchers found that when a model was given trigger text, it was able to generate a good question 63 percent of the time, whereas a human physician would ask a good question 80 percent of the time. Basically, a lot of work still needs to be done. The best way this AI model can be used is to help build and train a better AI system that can automatically answer physicians’ questions in an EHR. For the next step, they will use their dataset to train a machine-learning model that can automatically generate thousands or millions of good clinical questions, which can then be used to train a new model for automatic question answering.

Clear Need for Better Data 

As we can see from the created model, having quality training data is an important part of any AI project. For an AI system to learn what it needs to learn, While there is much work to be done, we can certainly view this AI model as a step in the right direction and to hopefully providing health professionals with the assistance they need to eliminate wasted time searching for information.