Medical imaging is an important component of today’s healthcare. It improves the precision, consistency, and development of treatments for a variety of disorders. Artificial intelligence has also been employed extensively to improve the procedure.
Traditional medical image diagnosis using AI algorithms, on the other hand, necessitates a huge number of annotations as supervision signals for model training. To obtain accurate labels for the AI algorithms, radiologists prepare radiology reports for each of their patients as part of their clinical routine, with annotation staff extracting and verifying structured labels from those reports using human-defined rules and existing natural language processing (NLP) tools. The quality of human labor and various NLP techniques determine the ultimate accuracy of derived labels. The procedure is labor-intensive and time-consuming, hence it comes at a high cost.
By enabling the automatic acquisition of supervision signals from hundreds of thousands of radiology reports at the same time, an engineering team at the University of Hong Kong (HKU) has developed a new approach called “REFERS” (Reviewing Free-text Reports for Supervision), which can reduce human costs by 90%. It achieves a high level of prediction accuracy, outperforming its AI-assisted counterpart in conventional medical picture diagnosis.
The novel technique represents a significant step forward in the development of generalized medical artificial intelligence. The breakthrough was described in the publication “Generalized radiograph representation learning via cross-supervision between pictures and free-text radiography reports” published in Nature Machine Intelligence.
Professor YU Yizhou, the team’s leader from the Faculty of Engineering’s Department of Computer Science, stated that AI-enabled medical image diagnosis has the potential to help medical specialists reduce their workload and improve diagnostic efficiency and accuracy, including but not limited to reducing diagnosis time and detecting subtle disease patterns.
The researchers feel that radiology reports’ abstract and complicated logical reasoning sentences provide enough information for acquiring easily transferable visual elements. Professor Yu noted that with the right training, REFERS can learn radiological representations directly from free-text reports without the requirement for human labeling.
The research team uses a public database to train REFERS, which contains 370,000 X-Ray pictures and related radiology reports on 14 common chest disorders such as atelectasis, cardiomegaly, pleural effusion, pneumonia, and pneumothorax.
The researchers were able to create a radiograph recognition model with just 100 radiographs and predict with an accuracy of 83%. When the quantity is increased to 1,000, their model performs exceptionally well, with an accuracy of 88.2 percent, outperforming a counterpart trained with 10,000 radiologist comments (accuracy at 87.6 percent ). The accuracy is 90.1 percent when 10,000 radiographs are employed. In general, prediction accuracy of more than 85% is useful in real-world clinical applications.
REFERS accomplishes this purpose by performing two report-related tasks: report production and radiograph–report matching. REFERS converts radiographs into text reports in the first task by encoding radiographs into an intermediate representation, which is subsequently utilized to predict text reports using a decoder network.
To train the neural network and update its weights, a cost function is constructed to measure the similarity between predicted and real report texts. This cost function is used to train the neural network and update its weights.
In the second task, REFERS initially encodes both radiographs and free-text reports into the same semantic space, where contrastive learning aligns the representations of each report and its related radiographs.
Dr. ZHOU Hong-Yu, the paper’s first author, claimed that, unlike other approaches that rely heavily on human annotations, REFERS can obtain supervision from each word in radiology reports. The amount of data annotation can be decreased by 90%, and the cost of developing medical artificial intelligence can be cut as well. This is a major step forward in the development of generalized medical artificial intelligence.
