Health Research in 4D - Data

We are in an office at the Fraunhofer Institute for Medical Image Computing MEVIS in Bremen. Dr. Jan Klein bends over the computer and clicks the mouse to call a picture to the screen – a CT image of a human head. “The cerebellum is in the back; the teeth are visible up front. The colored area is the parotid gland,” says the computer scientist. These markings help doctors in clinics map out the radiotherapy for head and neck tumors. Physicians want to spare the glands any unnecessary radiation to minimize side effects such as a dry mouth.

The small organ on Jan Klein’s screen is bordered by two lines. A doctor drew the yellow line. The green line was calculated by a learning algorithm based on a neural network. Initially trained with 450 reference data sets, it is now learning how to recognize certain structures in images autonomously – and very successfully at that. “As you can see, the two lines are close together. Our program often outperforms doctors because the daily workload in clinics usually leaves physicians pressed for time when they draw these lines,” says Klein.

Connecting medical data

This automatic segmentation – that is, the ability to recognize and mark a specific organ in an image – is just a small part of a larger project called QuantMed. Fraunhofer researchers are pursuing this initiative to build the technological foundation for an IT platform that will provide valuable services to medicine. Commenting on its goal, Institute Director Prof. Horst Hahn says: “We are working on digital infrastructure that links diverse medical data. Specifically, we are connecting image data from MR scanners and X-ray machines with other clinical information such as blood counts, genetic data and digitized tissue samples.”

Deep-learning algorithms can then comb through these complex, entwined data sets in search of telltale patterns. The results of these analyses support doctors in a number of ways, for example, to help identify cerebral hemorrhages, devise more patient-friendly radiotherapy strategies for tumors, and answer the question of which drug is the most promising option for chemotherapy. A term has been coined for using computers to combine images and clinical data to more accurately predict a therapy’s chances of success: experts call it radiomics.

Automatically detecting cerebral hemorrhages

The new technology builds on individual ‘knowledge nodes’ that can extract valuable information from data sets. “This type of node may be located in a clinic, for example. Our learning algorithms work completely autonomously so that no sensitive patient data is leaked,” says Klein.

Nodes can nonetheless exchange information via a computing hub that validates, consolidates and enhances the knowledge sourced from learning algorithms at multiple clinics. “For example, this could help optimize the way cerebral hemorrhages are automatically detected and classified on CT images. It would lighten the workload; rather than performing this time-consuming task manually, the doctor merely has to check the computer’s results,” explains Klein.

QuantMed kicked off in October 2016. Hospitals, hospital associations, a professional association and businesses are on board. “The first version of the system is ready midway through the project, and now we’re going to ramp up test operations. The end result should be a viable infrastructure that enables hospitals and research institutions to routinely extract new knowledge from vast quantities of data,” says Hahn.

We visited four teams. Each works on an entirely different project. But they do have one thing in common: they collaborate with experts in other disciplines under the umbrella of the Fraunhofer-Gesellschaft. Doctors, scientists, engineers and computer scientists work together across the boundaries of their chosen fields to make that 4D connection in the Fraunhofer Network.