Mathematics and Computer Science — Research

Machine Learning and Image Analysis

Machine Learning is the science of extracting knowledge from data and experience, and Image Analysis is one of its main applications. Deep learning – machine learning with large artificial neural networks – has recently enjoyed big successes in solving challenging real-world tasks such as natural language translation, automated game playing, and medical image analysis. Increasing the scope of deep learning and ensuring its trustworthiness are among the hottest topics in computer science and applied mathematics. To this end, current research asks how to automatically define good network architectures for a given problem, how to train well even when data is scarce, how to assess the reliability of the results, how to guarantee prediction performance mathematically, and how to explain the decisions of deep networks to humans.

Heidelberg University is among the leading institutions for machine learning and image analysis in Germany. Our research is characterized by its close relation to the natural and life sciences. We are also proud of our close connections to industry, which include the long-standing industry-on-campus project HCI and several AI-related start-ups originating from our groups. Many of our fundamental investigations are inspired by questions arising from our collaborators' applications, for example to establish new imaging modalities and to enable new analysis types in biology and medicine. Another focus lies on the design of reliable and interpretable machine learning methods that will eventually be equipped with empirical and mathematical performance guarantees. These efforts shall lay the foundations for machine learning to be trusted in safety-critical scenarios and to become a novel paradigm of scientific inquiry itself.

Image & Pattern Analysis

Computer Assisted Medical Interventions

Medical Image Computing

Computer Vision and Learning Lab

Biomedical Computer Vision

Statistical Natural Language Processing

Machine Learning for Image Analysis

Image data abound in many fields, from autonomous driving to medical diagnosis. For the last decade, progress in image analysis has been defined by deep neural networks, which continuously raise the state-of-the-art in image classification, object segmentation and tracking. The figure illustrates a major success of this work: the nnU-Net, a segmentation architecture that can configure itself for a wide variety of tasks and was thus able to win dozens of official benchmark challenges without any manual finetuning for different problem settings. Other research projects address advanced diagnostic modalities for medicine, e.g. photo-acoustic imaging and diffusion-weighted magnetic resonance tomography, and the automated analysis of microscopic data, e.g. the classification and tracking of dividing cells in microscopic videos, but also the interpretation of complex real-world scenes in the context of computer vision. A central theme in all these efforts is interdisciplinary research: Many of our scientific questions are inspired by unsolved problems in medicine, biology, neuro science, physics, psychology and so forth, and our solutions are stress-tested in challenging applications from these fields.

Trustworthy Machine Learning

Despite all its successes, machine learning cannot yet be applied to safety-critical scenarios like autonomous driving and automated medical diagnosis, because there are no guarantees that deep networks work properly in all circumstances. The figure illustrates a common challenge, here resolved by one of our invertible neural networks: Many problems – for example, the colorization of a grayscale image – do not have a unique answer. Trustworthy networks must be able to recover the full diversity of possible solutions and provide reliable assessments of their plausibility, an ability known as uncertainty quantification. Another road we are pursuing is to place network architectures and training procedures on firm mathematical ground by investigating their relations to Bayesian statistics, information theory, geometry, and large-scale optimization. Moreover, we design new protocols for the experimental validation of machine learning results, new methods for the economic creation of ground-truth data for validation reference, and user-friendly software libraries for the easy reproduction and application of our findings in practice.

Research Groups and Principal Investigators

We welcome students that want to dig deeper into machine learning and its numerous applications. Our master and PhD programs teach you essential and advanced knowledge and skills to solve machine learning and image analysis problems on your own and to pursue successful carriers in both academia and industry. The following short profiles lead you to the different research groups and give more detailed information about our activities.

Prof. Dr. Christoph SchnörrImage & Pattern Analysis Group

Mathematical Image Analysis, Dynamical Systems for Machine Learning and Data Analysis, Continuous and Discrete Optimization

Both teaching and research focuses on mathematical foundations of image analysis and machine learning combining concepts from statistics, information and differential geometry, and numerical optimisation. Prof. Schnörr is PI in the DFG-funded priority programme SPP 2298 on the Theoretical Foundations of Deep Networks and member of the steering board of the Heidelberg cluster of excellence STRUCTURES.

Prof. Dr. Lena Maier-HeinDKFZ, Div. Computer Assisted Medical Interventions

New Imaging Modalities, Precision Medicine, Method Validation

The mission of the Div. Computer Assisted Medical Interventions is to improve the quality of interventional healthcare in a data-driven manner. To this end, our multidisciplinary group builds upon principles and knowledge from a diversity of research fields including artificial intelligence (AI), statistics, computer vision, biophotonics and medicine. Committed to the ultimate goal of creating benefit for patients and medical staff, we aim to develop a holistic concept spanning the three significant topics perception, data interpretation and real-time assistance (see Fig. 1), and connecting them through a cycle of continuous learning: Novel spectral imaging techniques enabled by deep learning are being developed as safe, reliable and real-time imaging modalities during interventions. When interpreting the perceived data in the context of available knowledge, our division specifically addresses common roadblocks to clinical translation such as data sparsity, explainability and uncertainty handling. In close collaboration with clinical partners, these methods are leveraged for the development of context-aware interventional assistance systems. Finally, we place a strong focus on the reliable validation of AI algorithms for clinical purposes.

Prof. Dr. Klaus Maier-HeinDKFZ, Div. Medical Image Computing

Deep Learning and Biomedical Image Analysis, Precision Medicine, Semantic Segmentation, Object Detection, Unsupervised Learning, Probabilistic Modeling

The Division of Medical Image Computing (MIC) pioneers research in machine learning and information processing, with the particular aim of improving cancer patient care by systematic image data analytics. We structure and quantify imaging information from multiple time-points and imaging technologies, e.g. magnetic resonance imaging or computer tomography, and link it with clinical and biological parameters. As an initiator and co-coordinator of the Helmholtz Imaging Platform (HIP) we pursue cutting-edge developments at the core of computer science, with applications in but also beyond medicine. We are particularly interested in techniques for semantic segmentation and object detection as well as in unsupervised learning and probabilistic modeling. Methodologic excellence can only be achieved on the basis of a sophisticated research software system and infrastructure, for example to facilitate highly scalable data analysis in a federated setting. Our technological portfolio in this regard builds the foundation of various national and international clinical research networks, such as the National Center for Tumor Diseases (NCT), the German Cancer Consortium (DKTK) and the Cancer Core Europe (CCE). In collaboration with our clinical partners, we work on the direct translation of the latest machine learning advances into relevant clinical applications. Our vision is to advance the quality of healthcare through methodological advances in artificial intelligence research and their large-scale clinical implementation. We therefore have a particular interest in techniques that improve the applicability of data science in clinical settings, e.g. by providing more interpretable decision-making, by explicitly dealing with data uncertainty, by increasing the generalizability of algorithms or by learning more powerful representations. We further study image computing concepts that combine mathematical modelling approaches with current machine learning techniques. We are dedicated to open science and committed to maintaining several open source projects in order to share our advances with developers and the scientific community and to promote leveraging synergies.

Prof. Dr. Carsten Rother3D Computer Vision, Head of Computer Vision and Learning Lab

Computer Vision, 3D Reconstruction, Image Synthesis, Video Conferencing

Carsten Rother is head of the “Computer Vision and Learning Lab” and runs the “3D Computer Vision” group within the Lab. Computer Vision aims at extracting high-level information from images and videos, such reconstructing a 3-dimensional scene, detecting and tracking objects in video, or synthesis new images. In my group we focus on tasks that reasons about the 3-dimensional nature of the world that was captured. Examples are tracking of a dynamic face in 3D or inserting a 3D object into existing footage to generate additional training data for deep learning. To solve these tasks, we develop new Machine Learning methods, where oftentimes the best strategy is to bake in geometric knowledge and algorithms into new trainable neural networks. Besides the methodological aspects, I am also interested in developing solutions that are beneficial to society. One example is a novel solution for multi-party video conferencing where we collaborate with researchers from other disciplines, such Human-Computer Interaction. Towards this end, my group has a history of creating start-ups based on our research.

apl. Prof. Dr. Ullrich KötheExplainable Machine Learning @ Computer Vision and Learning Lab

Invertible Neural Networks, Bayesian Inverse Problems, Explainable and Trustworthy Machine Learning, Feature Discovery

Ullrich Köthe heads the “Explainable Machine Learning” group in the Computer Vision and Learning Lab. The group's focuses on deep learning methodology that strives to establish machine learning as a new way to gain insight and create new knowledge in the sciences. Models that merely make somewhat accurate predictions in a blackbox manner are not good enough for this goal. He and his group thus investigate the design and theoretical foundations of novel network architectures and training algorithms, which provide reliable self-assessment of their uncertainty, emerge humanly interpretable latent representations and are robust against distribution shifts in the data. In particular, they utilize invertible neural networks as an especially promising approach to the efficient and well-founded probabilistic treatment of inverse problems and generative modeling. They demonstrated the utility of these solutions in numerous applications from physics and astronomy to computer vision, biology, and medicine. To achieve its goals, the group puts high emphasis on interdisciplinary work with other fields of science, and on the rapid dissemination of its results via reusable open-source software libraries.

Dr. Bogdan SavchynskyyOptimization for Machine Learning @ Computer Vision and Learning Lab

Large-scale discrete optimization, graphical models, assignment and tracking, learning in combinatorial problems

We focus on large-scale combinatorial optimization problems with applications in computer vision and bio-informatics. "Combinatorial" means that the solution must be selected from a finite but exponentially growing set, as for example the set of all possible subsets of a given set, permutations or paths in a given graph. Most of such problems can be formulated in a common format of integer linear programs and solved by off-the-shelf software. "Large-scale" means that the size of the problems makes their solution with off-the-shelf methods inefficient or even practically impossible. Indeed, this is often the case in such applications as stereovision and image segmentation, estimation of object rotation and translation in space, as well as cell matching and tracking in microscopic images. We not only develop dedicated algorithms to optimize given combinatorial problems, but also learn problem parameters from training data. To this end we combine artificial neural networks with the most efficient combinatorial techniques.

PD Dr. Karl RohrBioQuant Center, IPMB, Biomedical Computer Vision Group

Biomedical Image Analysis, Cell Microscopy and Medical Images, Deep Learning, Model-Based Methods

The Biomedical Computer Vision Group headed by Karl Rohr develops methods and algorithms for computer-based analysis of biological and medical images, in particular, cell microscopy images and medical tomographic images. The focus is on deep learning and model-based methods for segmentation, tracking, and image registration.

Prof. Dr. Stefan RiezlerStatistical Natural Language Processing Group

Machine Translation, Conversational AI, Statistical Methods

The research of the statistical natural language processing group is at the intersection of machine learning and natural language processing. The complexity of natural language data presents a continued challenge for standard supervised learning techniques, thus requiring sophisticated machine learning methods that enable learning from ambiguous, noisy, and sparse natural language input. One focus of the group is on interactive machine learning, applied especially to the natural language processing problem of machine translation. Automatic translation of text and speech has made extraordinary progress in recent years, due to the ability to efficiently train highly expressive neural networks on large scale. However, the quality of machine translation output still does not meet professional standards and needs to involve human translators. One goal is making human interaction as effortless and easy as possible, for example, by leveraging weak human feedback in form of translation quality judgements instead of fully correct translations. The machine translation challenge is to extract a strong enough signal for learning from such ambiguous and noisy feed feedback signals, for example, by off-policy learning from deterministic interaction logs, reinforcement learning using reward estimators trained on human feedback, and self-supervised learning using new interaction modes such as error markings. The techniques for interactive sequence-to-sequence learning developed in our group are also important in other natural language applications such as conversational AI, or in machine learning of time series in medical informatics.

Prof. Dr. Stefania PetraMathematical Imaging Group

Mathematical Imaging, Convex Optimization, Learning by Assignment Flows, Sparse representations in Inverse Problems

The main research interests of our group lie in mathematical models and computational approaches for image analysis and machine learning using variational methods, numerical optimization and information geometry. Applications include imaging-based inverse problems in medicine and industry, like parameter estimation from ultrasound images, image classification and discrete tomography. Prof. Petra is PI in Informatics4Life. Our teaching focuses on mathematical aspects of modern image processing methods and convex and non-convex optimization. Special emphasis is given to the presentation of basic ideas and mathematical concepts.

Last updated on Jul 7, 2022 at 3:16 PM