Intelligent Medical Systems Lab (IMEDS)

AI for diagnostics & treatment of chronic conditions

About

Intelligent Medical Systems Laboratory is led by Dr. Aleksei Tiulpin and conducts machine learning research to advance the field of AI for preventive care. We are headquartered at the Department of Radiology, Weill Cornell Medicine, New York City. Prior to that, the lab was located at the University of Oulu, Finland, and some of the members still remain there, finishing their projects.

Our expertise is in building machine learning methods and tools that advance diagnostics, treatment, and scientific discover across diseases, organs, and modalities. Over the years, we have developed special expertise in non-communicable choronic diseases and conditions, such as osteoarthritis and low back pain. The goal of our research is to develop AI technology that saves lives and prevents disability.

Clinical aims:

Democratize
access to care

Prevent disease
before it happens

Prevent treatment
complications

From a machine learning standpoint, the key research efforts in the group are directed toward building methods that are

Scalable

Digest data from different modalities and with various sizes

Trustworthy

Can generate outputs that we, as humans, can trust

Deployable

Align with the downstream goals of healthcare

News

03.03
2026

Our work on subtyping of lumbar spine degeneration trajectories has won the ISSLS 2026 Prize and is now published at the European Spine Journal!

Link ↗

26.02
2026

One paper on autoregressive generative models is accepted to ICLR 2026!

Link ↗

24.02
2026

One paper on conformal prediction for ratio-based biomarkers is accepted to MIDL 2026!

Link ↗

23.02
2026

We have launched our website!

Featured Publications

Explore our latest research contributions in AI, medical imaging, and chronic diseases

View PDF

Spine 2026

OLSIA: Open Lumbar Spine Image Analysis - A 3D Slicer Extension for Segmentation, Grading, and Intervertebral Disc Height Index with Multi-Dataset Validation

Kowlagi, N., Kemppainen, A., McSweeney, T., Saarakkala, S., Noailly, J., Williams, F. M., ... & Tiulpin, A. (2025). OLSIA: Open Lumbar Spine Image Analysis-A 3D Slicer Extension for Segmentation, Grading, and Intervertebral Disc Height Index with Multi-Dataset Validation. Spine, 10-1097.

Low back pain and lumbar spine degeneration represent major global health challenges requiring accurate imaging assessment for diagnosis and treatment planning. This study developed and validated the Open Lumbar Spine Image Analysis (OLSIA) software, a user-friendly, no-code application that automates lumbar spine segmentation, grading, and intervertebral disc height index (DHI) calculations using deep learning models trained on the Finnish NFBC1966 dataset. The retrospective and cross-sectional study evaluated OLSIA's performance across six external datasets from diverse geographical regions (Hong Kong, UK, Spain, Hungary, Netherlands, and global sources), analyzing T2-weighted MRI mid-sagittal slices of vertebral bodies (L1-S1) and intervertebral discs (L1/2-L5/S1). The software demonstrated remarkable efficiency with a 222-fold improvement in processing time compared to manual analysis, while maintaining high inter-rater reliability (mean Dice similarity coefficient >90%) and minimal systematic bias in DHI measurements (mean difference of 0.02), making it a valuable tool for researchers from diverse backgrounds to accelerate radiomics and lumbar spine image analysis workflows without requiring coding expertise.

View PDF

WACV 2025

Image-level Regression for Uncertainty-aware Retinal Image Segmentation

Dang, T., Nguyen, H. H., & Tiulpin, A.

Accurate retinal vessel (RV) segmentation is a crucial step in the quantitative assessment of retinal vasculature, which is needed for the early detection of retinal diseases and other conditions. Numerous studies have been conducted to tackle the problem of segmenting vessels automatically using a pixel-wise classification approach. The common practice of creating ground truth labels is to categorize pixels as foreground and background. This approach is, however, biased, and it ignores the uncertainty of a human annotator when it comes to annotating e.g. thin vessels. In this work, we propose a simple and effective method that casts the RV segmentation task as an image-level regression. For this purpose, we first introduce a novel Segmentation Annotation Uncertainty-Aware (SAUNA) transform, which adds pixel uncertainty to the ground truth using the pixel's closeness to the annotation boundary and vessel

View PDF

IEEE JBHI 2025

End-to-End Prediction of Knee Osteoarthritis Progression With Multimodal Transformers

Panfilov, E., Saarakkala, S., Nieminen, M. T., & Tiulpin, A.

Knee Osteoarthritis (KOA) is a prevalent chronic musculoskeletal condition with no currently available treatment. Predicting its progression is difficult due to its varied manifestation. Recent studies highlight the potential of using multimodal data and Deep Learning (DL) for prediction, though evidence is still emerging. In our study, we leveraged DL, specifically, a Transformer model, to fuse multimodal knee imaging data. We analyzed its performance across different progression horizons – from short-term to long-term – using a large dataset (n = 3967/2421) from the Osteoarthritis Initiative. We show that structural knee MRI allows identifying radiographic KOA progressors on par with multimodal fusion approaches, achieving an area under the ROC curve (ROC AUC) of 0.70-0.76 and Average Precision (AP) of 0.15-0.54 in horizons from 2 to 8 years. Multimodal approach using X-ray, structural and compositional MR images was more effective for predicting 1-year progression, achieving a ROC AUC of 0.76 (0.04) and AP of 0.13 (0.04). Our follow-up analysis suggests that prediction from the imaging data is particularly accurate for post-traumatic cases, and we further investigate which subject subgroups may benefit the most. The study offers new insights into multimodal imaging of KOA and brings a unified data-driven framework for studying its progression end-to-end, providing new tools to enhance clinical trial design. The source code of our framework and the pre-trained models are made publicly available.

View PDF

UAI 2025

Bayesian Optimization over Bounded Domains with the Beta Product Kernel

Nguyen H., Zhou H., Blaschko M.B., & Tiulpin A.,

Bayesian optimization with Gaussian processes (GP) is commonly used to optimize black-box functions. The Matérn and the Radial Basis Function (RBF) covariance functions are used frequently, but they do not make any assumptions about the domain of the function, which may limit their applicability in bounded domains. To address the limitation, we introduce the Beta kernel, a non-stationary kernel induced by a product of Beta distribution density functions. Such a formulation allows our kernel to naturally model functions on bounded domains. We present statistical evidence supporting the hypothesis that the kernel exhibits an exponential eigendecay rate, based on empirical analyses of its spectral properties across different settings. Our experimental results demonstrate the robustness of the Beta kernel in modeling functions with optima located near the faces or vertices of the unit hypercube. The experiments show that our kernel consistently outperforms a wide range of kernels, including the well-known Matérn and RBF, in different problems, including synthetic function optimization and the compression of vision and language models.

View PDF

ACCV 2024

LoG-VMamba : Local-Global Vision Mamba for Medical Image Segmentation

Dang, T., Nguyen, H. H., & Tiulpin, A.

Mamba, a State Space Model (SSM), has recently shown competitive performance to Convolutional Neural Networks (CNNs) and Transformers in Natural Language Processing and general sequence modeling. Various attempts have been made to adapt Mamba to Computer Vision tasks, including medical image segmentation (MIS). Vision Mamba (VM)-based networks are particularly attractive due to their ability to achieve global receptive fields, similar to Vision Transformers, while also maintaining linear complexity in the number of tokens. However, the existing VM models still struggle to maintain both spatially local and global dependencies of tokens in high dimensional arrays due to their sequential nature. Employing multiple and/or complicated scanning strategies is computationally costly, which hinders applications of SSMs to highdimensional 2D and 3D images that are common in MIS problems. In this work, we propose Local-Global Vision Mamba, LoG-VMamba, that explicitly enforces spatially adjacent tokens to remain nearby on the channel axis, and retains the global context in a compressed form. Our method allows the SSMs to access the local and global contexts even before reaching the last token while requiring only a simple scanning strategy. Our segmentation models are computationally efficient and substantially outperform both CNN and Transformers-based baselines on a diverse set of 2D and 3D MIS tasks.

View PDF

MICCAI 2024

SiNGR: Brain Tumor Segmentation via Signed Normalized Geodesic Transform Regression

Dang, T., Nguyen, H. H., & Tiulpin, A.

One of the primary challenges in brain tumor segmentation arises from the uncertainty of voxels close to tumor boundaries. However, the conventional process of generating ground truth segmentation masks fails to treat such uncertainties properly. Those "hard labels" with 0s and 1s conceptually influenced the majority of prior studies on brain image segmentation. As a result, tumor segmentation is often solved through voxel classification. In this work, we instead view this problem as a voxel-level regression, where the ground truth represents a certainty mapping from any pixel to the border of the tumor. We propose a novel ground truth label transformation, which is based on a signed geodesic transform, to capture the uncertainty in brain tumors' vicinity. We combine this idea with a Focal-like regression L1-loss that enables effective regression learning in high-dimensional output space by appropriately weighting voxels according to their difficulty. We thoroughly conduct an experimental evaluation to validate the components of our proposed method, compare it to a diverse array of state-of-the-art segmentation models, and show that it is architecture-agnostic. The code of our method is made publicly available

View PDF

Spine 2024

Semiautomatic Assessment of Facet Tropism From Lumbar Spine MRI Using Deep Learning: A Northern Finland Birth Cohort Study

Kowlagi, N., Kemppainen, A., Panfilov, E., McSweeney, T., Saarakkala, S., Nevalainen, M., ... & Tiulpin, A.

Lumbar spine disorders, particularly low back pain affecting over 576 million people globally, are often associated with facet tropism (FT) - the asymmetry between left and right facet joint angles - which is considered a risk factor for disc degeneration and spinal stenosis, yet manual measurement of FT from medical images is time-consuming and limits research to small sample sizes. This study developed the first deep learning-based framework using a UNet++ model to automatically measure facet joint angles from T2-weighted axial MRI images, training on 430 participants from the Northern Finland Birth Cohort 1966 and achieving high segmentation performance (92.7% Dice score, 87.1% intersection over union). The researchers applied their method to analyze 1,288 participants, demonstrating strong inter-rater reliability between radiologists (r² = 0.84 for asymmetry measurements) and finding that facet tropism prevalence was higher in males (particularly at the L4/5 level with 44.6% affected using the >7° threshold), with measured facet joint angles closely matching previous literature values from smaller manual studies. This semiautomatic approach enables large-scale population studies of facet tropism that were previously impractical due to the labor-intensive nature of manual angle measurements, potentially advancing understanding of facet tropism as a biomarker for lumbar spine pathology.

View PDF

IEEE TMI 2024

Clinically-Inspired Multi-Agent Transformers for Disease Trajectory Forecasting From Multimodal Data

Nguyen, H. H., Blaschko, M. B., Saarakkala, S., & Tiulpin, A.

Deep neural networks are often applied to medical images to automate the problem of medical diagnosis. However, a more clinically relevant question that practitioners usually face is how to predict the future trajectory of a disease. Current methods for prognosis or disease trajectory forecasting often require domain knowledge and are complicated to apply. In this paper, we formulate the prognosis prediction problem as a one-to-many prediction problem. Inspired by a clinical decision-making process with two agents–a radiologist and a general practitioner – we predict prognosis with two transformer-based components that share information with each other. The first transformer in this framework aims to analyze the imaging data, and the second one leverages its internal states as inputs, also fusing them with auxiliary clinical data. The temporal nature of the problem is modeled within the transformer states, allowing us to treat the forecasting problem as a multi-task classification, for which we propose a novel loss. We show the effectiveness of our approach in predicting the development of structural knee osteoarthritis changes and forecasting Alzheimer’s disease clinical status directly from raw multi-modal data. The proposed method outperforms multiple state-of-the-art baselines with respect to performance and calibration, both of which are needed for real-world applications.

View all publications