heart structure and function
Traditional foundation models struggle to process high-dimensional linked data, limiting their clinical value. Multimodal foundation models (MMFM) offer a breakthrough by integrating diverse data sources such as:
Use case in
Health Domain
AI-Informed
Diagnosis Support
Echocardiographic imaging
Echocardiographic imaging
heart structure and function
Electrocardiograms (ECG)
Electrocardiograms (ECG)
Blood-based biochemistry
Blood-based biochemistry
This multimodal analysis is critical for diagnosing complex conditions like acute coronary syndrome, heart failure, and pulmonary embolism.
Key features
Dedicated Cardiology MMFM Agents
Analyze unstructured medical reports alongside structured data to detect abnormalities and inform patient care.
Privacy-Preserving Federated Training
Perform pre-training and fine-tuning across distributed data silos in European hospitals without moving data across institutions.
Cross-Modality Generation of Missing Medical Data
Generate missing medical information using available modalities. For example, reconstruct lab values or MRI-derived features from echocardiography and ECG data. This approach uses physics-informed, data-driven methods to address data gaps.
Key benefits
Modality Generation with Privacy
Generate synthetic but accurate missing data for clinical use.
Cost Savings
Reduce dependency on expensive or less accessible medical tests.
Privacy-Preserving Open Data
Create large synthetic datasets with safeguards against memorization for open research.
Patient-Facing Communication via Translation and Dialogue
A conversational agent automatically translates and simplifies patient letters. It enhances communication by supporting real-time dialogue, helping patients understand diagnoses and treatment options.
Key features
Improved Patient Understanding
Translate complex medical terms into simpler or native-language content.
User-Centric Interaction
Support patient engagement through natural, evaluable conversation.
Doctor Support for Predictive and Scenario-Based Clinical Decisions
Develop an agent that predicts a patient’s health trajectory—such as upcoming tests or lab results—based on medical records and treatments. It allows clinicians to simulate different treatment scenarios and assess potential outcomes.
Key features
Patient Trajectory Forecasting
Predict next medical events based on intervention history.
What-If Analysis
Help clinicians compare treatment options, especially in complex cases (e.g., coexisting depression and diabetes).
Uncertainty Estimation
Provide confidence levels for predicted outcomes, supporting informed clinical decision-making.