MedPerf

MedPerf is an open-source framework spearheaded by MLCommons aimed at standardizing the evaluation of medical AI models on decentralized, real-world data. Its architecture addresses the critical bottleneck of data privacy in healthcare by facilitating 'Federated Evaluation.' Instead of moving sensitive patient data to a central server, MedPerf orchestrates the movement of models (encapsulated in MLCubes) to the data owners' infrastructure. In the 2026 landscape, MedPerf has matured into a critical piece of the clinical validation pipeline, enabling researchers and regulatory bodies to assess algorithm performance across diverse populations without violating HIPAA or GDPR. The platform utilizes a three-pillar actor system: Benchmark Owners (who define tasks), Data Owners (who provide local clinical data), and Model Owners (who submit algorithms for testing). By ensuring reproducibility through containerization and providing an auditable trail of performance metrics, MedPerf bridges the gap between laboratory development and clinical deployment, fostering trust in AI-driven diagnostic and prognostic tools.