Introduction

Vera leverages sophisticated AI agents and Retrieval-Augmented Generation technology, synthesising knowledge from over 60 million peer-reviewed medical publications to deliver reliable, contextually-appropriate answers. This multi-benchmark assessment evaluates Vera's performance across three distinct medical knowledge domains: the United States Medical Licensing Examination (USMLE), the New England Journal of Medicine AI Q&A dataset (NEJM-AI), and the MedXpertQA benchmark.

Performance Metrics

• On USMLE, Vera achieved an exceptional overall accuracy of 97.5%, with step-specific accuracies of 97.9% (Step 1), 98.2% (Step 2 CK) and 96.7% (Step 3).


• On the NEJM-AI benchmark comprising 655 questions across five medical specialties, Vera demonstrated superior performance with 84.9% accuracy, outperforming leading AI models including OpenAI o4 Mini (77.1%), Claude 4 Sonnet (75.4%), and Perplexity Sonar Pro (74.4%).


• On the MedXpertQA benchmark comprising 500 questions across multiple body systems and medical tasks, Vera achieved 62.2% accuracy, demonstrating strong performance in specialized clinical reasoning scenarios.


• Vera achieved the highest accuracy in four out of five NEJM-AI medical specialties, with particularly strong performance in Pediatrics (93.9%) and Internal Medicine (87.3%).

Conclusion

These results across diverse evaluation frameworks underscore Vera's robust medical knowledge representation and reasoning capabilities, positioning it as a leading solution for clinical decision support. You can read the full paper here.