Shipping AI to a Million Patients Without an A/B Test — Jared Joselowitz, Ufonia
Jared Joselowitz from Ufonia describes how to safely launch Dora, an AI voice assistant that calls patients for follow-up after surgery in British hospitals.
Because the system provides medical advice, it is a regulated medical device—you cannot A/B-test on patients as that is unethical, and incorrect advice cannot be retracted. Instead, Ufonia uses simulation: one AI plays a patient against hazards written together with doctors, and another AI evaluates each call. The evaluating model was tested against 10 specialists on 240 cases and performed equally well or better. Instead of adjusting instructions by hand, they optimize them against a cost matrix that weighs over-diagnosis against missed warning signs.
Vibekollen prepared this summary with AI from the original publication. The content belongs to AI Engineer.
