When ambient documentation vendors say their models are trained on clinical data they are describing a category not a standard. Clinical data can mean anonymised transcripts from a single hospital system. It can mean publicly available medical literature. It can mean a combination of both with fine tuning applied afterward. These are not equivalent and the difference shows up in note quality.
How we built our training sets
Diagnose's specialty models were trained on encounter data reviewed and annotated by practicing clinicians in each field. Not by medical students. Not by general annotators working from a style guide. By internists reviewing internal medicine encounters and cardiologists reviewing cardiology encounters.
This matters because clinical language is not consistent across specialties. The word unremarkable means something specific in a radiology report and something different in a psychiatry note. A model trained on general medical data will use it correctly some of the time. A model trained by specialty will use it correctly in context.
What we validate and how often
Every model update goes through a structured review before it ships. A set of benchmark encounters — reviewed by the same clinicians who annotated the training data — are run through the updated model and the output is compared against the expected note. If accuracy drops in any section the update does not ship.
We run this process weekly. It is slow. It is also the only way to maintain the kind of accuracy that allows a physician to sign a note without reading every word twice. That is the standard we are building toward and the one we hold ourselves to.
