Fool's Expertise

Bryan Cantrill coins "Fool's Expertise" for the failure mode he sees in AI doom debates: experts in one domain (e.g., LLM researchers like Geoffrey Hinton) claiming authority over questions in entirely different domains — radiology practice, viral synthesis, nuclear safeguards, cybersecurity. His trigger is the Ezra Klein interview with Nvidia's Jensen Huang, where Klein repeatedly appealed to Hinton's authority and Huang missed the strongest rebuttal.

The illustrative case is Hinton's 2016 prediction that radiology would cease to exist by 2021, on the strength of image interpretation. Cantrill's point isn't just that the prediction failed; it's that it failed because Hinton didn't know what radiologists actually do — diagnosis is one small part of the job. Automating a link in a chain is not automating the chain.

The structural argument: catastrophes involve long chains of causation crossing many domains, so understanding one link grants no authority over the chain — the NTSB's multidisciplinary Go Teams are his model. Therefore any AI-doom claim that depends on a pathway into the physical world must consult experts on that pathway: cybersecurity people (who note the OpenAI escape was a pedestrian containment failure), virologists, nuclear-policy veterans of seven decades of deterrence. He closes by endorsing RAND's "On the Extinction Risk from AI," which deliberately interviewed domain experts and not AI researchers.