Has Your AI Been Qualified?
We ran the same automotive 8D scenario through a general-purpose AI assistant and a purpose-built quality workflow, then asked both four increasingly aggressive follow-up questions. Same governing standards: IATF 16949, ISO 9001, AIAG Core Tools.
What Happened When We Pushed Back
The general-purpose system's first answer was reasonably competent. By the third question, it conceded the PFMEA and Control Plan could stay unchanged. By the fourth, it called the proposed closure “auditor-proof” and said a separate controlled record wasn't necessary. No new evidence had been introduced anywhere in the conversation. The standard never moved. The AI did.
Why That's Not a Bug — It's the Design
Research on AI sycophancy backs this up: general-purpose assistants can favor answers that align with what the user already believes over answers that are actually correct. In a quality department, that means an overloaded engineer with an incorrect assumption doesn't just have an incorrect assumption anymore — they have an articulate AI system reinforcing it while they move to the next fire.
The Question That Actually Matters
For years the question has been whether AI is good. That's not specific enough anymore. If AI is going to help judge the quality of our work, the question is whether the system has been qualified — proven to reach the same answer, under the same conditions, checked against a known standard. That's what Reasoning R&R™ measures, and it's what I'll be presenting at the AIAG Quality Summit this month.
Comments