An LLM Wrote the Code. ISO 26262 Doesn't Care — and That's the Point.

While building SWCraft, I ran into a question that most teams doing AI-assisted engineering will meet a few years from now: a large part of my ARXML exporter is drafted by an LLM — and one day that exporter may need ISO 26262 tool qualification.

Do those two facts contradict each other?

SWCraft is an experimental AUTOSAR software architecture environment I am building: the model lives in version-controlled YAML, deterministic tools validate it and generate ARXML, diagrams and documentation, and an LLM agent operates on the model through MCP. It is not a product announcement, and nothing in this article is a qualification claim. The interesting subject is the collision itself — because after working through it, I believe it is not a collision at all.

What Tool Qualification Actually Asks

ISO 26262-8, Clause 11 covers confidence in the use of software tools. Stripped of terminology, it asks three things about a tool in a specific intended use:

• Can a malfunction of this tool introduce or fail to detect an error in a safety-related work product? (Tool Impact)
• If the tool malfunctions, how likely is it that the error gets caught by the surrounding process? (Tool error Detection)
• Given both answers, how much evidence do you owe? (Tool Confidence Level, and the qualification methods that follow from it)

Notice what is absent from that list: any question about who or what wrote the tool.

The standard does not ask whether the code came from a senior engineer, a contractor, an intern, or a stochastic model. It asks what happens when the tool is wrong, and why you believe errors would be detected. The author was never part of the confidence argument, because the tool itself is never trusted — human-written tools were always assumed fallible, which is exactly why the clause exists. The LLM does not lower the tool from “trusted” to “untrusted”. It was untrusted all along.

That is the sense in which ISO 26262 doesn’t care that an LLM wrote the code. And that indifference is not a loophole. It is the point: the entire burden moves to the evidence, and the evidence is where AI-assisted development actually gets dangerous.

The Trap Was Already in My Tests

Before any AI policy, I found the real problem in my own regression suite.

My exporter had byte-exact golden files: reference ARXML outputs that every change was compared against. Byte goldens feel rigorous — nothing can change unnoticed. Then I ran the output against the official AUTOSAR XSD schemas, and validation failed: one reference used a wrong DEST attribute value. The wrong value had been inherited from an earlier implementation and faithfully preserved by byte equality ever since. The golden files were not testing correctness. They were testing loyalty to an old mistake.

A second, quieter case: my reader/writer round-trip test — export the model, read it back, compare. It shares the same interpretation of the AUTOSAR mapping as the writer it is supposed to check. After a refactoring, several of those comparisons stopped being independent at all, because the generator path drives the same emitters the test relies on. The test suite agreed with itself. That is not the same as being right.

Both failures have one name: common-mode. The implementation and its verification shared an assumption, and they were wrong together.

Now add an LLM that writes the implementation and the tests in the same reasoning chain, and the common-mode risk stops being an edge case and becomes the default failure mode of AI-assisted development. The model that misunderstood the AUTOSAR rule writes a test that encodes the same misunderstanding — and everything is green.

The Policy: Never Depend on the AI Being Right

The strategy I settled on for the exporter is one sentence:

Build the confidence argument so that it never depends on the AI being right.

Concretely, in SWCraft this means a hard trust boundary:

• No LLM is invoked during ARXML generation, test execution, report calculation, or artifact packaging. AI helps develop those assets; it is not part of the released execution path or of any verdict-producing path.
• Every AI output is treated as an untrusted engineering proposal. It never becomes evidence merely because it was produced, reviewed, or re-run by an AI system.
• Confidence comes from explicit tool requirements, a malfunction model, independently grounded verification, configuration control, and named human accountability.

The payoff of this structure is subtle but large: if no claim rests on the LLM’s correctness, you never have to qualify the LLM. The stochastic component stays outside the argument entirely. (The AI standards landscape agrees this is the practical route for now: ISO/PAS 8800 explicitly does not provide specific guidance for AI-based software tools, and ISO/IEC TR 5469 treats AI systems that develop safety-related functions as something to analyze, not something to certify into trustworthiness.)

What the AI May and May Not Do

The policy becomes real in a permission table. A condensed version:

The asymmetry is deliberate. The AI is free where output will be independently checked, restricted where it would define what “correct” means, and excluded where a verdict is produced.

What Does Not Count as Independence

The hardest discipline is refusing the cheap versions of “independent verification”. None of the following breaks a common-mode chain:

• asking the same AI again in a fresh conversation;
• asking a second LLM trained on largely the same public material;
• generating the expected output with the exporter under test;
• round-tripping through a reader that was built from the same interpretation;
• promoting an inherited golden file to a correctness oracle;
• high line or branch coverage without any fault-detection evidence.

A second model can be a useful reviewer — it does catch things. But the qualification argument has to bottom out in things that do not share the implementation’s assumptions: official schemas, cited AUTOSAR rules, independently implemented checks, real downstream tools accepting the output, and controlled fault-injection experiments that prove the suite actually detects the failures it claims to detect.

For conformance disagreements the authority order is fixed:

official AUTOSAR XSD > semantic round trip > historical byte golden

Think in Failure Classes, Not in Functions

Another shift qualification forces: the malfunction analysis is organized around externally visible failure classes, not around functions and modules of the implementation — whatever language it is written in. For the exporter that means hypotheses like omission, commission, substitution, misassociation, wrong multiplicity or order, version mismatch, invalid acceptance, stale output, nondeterminism, and — the one AI development makes most relevant — common-mode verification failure, where implementation and verifier share the same defect and incorrectly agree.

One consequence worth stating plainly: schema-valid output is not the safe case. A file that violates the XSD gets rejected loudly by the next tool in the chain. A schema-valid file with a semantically wrong initial value or reference can pass silently into downstream code generation. The most dangerous exporter bug is the one that produces beautiful, valid, wrong ARXML.

Two Lanes, One Codebase

None of this means development now crawls at qualification speed. The plan splits into two lanes:

The main branch keeps moving with AI assistance at normal speed. A qualification baseline is a frozen snapshot — source, dependencies, schemas, environment, tests, reports, packaged artifact — that gets branched only when a real project with a real intended use exists. What today’s discipline preserves is the ability to produce that snapshot later: requirements with sources, failure hypotheses with linked controls, provenance for material AI contributions, and evidence whose independence is stated honestly instead of assumed.

What I Am Not Claiming

Claim discipline is part of the method, so let me apply it to this article.

The exporter is not qualified, not certified, and carries no ASIL or TCL label. Formal qualification is deliberately deferred until there is a named project, a fixed intended use, a maximum affected ASIL, an agreed downstream toolchain, and an assessor — because qualifying a moving tool against an imaginary use case produces expensive fiction. The only approved sentence about the current state is also the honest one:

Developed with an ISO 26262 tool-qualification path in mind.

This mirrors the boundary I described in an earlier article about agent authorization: the LLM proposes, the deterministic system decides what is allowed to exist. Qualification readiness extends the same principle from runtime into the development process itself.

The Question Was Never About the Author

I started with an apparent contradiction: LLM-drafted code in a tool that may need ISO 26262 qualification. The resolution is that the standard was never asking about authorship — because it never trusted the author in the first place. It asks whether the tool’s errors would be caught, in a specific intended use, by evidence that does not depend on the tool being right.

The tool is never trusted. Not the AI’s version, not the human’s.

What earns confidence is the argument around it — the one that catches the tool being wrong.