auto: 2026-08-10T16:23:35Z [skip ci]

content/essays/swe_vc.md

@@ -3,7 +3,7 @@ title: "Provenance Is Not Warrant"
 subtitle: "Code Was Never the Hard Part"
 date: 2026-08-10
 abstract: >
-  Software engineering does so little epistemic work that it hardly earns the name "engineering." The field has never learned to distinguish provenance — where an artifact came from — from warrant — why anyone is entitled to rely on it. Authorship, a passing test suite, and a completed review are routinely mistaken for the second when they are only ever the first; libraries are the rare exception, where warrant is actually constructed and amortized across users who never read the source. LLM-generated code inherits neither comforting story, and the discomfort that provokes is not a new problem but the oldest one in the profession, finally felt without the anaesthetic that provenance usually supplies. That discomfort is worth welcoming.
+  Software engineering does so little epistemic work that it hardly earns the name "engineering." The field has never learned to distinguish provenance — where an artifact came from — from warrant — why anyone is entitled to rely on it. Authorship, a passing test suite, and a completed review are routinely mistaken for the second when they are only ever the first; libraries are the rare exception, where warrant is actually constructed and amortized across users who never read the source. LLM-generated code inherits neither comforting story, and the discomfort that provokes is not a new problem but the oldest one in the profession, finally felt without the anaesthetic that provenance usually supplies.
 tags: [ai, tech, nonfiction/philosophy, education]
 status: Working model
 confidence: 60
@@ -35,7 +35,7 @@ Perhaps we can pinpoint the origin of the schism. I think much of it is due to t
 
 Now perhaps a third term should be introduced. Comprehension is distinct from both Provenance and Warrant. Merely understanding, to some extent, *how* a component works is not a warrant. Your understanding of a system's working is not a form of entitlement to rely on said system, but rather the capacity to produce the evidence that would entitle you.  To put this another way, one can only specify behavior that they understand, can only state invariants that they have grasped. One cannot construct an adversarial test for a failure mode that hasn't been imagined, cannot imagine failure modes in a system whose structure is entirely opaque to you in the name of "abstraction." If you build your software with legos, you'd best understand the shapes of the pieces and distribute the weight properly! Comprehension is perhaps the most valuable, as it forms the prerequisite for producing any form of warrant, rather than the warrant itself. 
 
-Not all "comprehension" is created equal, however. As I have elaborated on in [[Asymmetric Forgetting]], what a curriculum instills divides into two kinds of knowledge with entirely different decay properties. Procedural fluency is a sequence, one that dies without reactivation. Conceptual scaffolding, on the other hand, is a schema that persists because the world keeps incidentally touching and probing it through the beauty of overwhelming correlation. That distinction transposes onto software quite nicely. If you know how to write a bounded worker pool in a particular language, you have mastered a sequence. If you remember `public static void main(String[] args)`, you know a particularly monotonous (though useful!) sequence. On the other hand, if you know what a bounded worker pool is actually *for*, what it guarantees under load, what failure it exists to prevent, and what happens when the queue saturates, then you have a schema. 
+Not all "comprehension" is created equal, however. As I have elaborated on in [Asymmetric Forgetting](/essays/asymmetric-forgetting.html), what a curriculum instills divides into two kinds of knowledge with entirely different decay properties. Procedural fluency is a sequence, one that dies without reactivation. Conceptual scaffolding, on the other hand, is a schema that persists because the world keeps incidentally touching and probing it through the beauty of overwhelming correlation. That distinction transposes onto software quite nicely. If you know how to write a bounded worker pool in a particular language, you have mastered a sequence. If you remember `public static void main(String[] args)`, you know a particularly monotonous (though useful!) sequence. On the other hand, if you know what a bounded worker pool is actually *for*, what it guarantees under load, what failure it exists to prevent, and what happens when the queue saturates, then you have a schema. 
 
 In the pedagogical case, the concept will let you reacquire a lost procedure in minutes. Anyone could look at any basic information on the Java programming language to figure out how to set up their main method, much less the official Java documentation. Even better, LLMs can now be used to reacquire lost procedure at speeds never before imagined, and with much less mental burden, too, not that there ever was a terrible burden to begin with. The converse does not hold; a retained procedure cannot summon a concept that was never built. If you recognize the Java verbosity from before, but cannot explain why in the broader picture of Java `main` itself would be a method that needs to be invoked in this way, then you have merely memorized a procedure. Schema without procedure, then, is not more than a mild inconvenience. This is what documentation and search engines exist for. Procedure without schema, in stark contrast, is disabling, because the person who can produce the implementation without holding the concept fails to recognize what properties the component they're implementing must guarantee, cannot recognize what evidence its use, its warrant, would require. They are not failing to be diligent --- hardly at all, especially in the eyes of the software engineering industry --- but rather they are failing to notice that a question has even been posed in the first place. In other words, "I wrote it, so I know what it does." 
 
@@ -83,7 +83,7 @@ Writing is thinking when it builds a schema. When the act of implementation forc
 
 Where software does genuinely behave like engineering looks incredibly unlike ordinary LEGO coding. Engineering seeks to make clear what properties a component must satisfy, under what operating conditions, with what margin. It asks for evidence of these claims, somewhat analogous to science. Science asks what is the claim, what evidence bears on it, what are the controls and how reproducible are the results? I do genuinely believe that computer science as a proper field draws upon both of these. It is a true science when done right. LEGO coding substitutes something weaker in the place of these, as we have discussed. It is that an implementation was written, that the diff was reviewed, perhaps even *comprehensively*, that the suite was ran and CI was passed, and that everything was deployed.
 
-You might ask at this point whether LEGO coding *needs* what engineering can provide. But proportioning assurance to consequence is engineering itself, not a compromise of engineering. The scandal is not that ordinary software carries little warrant, but that the profession built on such software cannot say *why* that amount is appropriate to the consequence of failure. The vocabulary for hazard analysis, for goal structuring notation, for rigorous epistemic effort, has existed in the realm of computer science more broadly. It is simply quarantined in some way, taught and utilized in a few places like those I've mentioned previously, and unknown to everyone else. I do not think that the quarantine is solely an accident of professional culture, nor the result of optimizing for profits above all else. Computing education continuously optimizes for procedural fluency just as purely as any curriculum in the sciences can --- the same [[Asymmetric Forgetting|asymmetry]] at work in a new domain. The assessed object is always a program that compiles, runs, and passes a predefined test suite. If you are curious, [go](https://docs.google.com/document/d/1N5sOUQjdxuUJEZtOgrBsIClgYE9O16zvJA5vntbEhLA/edit?tab=t.0#heading=h.vqq71w99z8c9) [look](https://csci0300.github.io/syllabus/#grading) [at](https://cs.brown.edu/courses/cs019/2025/policy.html#%28part._.Grades%29) [some](https://cs.brown.edu/courses/csci1515/spring-2026/misc/debugging.html) [documents](https://cs.brown.edu/courses/csci1660/old/2023/files/assignments/handin.pdf) from my undergrad curriculum to see that autograders based on test suites are the core of what determines whether you graduate with a degree in CS or not. Interestingly, some of the syllabi will say that focusing on the autograder is entirely the wrong thing to do and that the code is reviewed and graded by humans, but when only about ~5% of the grade comes from that review, and 95% from the success of your code on a test suite, reality speaks louder than words. What a specification is, what would count as sufficient evidence for it, and what consequence bears on how much evidence one needs --- these are things that the computing curriculum do not teach. Thus, a discipline which trains the sequence and leaves the schema to unfavorable chance has produced exactly the world that we see: a world of practitioners fluent in memorized procedures and mute on assurance, who can build things but cannot say what would establish that what they've built actually works. The vocabulary didn't transfer because there was no scaffolding for it to attach to in the first place. 
+You might ask at this point whether LEGO coding *needs* what engineering can provide. But proportioning assurance to consequence is engineering itself, not a compromise of engineering. The scandal is not that ordinary software carries little warrant, but that the profession built on such software cannot say *why* that amount is appropriate to the consequence of failure. The vocabulary for hazard analysis, for goal structuring notation, for rigorous epistemic effort, has existed in the realm of computer science more broadly. It is simply quarantined in some way, taught and utilized in a few places like those I've mentioned previously, and unknown to everyone else. I do not think that the quarantine is solely an accident of professional culture, nor the result of optimizing for profits above all else. Computing education continuously optimizes for procedural fluency just as purely as any curriculum in the sciences can --- the same [asymmetry](/essays/asymmetric-forgetting.html) at work in a new domain. The assessed object is always a program that compiles, runs, and passes a predefined test suite. If you are curious, [go](https://docs.google.com/document/d/1N5sOUQjdxuUJEZtOgrBsIClgYE9O16zvJA5vntbEhLA/edit?tab=t.0#heading=h.vqq71w99z8c9) [look](https://csci0300.github.io/syllabus/#grading) [at](https://cs.brown.edu/courses/cs019/2025/policy.html#%28part._.Grades%29) [some](https://cs.brown.edu/courses/csci1515/spring-2026/misc/debugging.html) [documents](https://cs.brown.edu/courses/csci1660/old/2023/files/assignments/handin.pdf) from my undergrad curriculum to see that autograders based on test suites are the core of what determines whether you graduate with a degree in CS or not. Interestingly, some of the syllabi will say that focusing on the autograder is entirely the wrong thing to do and that the code is reviewed and graded by humans, but when only about ~5% of the grade comes from that review, and 95% from the success of your code on a test suite, reality speaks louder than words. What a specification is, what would count as sufficient evidence for it, and what consequence bears on how much evidence one needs --- these are things that the computing curriculum do not teach. Thus, a discipline which trains the sequence and leaves the schema to unfavorable chance has produced exactly the world that we see: a world of practitioners fluent in memorized procedures and mute on assurance, who can build things but cannot say what would establish that what they've built actually works. The vocabulary didn't transfer because there was no scaffolding for it to attach to in the first place. 
 
 ## Code was never the hard part!
 LEGO coding has built elaborate institutions for depending on code written by other people. Take semantic versioning, lockfiles, reproducible resolution, CVEs, automated advisory scanning, you name it. What has LEGO coding built for relying on code written by the developers themselves? Internal code is, in most cases, the least documented, least contract-bearing, and most important in the system. It implements the specific business logic that was so important that it gave its developers their jobs, after all. It often doesn't have versioning (the shipped product might, but minor changes to some functions aren't versioned, so hopefully you catch them in a diff!), no changelogs, no fuzzing, etc. It passes the lackluster test suites without any challenge because there is an implicit "warrant," which is truthfully just provenance: "someone here wrote it, so we own it and we understand it."