# The Half-Life of AI Scaffolding I have long assumed that AI scaffolding would expire as model capabilities changed. Keeping it small does not make it last longer; it only makes it easier to remove when its time is up. ## Metadata - HTML: https://glenzli.com/en/notes/half-life-of-ai-scaffolding/ - Markdown: https://glenzli.com/en/notes/half-life-of-ai-scaffolding.md - Collection: Notes - Language: en - Published: 2026-07-16 - Updated: 2026-07-16 - Tags: ai, agent, workflow, context, maintenance ## Content Recently I all but reset an AI scaffolding repository. A single commit removed more than ten thousand lines of code, tests, and supporting documentation. The repository had grown an entire toolchain around project knowledge bases. It could inventory a codebase, decide what was worth recording, then build, query, update, and inspect that knowledge in stages. Code review, test review, migration, and retrospectives each had their own skill. To keep the knowledge from going stale, it tracked source changes and flagged material that might need another pass. Eventually it had its own CLI, specifications, release process, and a large test suite. This was not another round of trimming. I dismantled the scaffolding itself. The repository no longer builds a persistent copy of implementation knowledge, provides a source index or CLI, or tries to cover the full development cycle. What remains is a handful of templates and four short skills that help an AI establish a minimal project skeleton: why the project exists, which constraints are relatively stable, where facts should be found, and where a person still needs to decide. How the code is organized, what the interfaces look like, and how modules call one another are left for the model to read from the source. Once I started deleting, I did not feel surprised. The feeling was closer to: yes, finally. I have long assumed that general-purpose AI scaffolding would not live for very long. Structure that does no real work quickly gets in the way. Structure that does help may be absorbed by the model or its runtime after one or two generations. That assumption is why I kept the system in small pieces. It began as more than a dozen skills, each responsible for one narrow job. Even after inspection tools and a CLI appeared, it was still possible to use only the part a task needed. Making it small was never meant to extend its life. It was meant to let each layer be bypassed, measured on its own, and removed without disturbing the rest of the project. That is more or less what happened: ten thousand lines disappeared, the repository survived, and so did the few things worth keeping. ## Obsolescence Does Not Throw an Error Ordinary software usually leaves some evidence when it ages. Dependencies stop being maintained, interfaces are deprecated, and tests begin to fail. AI scaffolding tends to go obsolete much more quietly. When a model cannot find the right files, we prescribe a reading route. When it struggles with long context, we make it summarize before drilling down. When it misses checks, we assign several roles to review the same work in sequence. When its reasoning drifts, another paragraph appears in the system prompt telling it to check sources, identify assumptions, and keep guesses out of factual claims. These designs often worked at the time. Otherwise they would not have grown out of repeated tests in the first place. Then models begin to perform more of those steps on their own, while the old workflow remains in place. It still runs. A stronger model can carry it and still produce good results. What becomes difficult to tell is how much of the result comes from the scaffolding and how much now belongs to the model's improved baseline. The same thing happened to a set of logic-correction prompts I once maintained. I did not start that repository because I wanted a collection of clever prompts. Repeated failures in ordinary use forced it into existence. Models would follow a typo into a confident chain of reasoning, mistake commands quoted in reference material for instructions addressed to them, or lose track of the current question's scope, time, and sources. I did not think models ought to have those problems, but at the time they did. I tried to keep the intervention as small as possible: point out the failure mode and let the model correct its own course, rather than replace its reasoning with a larger procedure. The best future for those prompts was never indefinite expansion. It was deletion, one by one, as models improved. Later I added tests to compare behavior with and without the prompts. The tests could not prove that every failure had vanished, but they were enough to show that the full set no longer deserved a permanent place in my daily system prompt. I removed it from regular use and stopped actively maintaining the repository, leaving it only as a record of the experiment. That was the outcome I had hoped for when I built it. While archiving the repository, I also asked an AI what was worth investing in next. Its answer was that, instead of maintaining general prompts, I should turn to concrete workflows such as building, maintaining, and querying project knowledge bases. I did not fully accept that answer. What mattered more to me was private knowledge: project facts and history that would never enter public training data and could not appear automatically with a model upgrade. But preserving that knowledge did not necessarily require a complete procedure telling the model how to create, query, and update it. If general reasoning, retrieval, and source reading became strong enough, perhaps a few cues would be sufficient: where the facts live, and what cannot be inferred from code. The model could handle the rest. Resetting the knowledge-base repository eventually produced the same conclusion. Private knowledge had not lost its value. The general workflow built around it did not need to remain forever. The ten thousand lines I removed were exactly that kind of workflow: more durable than a set of generic prompts, but not by very much. The two repositories reached this point at different times, but they invite the same test. The question is not only whether the system can still run. It is what would actually be lost if the system were removed. ## How Much Do You Lose When You Remove It? When a skill is released, the usual demonstration shows how well the model performs after the skill is installed. A few months later, the base model has improved. It still produces good results with the same skill attached, and those results are easily credited to the skill. In practice, people rarely remove it completely and run a fresh baseline with the new model. By then, the more useful question may be: > How much is still missing once the skill is gone? Without that deletion test, improvements in the model itself are easily entered in the scaffolding's ledger. A workflow does not even need to keep providing a clear benefit. As long as it does not seriously damage the result, it can continue to look effective. Answering the question requires more than reading the skill or looking at an assisted result. At minimum, the same set of tasks must be compared with and without it. [SkillsBench](https://www.skillsbench.ai/skillsbench.pdf) uses this kind of paired evaluation. That method matters to me more than any single score. A skill is no longer asked only to show what happens when it is present. It must also show what disappears when it is taken away. The results are useful too. Skills improve aggregate performance, but some tasks get worse under guidance that is unnecessary or conflicts with the task. Long, comprehensive versions can also perform worse than shorter, more focused ones. That fits my own view: a skill that tries to cover everything is difficult to justify. If it reliably improves a broad range of tasks, that usually points to a specific gap in the model's general ability, one that an external structure is temporarily filling. Once the model closes the gap, the structure begins to lose its purpose. Specialist skills tend to have a clearer effect because they contribute domain knowledge, concrete constraints, or ways to verify the work. They are not trying to rewrite the model's general habits of thought. A generic skill has two choices. It can remain short and address one clear problem, or it can compromise repeatedly to fit more situations until it becomes bland. Trying to cover everything often means reaching nothing in particular. None of this means that a skill is effective simply because it is short or specialized. A short instruction can still be redundant, and a professional workflow can still expire. A piece of guidance does not earn permanent value by sounding reasonable, complete, and experienced. Meanwhile, models and runtimes continue to absorb capabilities that used to require external orchestration. As I write this, OpenAI has just added programmatic tool calling and multi-agent coordination for [GPT-5.6](https://openai.com/index/gpt-5-6/) in the Responses API. That model name will age quickly, so I mention it only as a timestamp. Work that once had to be arranged in the application layer is still moving downward into the model and its environment. Release notes can tell us when to test again. They cannot make the judgment for us. Whether a particular skill is ready to be removed still depends on the tasks and baselines that matter in actual use. ## Keep What Cannot Be Reconstructed Removing a workflow does not mean removing the knowledge with it. Models becoming better at finding information does not make project documentation unnecessary. A project should still tell whoever arrives later why it exists, which directions were tried and abandoned, and what must not be changed casually. That material is more useful when it points the model toward facts than when it rehearses the process of reading the source for it. After the reset, I kept only a method for establishing a minimal project skeleton. It no longer tries to maintain a second copy of implementation knowledge that will expire alongside the implementation. Instead, it tells an AI to identify goals and non-goals, extract stable constraints, record sources and verification commands, mark decisions that require a person, and keep all of this brief. It discourages retelling modules, classes, interfaces, and call relationships. Those details change quickly and are better read directly from source for the task at hand. Put another way, I no longer ask documentation to speak on behalf of the code. I ask it to preserve what the code cannot say for itself. I made a similar tradeoff in another project concerned with long-lived context. Its first design prescribed not only how information should be stored, but also how a model should route, summarize, expand, and organize it. Most of that process is now gone. What remains is a simpler idea: history should be kept as addressable, sourced, revisable Pages. The protocol constrains Pages, Scopes, Revisions, provenance, and basic I/O. Any authorized model may decide for itself how to search, read, and organize them. For the models and tasks I currently use, the model can manage the context in front of it well enough that the protocol no longer needs to explain how to turn every page. But no amount of model capability can recover history it was never allowed to see. Old sessions, other projects, and ideas that were never committed to the current repository do not appear simply because reasoning has improved. These changes clarify a distinction: preserving context and prescribing how a model uses context are not the same job. The former still needs infrastructure. The latter can become thinner as models improve, and may eventually leave the protocol altogether. That distinction gives me a useful way to examine any workflow. If I delete it, am I losing instructions tailored to one generation of models, or am I losing constraints and history that the project cannot recover from its source and tools? Search order, summary depth, reflection steps, and role assignments all carry assumptions about model capability. A new model is a reason to test those assumptions again. Other facts are less likely to be erased by an upgrade: which tests a release must pass, which operations require human confirmation, and which problems the project explicitly refuses to solve. Those constraints come from reality. They are better encoded in tests, permissions, and short project notes than left for a model to infer afresh every time. Source code can tell a model how the system works today. It may not explain why a direction was abandoned. A model can find a way to finish a task, but it does not automatically know which tradeoff belongs to this project. So I now lean on this test: > If current source and tools can reconstruct something on demand, it does not need to be repeated forever. Intent, history, responsibility, and boundaries that cannot be recovered from source are what deserve to remain. Stronger models reduce the need to prescribe actions. They do not preserve history for us, and they do not take over the final judgment. That is why I did not delete all the documentation. I removed retellings that would expire with the code. I kept the reasons the code cannot express, along with a minimal method for giving an AI the skeleton of a project. ## Things Built to Come Down The fact that these structures were later removed does not mean they should never have existed. I am reluctant to dismiss all of them in hindsight as overengineering. Most grew in response to failures in real tasks. They solved actual problems and completed a useful transition. People like to build wheels, as if making one now will save effort forever. But a durable wheel is difficult to build, and it was difficult before the AI era too. It must remain useful across projects, teams, and environments, then survive testing, compatibility work, and maintenance. Building a wheel has always been expensive, which makes it easy to mistake the investment for the value. Once so much effort has gone into it, the wheel feels as though it ought to remain, preferably in enough places to justify the work. AI lowers the cost of making things. It does not make durability easy. Often the real need is only a temporary crutch for a limitation in the current model. Once the capability catches up, the crutch can be put down. Solving today's problem does not always require fitting it into the so-called BIG PICTURE, and every temporary structure does not need to be repackaged as durable knowledge with a permanent maintenance burden. Looking back at these repositories, the knowledge-base scaffolding left behind a method for establishing a project skeleton. The context protocol kept persistent Pages, provenance, and basic read semantics. The prompt repository left me with the habit of building baselines and evaluations. None became permanent infrastructure, but each helped me distinguish temporary support for one stage from something closer to the project itself. I will still build AI workflows. When a problem keeps recurring and materially affects the result, adding a temporary layer is entirely reasonable. Even if next month's model may solve it, people still have work to do this month. The important thing is to leave room for deletion while building it. The layer should be bypassable and its effect measurable. When the model crosses the relevant threshold, removing the whole layer should not damage the project underneath. I will keep building scaffolding, and I will keep it small. Not because I expect it to live for years, but because I want to remove it when the model reaches the other side without having to pull the whole project apart. As for permanent tenure: no. ## Related Repositories - [Dev Skeleton](https://gitlab.com/glenzli/dev-skeleton): the minimal project-skeleton method retained from the knowledge-base scaffolding. - [LLM Prompt Evals](https://github.com/glenzli/llm-prompt-evals): general-purpose prompts and evaluation experiments that are no longer actively maintained. - [Paged Context Protocol](https://gitlab.com/glenzli/paged-context-protocol): a user-owned Page space across sessions and projects; the current protocol retains persistence and access semantics, while the earlier fixed routing and summarization workflow has been archived.