TL;DR
The project instructions (CLAUDE.md, AGENTS.md), skills and commands you start a project with were written before the project had told you anything about itself, which makes them a first draft by construction. Every session that went wrong, or only went right because you stepped in, can show you something the setup should have known. Improve it while that session is still in context: the agent still has the whole run in view, so you can work out together what went wrong and what would have prevented it. After every session, ask what would make the next one run cleaner, for as long as the project runs.
The setup is done. That is how it gets reported at the end of the first week of an agentic project, meaning the guidelines are written, the skills are in place, the connectors are wired, and the team is producing work against all of it. What is worth stopping on is the word done.
A starter set is generic on purpose, and the generic parts are the good parts: an instruction file from a template, a review skill, a commit skill, a session handover, the connectors everyone wires up first. They encode what is true of every project, which is exactly the property that makes them unable to encode anything true only of this one.
The domain vocabulary, the parts of the codebase where the agent keeps going wrong, the conventions the team settles on as it goes, the review that keeps finding the same class of problem: none of that existed as knowledge when the setup was written.
Calling the setup done at the point it starts working treats the first week as the week you learned the most about the project, and it is the week you knew least.
Every correction is worth asking whether the setup could have prevented it.
#Sometimes the answer is no, because work with an agent goes wrong now and then, the way work with a colleague does. Other times it is yes, because you told the agent something the setup could have told it: that this repository runs its tests with its own command, that endpoints in an API follow a fixed naming pattern, that a particular helper is the one to use in a given context. The output came back right and the session gets filed as a success, but what you supplied by hand is gone the moment the session closes. Next week the same situation comes up and you supply it again.
Asked cold whether the setup can be improved, an agent will find something.
#It will be plausible, too: a tightened instruction, an unhandled edge case, a section that could be clearer. None of it will be traceable to anything that happened. This is : these models answer in a way that pleases and affirms the person asking, and anyone who works with them daily has watched it happen. Ask Claude Code whether something could be better and it opens by telling you what a good question that is, then proposes changes that may or may not help, because you asked for improvements.
What makes it expensive here is not that the suggestions are bad. It is that you have nothing to validate them against: with no failure behind them, each one stays a potential improvement, and whether it added any real value you never find out.
With the run still loaded, the agent can tell you what sent it the wrong way.
#The trigger is not a schedule and not a review cycle. It is a specific observation: this session took a lot of correcting to get where it got, from you or from the agent catching itself and going round again. Either way something in the setup may have been missing or wrong, and right now the entire run is still in the context window: what it was asked, what it did, where it doubled back, what had to be supplied by hand. That is the material, and this is the only moment you will have all of it at once.
So discuss it. Ask what sent it down that path, push back on the first answer, and find out together what the setup was missing: a fact the instructions never state, a step a skill or command skips, or a rule nobody enforces yet, which becomes a check the agent runs before it reports done.
A new skill or command is warranted by a need the project surfaced.
#The rule that governs edits governs additions too. Later in the project, tasks turn up that no starter set could have covered: a release check, adding an API endpoint the way this codebase does it.
The signal that one deserves its own skill is repetition: you have walked the agent through it by hand more than twice, and it will keep coming back. No best-practice list will tell you which tasks those are. The project tells you, and only after it has run for a while.
None of this is a large amount of work: a line here, a check there, each one making the next session run a little cleaner. What it does take is discipline from every developer on the project to make those changes as the need shows up. Over a long stretch of agentic coding on a live product, the most valuable parts of the setup have been the ones improved step by step over months.
Keep exploring
In the Reference Project