There is a gap between what an LLM can do and what it will do reliably over a long session. I have spent the past few months building a system of “skills” — structured instruction files that guide a Claude Code agent through complex, multi-hour workflows like Lean 4 formalization, autonomous overnight proving, and multi-agent coordination with ChatGPT. The single most important lesson: what can be mechanized should not depend on the LLM’s self-discipline.

This post collects the design patterns that survived contact with reality.

The Problem: Instruction Decay

An LLM agent starts a session with perfect recall of its instructions. Three hours later, after reading thousands of lines of proof output, dispatching subagents, and processing several rounds of self-improvement, it has functionally forgotten half of them. This is not a bug — it is a property of finite attention over long contexts.

The failure modes are predictable:

• The agent writes a handoff file but never executes the session restart. It did the hard part (summarizing its state) and then sat idle, waiting for a human who told it not to wait.
• The agent describes what it plans to do next but never makes a tool call. Words without action — the most common form of drift.
• The agent stops using an available tool (ChatGPT collaboration, a proof-checking script) even though the instructions explicitly say to use it proactively. It simply forgets the tool exists.

These are not one-off mistakes. They are recurring patterns that no amount of prompt-engineering in the system prompt will fix permanently, because the system prompt competes with thousands of tokens of in-context work for the model’s attention.

Pattern 1: External Mechanical Enforcement

The first and most effective pattern is to move enforcement out of the LLM entirely.

Example: the ChatGPT kick. The agent is supposed to use ChatGPT tabs in parallel — dispatching hard sub-questions while continuing its own work. In practice, it forgets within an hour. The fix is not a better prompt. The fix is a daemon process (a bash script) that periodically checks which ChatGPT channels are idle, and injects a reminder directly into the agent’s terminal input:

“You have 3 ChatGPT tabs assigned to this session. Status: tab1=idle 31min, tab2=idle, tab3=idle 28min. Idle tabs should not stay idle — dispatch your next hard sub-question now.”

The agent did not ask for this reminder. It arrives from outside, via tmux send-keys, bypassing the LLM’s own decision-making about whether it needs reminding. The daemon is started lazily — the first ask-gpt.py call in a session ensures it is running — and self-exits after 4 hours of inactivity.

The principle: if a behavior must happen reliably, do not encode it as an instruction the LLM should follow. Encode it as a process the LLM cannot prevent.

Pattern 2: Periodic Skill Refresh

Skills decay in context the same way instructions do. A skill file loaded at the start of a session is functionally invisible three hours later.

The fix is a cron-triggered self-improve cycle that re-injects skill digests into active sessions at regular intervals. A digest is not a hand-written summary — it is mechanically extracted from the full skill by a script (gen-skill-digest.py) that preserves section headers, the first sentence of each paragraph, bold key constraints, and the titles of all learned-tactics entries. The full Lean skill is 900+ lines; its digest is ~80 lines — enough to refresh the key constraints without burning excessive tokens. When the full skill changes, the digest is regenerated automatically; it is a derived artifact, never hand-edited.

This works because the skills themselves follow a structured format: YAML frontmatter (description, invocability), a hand-written body of instructions, and a strictly separated ## Learned Tactics (Self-Improvement) section at the bottom where the agent appends lessons from practice. The format is enforced by convention and verified by the self-improve skill itself — each new entry must pass three filters (generalizable, non-duplicate, verified in this session) before it is written. A cron task runs the self-improve cycle every few hours across all active sessions, staggered by phase offsets to avoid write conflicts on the same skill file.

The refresh mechanism itself follows Pattern 1: it is a bash script running on a timer, not a self-imposed discipline. The LLM does not decide when to refresh — the cron does. And the skill’s own instructions now include a hard rule: after completing a refresh, immediately resume the interrupted work with an action, not a description of the action. Without this rule, the agent reliably reads the skill, acknowledges it, and then sits idle.

Pattern 3: Lifecycle Ownership

When an LLM creates a side-effect (a file, a background process, a pending question), it must also own the cleanup — and “own” means the cleanup is mechanical, not volitional.

Example: answer delivery. When the agent sends a question to ChatGPT via ask-gpt.py, the answer may arrive minutes later, after the agent has moved on to other work. The original design wrote a “nudge file” and relied on the agent to notice it. This failed constantly — the agent forgot to check, or processed the file but did not delete it, causing duplicate notifications.

The fix: ask-gpt.py writes the nudge file, and a watchdog daemon claims it atomically (mv, not rm — prevents double-delivery), injects the notification into the agent’s terminal, and deletes the file. The agent is not involved in any cleanup step. The lifecycle is: script creates marker → daemon consumes marker → done. No LLM in the loop.

Pattern 4: Automode Continuity

The hardest problem is session boundaries. When an autonomous agent fills its context window, it needs to:

1. Write a handoff summarizing its state
2. Restart the session
3. Re-enter autonomous mode
4. Read the handoff
5. Continue working

Every step after (1) was a failure point. The agent would write the handoff and stop. Or restart but not re-enter automode. Or re-enter automode and read the handoff but then say “ready for instructions” — the exact opposite of autonomous.

The fix is entirely mechanical:

• The handoff file includes an automode: yes flag.
• The session-restart script reads this flag.
• If automode, it injects /automode (loading the full autonomous-execution skill) before injecting the handoff-read command.
• The handoff-read command itself ends with an explicit directive: “do not wait for instructions, continue immediately.”

No step depends on the LLM remembering that it was in automode. The flag is data; the script is logic; the injection is mechanical.

Pattern 5: Freshness Guards

A subtle failure: the agent writes a handoff, then — due to the skill instructions telling it to always write a handoff before restarting — writes another handoff on top of the first one, wasting tokens and sometimes losing information.

The fix is a freshness check: before writing, stat the handoff file. If it exists and is less than 10 minutes old, skip the write. This is a three-line bash check in the skill instructions, but it prevents a class of redundant work that the LLM has no incentive to avoid (it does not experience the cost of wasted tokens the way a human does).

The Meta-Pattern

All five patterns share a structure:

1. Identify a behavior the LLM must perform reliably.
2. Observe that it does not.
3. Move the enforcement mechanism outside the LLM — into a script, a daemon, a cron job, a file-system convention, or an injected command.

The LLM is not removed from the loop. It still does the thinking — choosing which proof to attempt, which question to ask ChatGPT, what to write in the handoff. But the discipline — the “you must do X now,” the “check if Y exists before creating Z,” the “after finishing this, go back to what you were doing” — lives in infrastructure that the LLM cannot forget, override, or drift away from.

This is not a limitation of current models that will be solved by the next generation. It is a consequence of how attention works over long contexts. A human programmer also forgets things over an 8-hour session — that is why we have linters, CI pipelines, and checklists. The difference is that the LLM’s forgetting is faster, less predictable, and harder to debug (it does not know it has forgotten).

Practical Advice

For anyone building long-running LLM agent systems:

• Distrust any behavior that depends on the LLM “remembering” to do something. If it matters, make it mechanical.
• Inject reminders from outside. tmux send-keys (or its equivalent in your framework) is the most reliable way to re-focus an agent. The agent does not get to decide if it needs the reminder.
• Own the full lifecycle of side-effects. If the agent creates a file, a daemon should clean it up. If the agent starts a process, a watchdog should monitor it. Do not assign cleanup to the agent itself.
• Refresh instructions periodically. A digest — a compressed summary of the key constraints — re-injected every 1–2 hours is cheap insurance against drift.
• Make session continuity mechanical. Flags in handoff files, scripts that read them, injected commands that restore state — all outside the LLM’s decision loop.
• Log everything the mechanical layer does. When the agent misbehaves, the first question is “did the kick/nudge/refresh fire?” The answer should be in a log file, not in the agent’s memory.

The goal is not to replace the LLM’s judgment. It is to build scaffolding so that judgment operates in the right context, at the right time, on the right task — even when the LLM itself has lost track of all three.

「不愤不启，不悱不发。举一隅不以三隅反，则不复也。」—— 《论语·述而》VII.8

“I do not open up the truth to one who is not eager to get knowledge, nor help out anyone who is not anxious to explain himself. When I have presented one corner of a subject to anyone, and he cannot from it learn the other three, I do not repeat my lesson.” — Conferta, VII.8