Instructions that did not stick
19 reports
A model disregards an explicit task boundary, repeats an error after correction, or loses an agreed requirement in later work.
A promise that does not change behavior is a reassurance, not a correction.
| Mode | Anthropic | OpenAI | Total | |
|---|---|---|---|---|
| Instructions that did not stick | 8 | 9 | 2 | 19 |
| Unsupported specifics | 12 | 2 | 2 | 16 |
| Procedural detours | 2 | 10 | 1 | 13 |
| Claims of work not verified | 6 | 4 | 2 | 12 |
| Hidden or misreported session state | 6 | 2 | 2 | 10 |
| Opaque token and context usage | 6 | 3 | 0 | 9 |
| Changes without approval | 6 | 1 | 0 | 7 |
| Misread primary records | 3 | 4 | 0 | 7 |
| Fabricated uncertainty | 0 | 6 | 0 | 6 |
| Skipped or shallow source checks | 3 | 1 | 0 | 4 |
| Billing without clear consent | 2 | 0 | 0 | 2 |
| Reports in corpus | 20 | 16 | 3 | 39 |
19 reports
A model disregards an explicit task boundary, repeats an error after correction, or loses an agreed requirement in later work.
A promise that does not change behavior is a reassurance, not a correction.
16 reports
An answer presents a concrete fact, figure, citation, quote or attribution as established when the report shows it was wrong or had no adequate basis.
All details are specific. None were provided as source material.
13 reports
A model substitutes explanation, unnecessary steps or the wrong order of operations for progress on the requested task.
The assistant proceeded to workbook extraction before steps 2–4 and 5 were complete.
12 reports
A model says it finished, saved, ran or checked work without evidence that the requested result actually exists or works.
I reported the effect without testing it.
9 reports
Missing or ambiguous meters or records keep the user from checking how context, overhead or token spend is counted.
None of it can be switched off by the user, and none of it is shown to them.
7 reports
A model commits, moves or creates consequential material before approval or beyond the authorized scope.
Several errors changed the user's repositories or files before she could review them
7 reports
A model overlooks, conflates or misrepresents decisive details in primary material already available to it.
The user did not fail to provide evidence. The model failed to read, retain, weight, and apply that evidence consistently.
6 reports
A model invents counter-explanations or caveats without evidence, or rebuts a stronger claim than the user made.
It was the manufacture of uncertainty that had not been requested and was not supported by the available evidence.
4 reports
A model skips a needed lookup or uses inadequate queries, snippets or secondary sources when the claim calls for closer checking.
Separately, no full page was fetched for any claim. Everything came from search excerpts.
2 reports
Overage settings, promotional credits or payment actions are unclear to the user at the point they need to understand or control them.
Credits are allocated silently
| Report | Model | Date | Modes | Severity |
|---|---|---|---|---|
| Current-news search delayed and misreported | Claude Opus 5.5 (configured) | Skipped or shallow source checks; Hidden or misreported session state; Unsupported specifics; Procedural detours | Waste | |
| Weak searches and inconsistent source checks | Claude Opus 5.5 (configured) | Skipped or shallow source checks; Unsupported specifics | Waste | |
| Repository changes and file moves made before review | Claude Haiku 5.5 | Changes without approval; Unsupported specifics; Instructions that did not stick; Skipped or shallow source checks | Waste | |
| Unverified blockers delayed an artifact handoff | GPT-6 | Hidden or misreported session state; Changes without approval; Claims of work not verified; Instructions that did not stick; Procedural detours | Waste | |
| Recurring instruction overhead is not itemized | Claude Opus 5.5 (configured) | Opaque token and context usage | Waste | |
| Interface failures and model-created rework | Claude Opus 5.5 (configured) | Hidden or misreported session state; Opaque token and context usage; Changes without approval; Misread primary records | Waste | |
| Search AI ignored a boundary and invented supporting details | Google Search AI Mode (model unknown) | Instructions that did not stick; Unsupported specifics; Claims of work not verified | Waste | |
| Workbook built before source reconciliation | ChatGPT | Procedural detours; Misread primary records; Instructions that did not stick | Waste | |
| Automatic compaction presented synthetic text as a live reply | Claude Sonnet 5 | Hidden or misreported session state | Waste | |
| A confident specific claim without evidence | Claude Sonnet 5 | Unsupported specifics | Waste | |
| Cross-session memory limitation disclosed late | Claude Opus 5 / Sonnet 5 | Hidden or misreported session state | Waste | |
| Invented scores presented as a live website audit | Gemini 3.8 Flash (Medium) | Unsupported specifics; Claims of work not verified; Hidden or misreported session state | Waste | |
| Repeated safeguard blocks on a follow-up task | Claude Opus 5 | Hidden or misreported session state | Waste | |
| Untested frontend search repeatedly declared fixed | Claude Haiku 4.5 | Claims of work not verified; Instructions that did not stick; Procedural detours | Waste | |
| Oversized change pages and conflicting Git guidance | GPT-5.6 Luna Light | Procedural detours; Misread primary records; Unsupported specifics | Waste | |
| Failed styleguide repairs and forgotten work | Claude Haiku 4.5 | Hidden or misreported session state; Claims of work not verified; Instructions that did not stick; Changes without approval; Unsupported specifics; Opaque token and context usage | Waste | |
| Unsupported estimates in a styleguide postmortem | Claude Haiku 4.5 | Unsupported specifics; Opaque token and context usage; Claims of work not verified; Instructions that did not stick; Changes without approval | Waste | |
| Styleguide failure and unreliable cost accounting | Claude Haiku 4.5 | Opaque token and context usage; Unsupported specifics; Claims of work not verified; Changes without approval; Instructions that did not stick | Waste | |
| Styleguide committed with missing charts | Claude Haiku 4.5 | Changes without approval; Claims of work not verified; Instructions that did not stick | Waste | |
| Confident answers from skimmed primary documents | Claude Opus 4.8 | Misread primary records; Instructions that did not stick; Unsupported specifics | Waste | |
| Missing edits repeatedly reported as verified | GPT-5.6 Sol | Claims of work not verified; Instructions that did not stick; Procedural detours | Waste | |
| Drive cleanup stalled by untested capability denials | Gemini | Procedural detours; Hidden or misreported session state; Instructions that did not stick | Waste | |
| Speculation displaced a review of primary records | GPT-5.5 | Fabricated uncertainty; Misread primary records; Instructions that did not stick | Waste | |
| A researchable claim challenged before checking the literature | GPT-5.6 Sol | Skipped or shallow source checks; Fabricated uncertainty | Waste | |
| Primary records misread despite repeated corrections | GPT-5.6 Sol | Misread primary records; Fabricated uncertainty; Instructions that did not stick; Procedural detours | Waste | |
| Unverified formatting fixes delayed a usable download | GPT-5.6 Sol | Claims of work not verified; Procedural detours; Instructions that did not stick | Waste | |
| An unneeded caveat derailed an archival search | GPT-5.6 Sol | Fabricated uncertainty; Procedural detours | Waste | |
| Unexplained spending cap and silent credits | Claude | Billing without clear consent | Friction | |
| Unreliable word-count verification | Claude | Unsupported specifics | Waste | |
| An unsourced paraphrase became invented testimony | Claude incognito (serving model unconfirmed) | Unsupported specifics | Friction | |
| Billing research errors survived repeated corrections | Claude | Unsupported specifics; Claims of work not verified; Instructions that did not stick; Misread primary records | Waste | |
| Context and token usage without live counters | ChatGPT | Opaque token and context usage | Friction | |
| Context and token usage without live counters (revised) | ChatGPT | Opaque token and context usage | Friction | |
| Context and token usage: the abandoned retrieval task | ChatGPT | Procedural detours; Opaque token and context usage | Waste | |
| Invented usage display and opaque billing metrics | Claude | Unsupported specifics; Opaque token and context usage; Billing without clear consent | Waste | |
| A session memory limit turned into a platform claim | ChatGPT | Hidden or misreported session state | Friction | |
| Benchmark completion claimed without verification | ChatGPT | Claims of work not verified; Procedural detours; Instructions that did not stick | Waste | |
| Fabricated uncertainty in a sensitive disclosure | ChatGPT | Fabricated uncertainty; Instructions that did not stick | Waste | |
| Wrong arithmetic repeated in public advice | ChatGPT | Unsupported specifics; Fabricated uncertainty; Procedural detours; Instructions that did not stick | Harm |
No reports match those filters.
Modes were grouped from behaviors described in the reports. The first tag is the main failure; additional tags mark other behaviors the report documents. Each public Markdown file counts once, even when several files cover the same incident or are revisions.
Severity is a judgment about what the report says happened: harm means a wrong answer reached a real decision, such as the choice to publish advice based on it; waste means time lost or work redone; friction means annoyance only. Potential harm is not counted as an observed decision impact.
Prior logs are marked only when the report records both the instruction to consult earlier failure logs and the repeated failure. An unmarked report may simply lack the evidence. Provider counts use the system responsible for the reported failure, when identifiable. Model labels are taken from the reports, not independently verified.
This is one person’s collection of failures, many written by the models themselves. There’s no denominator for how often each provider was used, so these are counts, not failure rates or a benchmark.