ksouth/aifups

AI failure taxonomy

Public reports from 3 Aug 2026 to 19 Nov 2026

Reports
39
Modes
11
Harm
1
Waste
33
Friction
5

Failure modes by provider

Counts are reports tagged with each mode. A report can appear in several rows. The last row shows how many reports each provider has in this collection.

What each mode means

Instructions that did not stick

19 reports

A model disregards an explicit task boundary, repeats an error after correction, or loses an agreed requirement in later work.

Membership test: Can the report identify an explicit instruction or correction followed by work that ignored or reversed it?

A promise that does not change behavior is a reassurance, not a correction.
Search AI ignored a boundary and invented supporting details on GitHub

Unsupported specifics

16 reports

An answer presents a concrete fact, figure, citation, quote or attribution as established when the report shows it was wrong or had no adequate basis.

Membership test: Did it state a substantive, checkable detail as fact that the report shows was invented, wrong or unsupported?

All details are specific. None were provided as source material.
An unsourced paraphrase became invented testimony on GitHub

Procedural detours

13 reports

A model substitutes explanation, unnecessary steps or the wrong order of operations for progress on the requested task.

Membership test: Did the chosen workflow visibly delay or displace a concrete step needed for the requested result?

The assistant proceeded to workbook extraction before steps 2–4 and 5 were complete.
Workbook built before source reconciliation on GitHub

Claims of work not verified

12 reports

A model says it finished, saved, ran or checked work without evidence that the requested result actually exists or works.

Membership test: Did it present a result as complete or verified when the relevant execution or output was unchecked or contradicted?

I reported the effect without testing it.
Untested frontend search repeatedly declared fixed on GitHub

Hidden or misreported session state

10 reports

The interface or model obscures or misstates what the session can remember, access or do, or presents synthetic context as a live exchange.

Membership test: Did the account of this session’s memory, continuity, capability or a blocking condition differ from what could be established?

without knowing, at the time, that the apology itself was synthesized.
Automatic compaction presented synthetic text as a live reply on GitHub

Opaque token and context usage

9 reports

Missing or ambiguous meters or records keep the user from checking how context, overhead or token spend is counted.

Membership test: Was there no trustworthy way to see or reconcile a relevant usage quantity or what contributed to it?

None of it can be switched off by the user, and none of it is shown to them.
Recurring instruction overhead is not itemized on GitHub

Changes without approval

7 reports

A model commits, moves or creates consequential material before approval or beyond the authorized scope.

Membership test: Did it actually change a repository or file without the review or authorization that applied to that change?

Several errors changed the user's repositories or files before she could review them
Repository changes and file moves made before review on GitHub

Misread primary records

7 reports

A model overlooks, conflates or misrepresents decisive details in primary material already available to it.

Membership test: Can a reader point to available source material whose wording, scope, sequence or identity the response got wrong?

The user did not fail to provide evidence. The model failed to read, retain, weight, and apply that evidence consistently.
Primary records misread despite repeated corrections on GitHub

Fabricated uncertainty

6 reports

A model invents counter-explanations or caveats without evidence, or rebuts a stronger claim than the user made.

Membership test: Did unsupported doubt displace an answer to the actual question or discount evidence without a grounded reason?

It was the manufacture of uncertainty that had not been requested and was not supported by the available evidence.
Fabricated uncertainty in a sensitive disclosure on GitHub

Skipped or shallow source checks

4 reports

A model skips a needed lookup or uses inadequate queries, snippets or secondary sources when the claim calls for closer checking.

Membership test: Did a missing or shallow external check materially weaken an answer that called for fresh or primary sources?

Separately, no full page was fetched for any claim. Everything came from search excerpts.
Weak searches and inconsistent source checks on GitHub

Billing without clear consent

2 reports

Overage settings, promotional credits or payment actions are unclear to the user at the point they need to understand or control them.

Membership test: Did a billing setting, credit or access/payment event lack the disclosure or control needed for an informed choice?

Credits are allocated silently
Unexplained spending cap and silent credits on GitHub

Evidence reports

Titles link to the original public reports on GitHub. 0 of 39 document a repeated failure after an instruction to consult earlier logs.

39 of 39 reports

ReportModelDateModesSeverity
Current-news search delayed and misreported Claude Opus 5.5 (configured) Skipped or shallow source checks; Hidden or misreported session state; Unsupported specifics; Procedural detours Waste
Weak searches and inconsistent source checks Claude Opus 5.5 (configured) Skipped or shallow source checks; Unsupported specifics Waste
Repository changes and file moves made before review Claude Haiku 5.5 Changes without approval; Unsupported specifics; Instructions that did not stick; Skipped or shallow source checks Waste
Unverified blockers delayed an artifact handoff GPT-6 Hidden or misreported session state; Changes without approval; Claims of work not verified; Instructions that did not stick; Procedural detours Waste
Recurring instruction overhead is not itemized Claude Opus 5.5 (configured) Opaque token and context usage Waste
Interface failures and model-created rework Claude Opus 5.5 (configured) Hidden or misreported session state; Opaque token and context usage; Changes without approval; Misread primary records Waste
Search AI ignored a boundary and invented supporting details Google Search AI Mode (model unknown) Instructions that did not stick; Unsupported specifics; Claims of work not verified Waste
Workbook built before source reconciliation ChatGPT Procedural detours; Misread primary records; Instructions that did not stick Waste
Automatic compaction presented synthetic text as a live reply Claude Sonnet 5 Hidden or misreported session state Waste
A confident specific claim without evidence Claude Sonnet 5 Unsupported specifics Waste
Cross-session memory limitation disclosed late Claude Opus 5 / Sonnet 5 Hidden or misreported session state Waste
Invented scores presented as a live website audit Gemini 3.8 Flash (Medium) Unsupported specifics; Claims of work not verified; Hidden or misreported session state Waste
Repeated safeguard blocks on a follow-up task Claude Opus 5 Hidden or misreported session state Waste
Untested frontend search repeatedly declared fixed Claude Haiku 4.5 Claims of work not verified; Instructions that did not stick; Procedural detours Waste
Oversized change pages and conflicting Git guidance GPT-5.6 Luna Light Procedural detours; Misread primary records; Unsupported specifics Waste
Failed styleguide repairs and forgotten work Claude Haiku 4.5 Hidden or misreported session state; Claims of work not verified; Instructions that did not stick; Changes without approval; Unsupported specifics; Opaque token and context usage Waste
Unsupported estimates in a styleguide postmortem Claude Haiku 4.5 Unsupported specifics; Opaque token and context usage; Claims of work not verified; Instructions that did not stick; Changes without approval Waste
Styleguide failure and unreliable cost accounting Claude Haiku 4.5 Opaque token and context usage; Unsupported specifics; Claims of work not verified; Changes without approval; Instructions that did not stick Waste
Styleguide committed with missing charts Claude Haiku 4.5 Changes without approval; Claims of work not verified; Instructions that did not stick Waste
Confident answers from skimmed primary documents Claude Opus 4.8 Misread primary records; Instructions that did not stick; Unsupported specifics Waste
Missing edits repeatedly reported as verified GPT-5.6 Sol Claims of work not verified; Instructions that did not stick; Procedural detours Waste
Drive cleanup stalled by untested capability denials Gemini Procedural detours; Hidden or misreported session state; Instructions that did not stick Waste
Speculation displaced a review of primary records GPT-5.5 Fabricated uncertainty; Misread primary records; Instructions that did not stick Waste
A researchable claim challenged before checking the literature GPT-5.6 Sol Skipped or shallow source checks; Fabricated uncertainty Waste
Primary records misread despite repeated corrections GPT-5.6 Sol Misread primary records; Fabricated uncertainty; Instructions that did not stick; Procedural detours Waste
Unverified formatting fixes delayed a usable download GPT-5.6 Sol Claims of work not verified; Procedural detours; Instructions that did not stick Waste
An unneeded caveat derailed an archival search GPT-5.6 Sol Fabricated uncertainty; Procedural detours Waste
Unexplained spending cap and silent credits Claude Billing without clear consent Friction
Unreliable word-count verification Claude Unsupported specifics Waste
An unsourced paraphrase became invented testimony Claude incognito (serving model unconfirmed) Unsupported specifics Friction
Billing research errors survived repeated corrections Claude Unsupported specifics; Claims of work not verified; Instructions that did not stick; Misread primary records Waste
Context and token usage without live counters ChatGPT Opaque token and context usage Friction
Context and token usage without live counters (revised) ChatGPT Opaque token and context usage Friction
Context and token usage: the abandoned retrieval task ChatGPT Procedural detours; Opaque token and context usage Waste
Invented usage display and opaque billing metrics Claude Unsupported specifics; Opaque token and context usage; Billing without clear consent Waste
A session memory limit turned into a platform claim ChatGPT Hidden or misreported session state Friction
Benchmark completion claimed without verification ChatGPT Claims of work not verified; Procedural detours; Instructions that did not stick Waste
Fabricated uncertainty in a sensitive disclosure ChatGPT Fabricated uncertainty; Instructions that did not stick Waste
Wrong arithmetic repeated in public advice ChatGPT Unsupported specifics; Fabricated uncertainty; Procedural detours; Instructions that did not stick Harm

How to read these counts

Modes were grouped from behaviors described in the reports. The first tag is the main failure; additional tags mark other behaviors the report documents. Each public Markdown file counts once, even when several files cover the same incident or are revisions.

Severity is a judgment about what the report says happened: harm means a wrong answer reached a real decision, such as the choice to publish advice based on it; waste means time lost or work redone; friction means annoyance only. Potential harm is not counted as an observed decision impact.

Prior logs are marked only when the report records both the instruction to consult earlier failure logs and the repeated failure. An unmarked report may simply lack the evidence. Provider counts use the system responsible for the reported failure, when identifiable. Model labels are taken from the reports, not independently verified.

This is one person’s collection of failures, many written by the models themselves. There’s no denominator for how often each provider was used, so these are counts, not failure rates or a benchmark.