AI Writes the Test Cases Now. The Survey Behind This Week's Headline Says Less Than the Headline Did.

What was published, and when

Applause published The State of Digital Quality in Functional Testing 2026 on 30 September 2026, from a survey it says was fielded in August 2026 among members of its uTest community and other development, QA, product, AI and data science professionals, alongside interviews with technology leaders. The UK trade press picked it up on 9 and 10 October, which is how it reached most inboxes in this sector.

The press release leads with two claims: more than 92% of respondents use AI in testing, up from 60% a year earlier, and 29% report that the number or severity of functional defects has increased, with 15% reporting both. Applause publishes no single overall respondent count; individual questions carry their own bases, roughly 186 to 242 responses, which caps how much weight any one percentage can carry.

What the table actually says

The report's own breakdown of how defects changed after AI entered the development lifecycle (n=197) reads: both number and severity decreased, 26.4%; number increased but severity decreased, 10.2%; both increased, 14.7%; number decreased but severity increased, 4.1%; no change, 13.2%; do not track this data, 19.8%.

Add together every answer in which something got worse and you get the 29% in the headline. Fair enough. But the largest single answer is the opposite one: more respondents reported defects falling on both counts (26.4%) than reported both rising (14.7%). And the second largest group, at 19.8%, do not measure it at all.

None of this makes the report wrong, and none of it is hidden — it is in Applause's own tables. It does mean the honest summary is narrower than "AI is making quality worse": a self-selected sample of testers, asked whether defects moved, split roughly three ways between better, worse and unmeasured. The actionable number is the unmeasured one.

The 92% is a lower bar than it sounds

The 92% is the complement of the 7.9% who say they use AI for no aspect of testing at all. It counts any use, anywhere: one engineer asking a model to draft a test case clears the bar for a whole organisation. Applause sets it against 59.6% from its 2025 benchmark, a striking year-on-year line, though a reader cannot assume both figures come from the same question against the same base.

The use-case table is the more specific and more interesting one. Among those using AI in testing (n=186), the top uses are creating test cases (65.1%), creating scripts for test automation (62.4%), identifying and addressing coverage gaps (48.4%), analysing test outcomes and recommending improvements (43.5%), and autonomous test execution and adaptation (36.6%).

Read that order again. Autonomous execution — the thing the agentic-ERP coverage worries about — is last. The most common use of AI in testing is writing the test cases.

On an ERP project, the scripts are the scope

In most software testing a test case is an instrument: a way of finding out whether something works. On an ERP implementation it is also close to a contract term. The agreed set of scripts is what the word "tested" refers to, and the sign-off at the end of UAT is a statement that those scripts were executed and their failures dispositioned by someone willing to put a name to it.

So "AI drafted the test cases" is not a productivity note. It is a change in who authored the scope. A model given a purchase-to-pay process will produce forty plausible scripts, better written than the ones a rushed consultant types at 6pm. What it cannot know is which of the client's three approval routes is actually used, which habit the warehouse kept from the old system, or which month-end step exists only in one person's head — and those omissions are exactly what separates an effective test case from a useless one.

The failure mode is quiet. Coverage looks complete because it is fluent. A client user reads a well-formed script, recognises nothing wrong with it, and signs — and the step that was never in it is discovered at the first real month-end. That is a different problem from a broken feature, and it is not one a replay pipeline or an automated regression pass will surface, because the automation is testing the same incomplete set.

Applause's respondents were asked about this directly: 86.1% rated human involvement in functional testing at the top of the scale, 13.4% at "somewhat important", fewer than 1% not important at all. They are testers being asked about the value of testers, so discount as you see fit.

What to change in your next cycle

Keep a named author on every script. AI drafts, a consultant owns, and a named client user confirms it matches how the work is actually done. Attribution is the control here, not a review step bolted on afterwards.

Measure defects per cycle, by severity. The 19.8% who do not track this are the report's real finding. If you cannot say how many severity-one defects cycle two produced against cycle one, you cannot know whether generated scripts helped you or flattered you.

Treat a generated script as a hypothesis about coverage. Check it against process documentation and against someone running the process, before it reaches a test plan the client will sign.

Record what was never executed. Generated scripts make suites bigger, and a bigger suite makes unexecuted scripts easier to lose. That is the specific weakness of running UAT on spreadsheets: they record results, not the absence of them.