Pointing AI at your manual test cases doesn’t automate your testing. It automates a machine’s guesses about everything those cases left out.
Take a QA lead with three hundred cases sitting in the backlog tool, most written eighteen months ago by testers who’ve since left. She runs the generator. Two weeks on, the suite it hands back has checks nobody wanted, and the flows the team assumed were covered aren’t in it. The acceptance criteria never named them. The cases skipped them too, so the tool had nothing to build from.
Manual cases were written for people, not machines
The pitch is always the same: point it at what you’ve got, get automated coverage back, no scripting, no separate automation effort. But those cases were never written as machine input. Each one assumed a tester relying on her own judgement to fill gaps. A person told to check the checkout flow notices a dozen things nobody wrote down. A tool handed the same line has to invent the specifics, once, and lock them in.
The fine print is in the onboarding docs
The vendors know. It’s buried in their onboarding guidance, near enough word for word across the category: output quality tracks input quality. A story with specific acceptance criteria produces solid cases. A story with only a title produces something thin. Standardise what a good story looks like before you generate anything, they say, and review every batch before it runs. None of that is on the landing page. It turns up three scrolls down, in the implementation notes, well after the promise of a suite in minutes.
Against three hundred legacy cases, that standard turns the review into a rewrite. Years of accumulated cases lean on a competent tester reading between the lines. A machine doesn’t read between anything.
The rewrite is the real project
So the effort hasn’t gone away. It’s been shifted to the front and relabelled “automating what you already have.” Your existing cases aren’t in a fit state to automate, and getting them there is the real project. The marketing is built so you forget it exists.
Where the tool does earn its keep
I’ll give the other side this much: forcing teams to specify cases properly is good practice with or without AI. A vague test case is a liability however it runs. If the tool is what finally drags a team into better test design, that counts for something. But “this will make you rewrite your test documentation, which will take weeks” is a much harder sell than “point this at what you’ve already got.” Only one of those is on the tin. Teams that budget a few days of setup end up several sprints deep before the suite is trustworthy enough to run unsupervised. Buy the tool, by all means. Just price in the part that isn’t on the tin.
Eggplant DAI Brochure | Application Testing Solutions | Contact us
Sign up for our newsletter | Keysight
