Whenever I think about where software testing is heading, I find myself coming back to vibe coding. The act of developers describing what they want to an AI, and having the tool write the code for them.

It's definitely here to stay, and it's definitely changing the way software gets built.

But it's worth asking, is it good for software testing?

In this piece, I want to look at what vibe coding is really doing to the code that reaches testing, why I don't think vibe coding itself is a problem, and what testing needs to look like to keep up.

01 / The pressureMore code, less checking

Vibe coding isn't a passing trend. AI coding tools are already part of how South African banks build software. At one bank alone, more than a thousand developers now use them. And it's easy to see why. Teams can build far more, far faster.

But that speed comes with a cost, and the numbers on it are hard to ignore. In 2025, Faros AI tracked more than 10,000 developers across 1,255 teams.

On teams that used AI heavily, developers merged 98% more pull requests than before. So, these teams were pushing through nearly twice as many changes, and those changes were also 154% bigger. More code was going in, and each piece of it was larger.

As a result, the time spent reviewing them went up by 91%. The people checking the code simply couldn't keep up with how much was coming in. In addition to that, according to a follow-up study, bugs per developer were up 54%.

And there's also the issue of the code carrying mistakes as well. In December 2025, CodeRabbit analysed 470 code changes from real open-source projects. The ones written with AI's help had about 1.7 times more issues than those written by people alone, and logic errors were about 75% more common.

More code, with more mistakes, and less human attention before it reaches testing. Looking at these issues, it's easy to believe that vibe coding is the problem.

But that's not the real issue.

02 / The bottleneckThe real issue is how we test it

The actual problem is that most teams are still testing vibe-coded software the same way they tested human code.

Traditional testing was built for a slower pace. Code arrived at the speed people could write it, and testers could keep up by writing tests for each change by hand.

That is not the current state anymore. When a team is suddenly pushing through nearly twice as many changes, and each one is bigger, the testers start falling behind. Sure, you can hire more, but that's slow and expensive, and it's hard to grow a team as fast as AI can grow the code.

The obvious solution is to get AI to do the testing too. It's the only way testing can match the pace of vibe coding. But how you use it makes all the difference.

03 / The trapThe wrong way

Allowing AI to test your code is not as simple as just copying and pasting. Every test needs a right answer to check against. When a tester writes a test, they get that answer from what they know about the business and how the system is supposed to behave. AI doesn't have that knowledge. It only knows what you give it. And if you give it your code, it tends to assume that the given code is the best approach. It always benchmarks against what it was given.

In a 2026 study presented at ISSTA (“Evaluating and Mitigating the Misguidance Effect of Buggy Code in LLM-Generated Unit Tests”, Zhao et al.), researchers gave AI models code to write tests for, some of it with bugs in. They wanted to know whether a bug in the code would trick the AI into treating it as correct.

It did. When the code had the bug, the models were almost eight times more likely to write a test that confirmed the bug as the correct behaviour.

Think about what that means. A test is supposed to be the thing that catches the bug. Instead, the AI looked at the broken code, took it as right, and wrote a test to confirm it. That test passes. And every time it runs, it tells your team the bug is exactly how the system is meant to work.

So when vibe-coded errors reach testing, a testing AI that learns from the code won't always catch the errors. Sometimes it will lock them in as the right answer, and the tests will pass.

04 / A better source of truthThe right way

The same ISSTA study also shows the way around this. When the researchers gave the models a description of what the system was supposed to do, instead of the code, the models caught more bugs.

That makes sense. A description of what the system should do gives the AI an approach that doesn't depend on the code. If the code has a bug, the test is checking it against something the bug can't change. The AI is no longer grading the code's homework against the code's own answers.

We saw the same thing when we tried it ourselves. We gave our AI-First Quality Assurance tool only the publicly available API documentation of twelve South African financial services organisations, never their code. Because every test got its right answer from what the documentation said the system should do, a bug in the code couldn't make those tests pass.

It also picked up failures the documentation implied but never stated, like what should happen when a request arrives without a login. And where it couldn't confirm something without access to the real system, it marked those tests as “waiting” instead of passing them.

That's the right way: AI that tests against what your system is supposed to do and is upfront about what it doesn't know yet.

05 / The human judgmentWhere people still come in

Even done this way, AI can only work with what it's given. It can work out that a refund should fail when the amount is invalid. What it usually can't know is exactly how much a valid refund should pay.

Say the correct refund is R90, but a bug makes the system pay R100. If nothing the AI was given says R90, it has no way of knowing R100 is wrong. It writes a test that expects R100, and the bug stays hidden.

That's where QA earns its place. Someone has to know what the system is actually meant to do, and make sure every test is held to it. As vibe coding grows, that job only gets more important.

06 / The conclusionVibe coding isn't the risk

Vibe coding has changed how code gets written, and it isn't going away. The teams that get into trouble won't be the ones whose developers use AI, but rather the ones whose testing stayed the same while the code around it changed.

The answer is AI testing, done the right way. Instead of a general-purpose AI chatbot, use a specialist AI testing tool to do the reading, the working out and the writing, at the speed your code now moves. And make sure its tests get their right answers from what your system is supposed to do, not from the code they're checking.

And most importantly, have your people decide what's correct in the end.