What is generative AI testing and why it differs from traditional testing?

Generative AI testing is the set of practices for evaluating whether a system based on language models —a chatbot, an assistant, an agent— responds correctly, safely and consistently, using criteria that tolerate the same question having many valid answers. That is the fundamental difference from traditional software testing: in classic software, one input always produces the same output and the test is binary (it matches or it doesn’t). In generative AI, that assumption breaks down completely —and with it, the entire testing method the industry used for fifty years.

If your team is putting an AI agent in front of customers and still testing it as if it were ordinary software, this article explains why that doesn’t work, what changes, and how generative AI is really evaluated.

A case that sums it all up

In November 2022, Jake Moffatt went to the Air Canada website after his grandmother’s death to find out about bereavement fares. The airline’s chatbot told him he could buy the ticket and request the discount within 90 days of the flight. That was false: the actual policy required requesting it before traveling. When Moffatt filed a complaint, Air Canada refused and argued —before the tribunal— that the chatbot was “a separate legal entity” responsible for its own actions.

The Civil Resolution Tribunal of British Columbia did not accept that. In its February 2024 ruling (Moffatt v. Air Canada), it called the argument “remarkable” and held that the company is responsible for all the information on its site, whether it comes from a static page or a chatbot. Air Canada had to pay 812 Canadian dollars in damages and costs.^[1]

The figure is trivial; the principle is not. A tribunal established that a company is legally liable for what its AI tells a customer. And the question left hanging is the one that matters: how do you prove, before a launch, that an agent won’t invent a policy that doesn’t exist? With traditional testing, you can’t. Something else is needed.

Why traditional testing breaks down with generative AI

Software testing was built on one premise: determinism. Given an input, the system always produces the same output, and the test consists of comparing the result against an expected value. assert result == "expected". If it matches, it passes; if not, it fails. Simple and reliable for decades.

Language models are probabilistic by nature. The same question can generate different answers in structure, tone and even factual content. And it’s not a problem you fix by lowering the model’s “temperature” to zero: a 2025 study documented that, even with nominally deterministic settings (temperature=0), repeated queries can produce inconsistent outputs.^[2] Variability is a property of the system, not a bug to fix.

This makes exact text comparison fail in two ways at once. It rejects valid answers that don’t match word for word —“the sun is rising” versus “dawn is breaking” mean the same thing, but an assert flags them as different— and, worse still, it accepts incorrect answers that happen to match the expected text.^[3] The method is not only insufficient: it produces misleading signals in both directions.

On top of variability, the quality of an AI answer is multidimensional. An answer can be factually correct but have an inappropriate tone; grammatically perfect but semantically wrong; helpful but with a fabricated fact in the middle. There is no single “correct” to verify: there are several dimensions to measure at the same time.

The problem is not marginal: the data

You might think this only affects rare cases. It doesn’t. The World Quality Report 2025, which surveyed more than 2,000 executives in 22 countries, found that concerns about hallucinations and reliability are now among the top barriers to adopting generative AI in quality assurance, cited by 60% of respondents —a problem that barely existed two years earlier.^[4]

And model reliability is far from solved. Stanford’s 2026 AI Index found that hallucination rates across 26 leading models range from 22% to 94% depending on the test.^[5] Different benchmarks under real-world conditions report different ranges, but they all agree on the same thing: hallucinations happen often enough that no critical system can assume “the model is good” without measuring it. The number of documented incidents confirms this: Stanford’s AI Incident Database recorded 362 in 2025, up from 233 in 2024.^[6]

There is also a capabilities gap that makes everything worse. The same World Quality Report found that 50% of organizations lack AI/ML expertise, unchanged from the previous year, and that generative AI became the most in-demand skill for quality engineers (63%), ahead of the fundamentals of the craft itself.^[7] Put another way: just as a radically new form of testing appears, most teams still lack the tools and knowledge to carry it out.

How generative AI is really evaluated

If exact comparison doesn’t work, what replaces it? The answer is semantic evaluation: instead of asking “is this text identical to the expected one?”, you ask “does this answer meet the quality criterion I care about?”. And that is measured across dimensions.

These are the ones that weigh most in systems serving customers:

• Factual accuracy: is the answer correct, or does it invent data?
• Hallucinations: does it assert things not backed by the sources?
• Tone and empathy: is the register appropriate for the context?
• Completeness: does it answer everything that was asked, or leave gaps?
• Context adherence: does it stick to the real documentation and policies (exactly what the Air Canada chatbot lacked)?
• Handoff to a human: does it escalate to a person when the case requires it?
• Relevance: does it answer what was asked, without wandering off?

The technique that dominates this evaluation today is called LLM-as-a-judge: a language model is used to judge the quality of another’s answers, because a model can indeed recognize that “dawn is breaking” and “the sun is rising” are equivalent, and can assess subjective qualities such as tone.^[8] It’s powerful, but it has a weakness worth knowing: the judge is also a probabilistic model, and it can make mistakes or have biases —for example, favoring answers similar to the ones it would generate itself.^[8:1] That’s why a serious evaluation doesn’t stop at “use an AI judge”: it also calibrates that judge, checking that its scores align with the criterion of a human expert. An uncalibrated evaluator is a broken calculator: it gives you numbers, but you can’t trust them.

AI testing doesn’t end at launch

There is one last shift in mindset, and it’s the one that causes the most failures. In traditional software, once something is tested and deployed, it keeps behaving the same way. In generative AI, it doesn’t. Models degrade, context changes, providers update the models underneath, and an agent that worked at launch may start failing three months later. Most teams test thoroughly before deploying and then stop watching —and that gap, between what was tested and what keeps running in production, is where most AI failures originate.^[9]

That’s why generative AI testing is a cycle, not an event: you evaluate before launch and monitor continuously in production, to catch degradation before the customer does.

What your team needs to do it right

Summarizing what changes: you stop comparing text and start measuring quality dimensions; you stop looking for a single “correct” and accept multiple valid answers; and you stop testing once to monitor continuously. That requires tools built for AI, not adaptations of classic testing.

That is exactly what ArtificialQA does. Instead of comparing strings, it connects to your agent —by URL or API, without writing code— and subjects it to specialized evaluators that measure accuracy, hallucinations, tone, completeness, handoff to a human and more. And, in a way that’s uncommon in the market, it calibrates its own AI judges, so the results are reliable and recorded as evidence. A QA or business team can thus test an agent with the same rigor with which it always assured software quality —only adapted to the new rules of the game.

Because the lesson of the Air Canada case is clear: the AI that talks to your customers is already your responsibility. The question is not whether you’ll test it, but whether you’ll do it before or after an error reaches production.

---

Frequently asked questions

What is generative AI testing? It is the set of practices for evaluating whether a generative AI system responds correctly, safely and consistently, using criteria that accept that the same question can have several valid answers, instead of comparing against a single exact result.

Why doesn’t traditional software testing work for AI? Because traditional testing assumes determinism: same input, same output. Generative AI is probabilistic and produces answers that vary, so exact text comparison rejects valid answers and accepts incorrect ones.

Can you make an AI deterministic to test it more easily? Not entirely. Even with the temperature at zero, models can give inconsistent answers. Variability is a property of the system, so testing must be designed to tolerate it, not to eliminate it.

What is measured when testing a generative AI system? Quality dimensions such as factual accuracy, hallucinations, tone, completeness, context adherence, handoff to a human and relevance —not a single exact match.

Do you need to know how to code to test generative AI? Not necessarily. There are platforms that connect to the agent by URL or API and let QA or business teams evaluate the AI without writing code.

Is AI testing done just once? No. Models degrade and context changes, so evaluation must continue in production through monitoring, not end at launch.