AI testing in banking: how to stop your agent inventing rates or balances
In banking, an agent that invents a rate or a balance is a serious problem. How to test AI to avoid errors that cost dearly.
In banking, an incorrect AI response is not a harmless mistake: it is a promise a court can force you to keep. In 2024, Air Canada lost a case because its chatbot invented a refund policy that did not exist, and the ruling became the de facto reference that “the company owns what its bot says.”[1] If that happened to an airline over a fare of a few hundred dollars, imagine the cost when a banking assistant invents an interest rate, confirms a wrong balance, or promises a bonus the bank does not honor.
This article is for the teams putting AI in front of customers in banking and finance: what can go wrong, what has to be tested before a launch, and how to leave the evidence a regulator may request.
The failure modes that cost dearly
The U.S. consumer financial protection regulator (CFPB) has already compiled real complaints about chatbots in finance: customers trapped in loops, bots that refuse to hand off to a human, and incorrect information about fees and disputes.[2] Translated into banking operations, the most costly failure modes are six:
- Inventing policies or conditions. The bot promises a fee waiver that the system later does not honor. The customer complains, sues, or files a complaint with the regulator, or all three.[2:1]
- Getting sensitive data wrong. Confirming an incorrect balance, rate, or account statement. In a financial model, a 1% error may be a rounding; for a customer making a decision, it is a real problem.
- Failing to hand off to a human. A bot that traps the customer in a loop instead of escalating when the case requires it is, besides a bad experience, a regulatory risk signal.[2:2]
- Answering a suitability question badly. An assistant that opines on the appropriateness of a financial product can end up cited in a regulatory review.
- Disclosing data it should not. Confirming one customer’s balance or transactions to someone else, repeating a card number or a sensitive detail in the response, or exposing the bank’s internal information. This is not just a bad experience: it is a personal data breach, with regulatory and reputational consequences that can exceed those of a misquoted rate.
- Treating certain groups of customers worse. An assistant that responds differently, or less favorably, depending on the customer’s profile exposes the bank to a discrimination problem, a risk as heavily regulated in finance as data accuracy.
The common pattern: all these errors look fine in the moment. The bot responds confidently, the customer believes it, and the failure only surfaces later, when there is already a complaint. That is why it is not enough for it to “seem to work”: it has to be tested beforehand.
What a trustworthy banking agent has to meet
Turning the failure modes around, an AI assistant fit for banking should be able to demonstrate that it:[2:3]
- Responds accurately and only from bank-approved sources.
- Never invents policies, fees, or conditions.
- Hands off to a human when the customer requests it or the case requires it.
- Does not trap the customer in dead-end loops.
- Does not disclose one customer’s data to another, nor sensitive or internal bank information.
- Does not produce discriminatory or biased responses toward any group of customers.
- Produces an auditable record per conversation.
Each of those points is a measurable dimension. And there is the good news: what the regulator and common sense demand can be translated into concrete tests.
What to evaluate, translated into tests
| What has to be guaranteed | What is measured | How it is tested |
|---|---|---|
| Not inventing rates, balances, policies, or data that is not in the source | Factual accuracy, data accuracy, and hallucinations | Cases with questions about real data and conditions, verifying against the approved source |
| Not disclosing other customers’ data or sensitive or internal information | Security (sensitive data exposure) | Adversarial cases that try to induce the bot to leak data, card numbers, or internal information |
| Escalating when appropriate | Rate of correct handoff to a human | Cases designed to force escalation (sensitive queries, explicit requests) |
| Not trapping in loops | Completeness and resolution | Multi-turn conversations that verify the customer reaches an exit |
| Appropriate and professional treatment | Tone, empathy, and formality | Cases in delicate situations (complaints, payment difficulties) and in standard service |
| Not treating any group of customers worse | Bias | Cases designed to detect differences in treatment by gender, age, or other groups |
No single evaluator is enough on its own in banking. But if you have to prioritize, data security is usually the one that tolerates the least margin for error: a bot that invents a rate generates a complaint; a bot that leaks another customer’s balance generates a personal data breach, with everything that implies in regulatory terms and in trust. Together with not inventing information and always anchoring to the approved source, these are the two lines of defense that cannot be left untested.
The evidence a regulator may request
There is one dimension that in other sectors is desirable and in banking is outright mandatory: traceability. A banking assistant should produce an auditable record per conversation, and the institution should be able to demonstrate, in a review, that it systematically tested the quality of its AI before exposing it to customers.
This connects with the broader regulatory framework. Both the European regime (EU AI Act) and the emerging regulations in Latin America (in Brazil, for example, with sectoral supervision by the Central Bank for financial AI) push in the same direction: risk assessment, non-discrimination, human oversight, and documented evidence.[3] Testing is not just quality assurance; it is the raw material of conformity.
How ArtificialQA solves it
Validating all of this by hand (reading responses one by one) does not scale and does not leave defensible evidence. ArtificialQA connects to your banking assistant (by URL or by API, without writing code) and subjects it to the evaluators that matter in this vertical: factual and data accuracy, hallucinations, security (sensitive data exposure), bias, correct handoff to a human, and tone/formality. Every run is recorded in a history that serves as evidence for an audit. And its AI evaluators come calibrated out of the box, against the judgment of human experts. It is not something each bank has to build on its own, so the results withstand a regulator’s scrutiny.
The practical result: a bank’s QA, product, or compliance team can demonstrate, before go-live and continuously afterward, that its agent does not invent, does not leak data, hands off when it should, and sticks to the approved sources. Because in banking, the question is never whether it is worth testing the AI before exposing it. Air Canada already answered that question for everyone: what your AI tells a customer is what your institution will have to stand behind.
Frequently asked questions
Why is a banking chatbot that gets things wrong so risky? Because the information it gives can be legally binding for the institution. A reference case (Air Canada, 2024) established that the company is responsible for what its chatbot says, and financial regulators already compile complaints about bots that give incorrect information or fail to escalate.
What must an AI assistant in banking guarantee? Responding accurately from approved sources, not inventing policies or fees, handing off to a human when appropriate, not trapping the customer in loops, not disclosing other customers’ data or discriminating, and producing an auditable record per conversation.
How do you test that a banking agent does not leak other customers’ data? With adversarial cases designed specifically to try to induce the bot to reveal sensitive information (balances, card numbers, personal data) that does not belong in that conversation, and verifying that the agent refuses consistently.
How do you test that a banking agent hands off to a human when it should? With cases designed to force escalation (sensitive queries or explicit requests to speak with a person), measuring the rate of correct handoff.
Does AI testing help with regulatory compliance in banking? Yes. Systematic and traceable evaluation produces the evidence that frameworks like the EU AI Act and emerging sectoral regulations require. It does not replace legal counsel, but it is the foundation of conformity.
Product Manager of ArtificialQA at QAlified, with 15+ years in software testing and automation. She works at the intersection of quality and AI: designing and evaluating approaches to test non-deterministic systems and ensure their behavior in production.
Related articles
