Responsible AI
Responsible AI: what it is and how to measure it in practice
Natalia Nario · June 30, 2026
Generate the technical evidence that ISO 42001, the EU AI Act and NIST AI RMF ask for: auditable scores for your AI agents, no code.
AI governance frameworks ask you to demonstrate that the system is accurate, that it does not discriminate and that it is overseen by people. Demonstrate, not claim. And that takes measurement: numbers, dates and a record that someone external can review.
65% vs 25%
of organizations find bias in their AI agents, but only 25% audit for it actively. Three out of four are not measuring what they already know is failing.
MIT Sloan Management Review
37.65%
of responses from leading models contained some form of bias, according to an academic benchmark. In sensitive decisions, that is direct risk.
BEATS benchmark (arXiv, 2025)
Audit
An auditor does not evaluate your intentions: they ask for traceable records, fairness tests across groups, pass thresholds and technical documentation.
ISO 42001 · EU AI Act · NIST AI RMF
You connect your bot or your agent (by URL or by API, without writing code) and evaluate it with more than 15 AI evaluators, each calibrated for one quality dimension. Every run leaves a record.
It compares the responses of the system across groups and reports the disparities it detects. It measures bias where it becomes observable: in what the system actually answers.
Who ran it, when, on which version and with what score. An immutable record per run, which is exactly what an auditor asks to see.
What was tested, with which criteria and with what result, in a format you attach to your technical documentation.
Handoff to a person is an evaluated dimension: we measure whether your agent escalates when it should: an explicit request, a sensitive case or something outside its scope.
A real screenshot of the platform; the data comes from a test workspace.
ISO 42001, the EU AI Act and NIST AI RMF are not alternatives to choose between: they are layers that overlap, and almost any large organization operates under several at once. This is what each one asks for in terms of quality and audit, and where ArtificialQA contributes.
The globe cycles through the frameworks on its own. Pick one to stay on it; the detail appears below.
A formal, certifiable AI management system: impact assessment, defined controls, a person in the decision loop and continuous monitoring, all verifiable by an external auditor.
It is moving from differentiator to requirement: more and more procurement teams demand to see evidence of a formal AI governance system before signing.
It is a law, with fines of up to 35 million euros or 7% of global turnover. If your system is high risk (credit and essential services, employment, health, biometrics, critical infrastructure, education, migration or justice) the obligations of Articles 9 to 15 apply: risk management, data governance, technical documentation, traceable records, transparency, human oversight and adequate levels of accuracy and robustness.
The high-risk obligations have August 2026 as their reference date, with an open discussion about possible timeline adjustments.
Several of those obligations are not met with a document: you have to demonstrate behavior. The most direct fit is in Art. 14 (human oversight) and in Art. 11-12 (documentation and record-keeping).
For the rest, the contribution is partial or supporting: the detail, article by article, is below.
| Art. | What it requires | Evidence you get | Fit |
|---|---|---|---|
| Art. 9 Risk management |
Identify, assess and mitigate risks continuously across the lifecycle. | A test history per identified risk; results by dimension over time; regression tests after every change. | Partial |
| Art. 10 Data and non-discrimination |
Training and test data that are relevant, representative and examined for possible bias. | Fairness tests comparing outcomes across groups. We do not govern your training data, but we do measure bias in the responses. | Partial |
| Art. 11 Technical documentation |
Produce and maintain technical documentation that demonstrates compliance. | Exportable evaluation reports: what was tested, how, with which criteria and with what result, versioned. | Direct |
| Art. 12 Record-keeping (logs) |
Automatic recording of events across the lifecycle of the system. | An immutable run history, with a record per run and traceability of which version was evaluated and when. | Direct |
| Art. 13 Transparency |
The system must be transparent enough for whoever deploys it to interpret its output. | Tests showing that your agent states its limits and does not assert with confidence what it does not know. | Supporting |
| Art. 14 Human oversight |
The system must allow oversight by people, with the ability to intervene. | The rate of correct handoff to a human and documented escalation cases. | Direct |
| Art. 15 Accuracy and robustness |
Adequate levels of accuracy and robustness, sustained consistently over time. | Accuracy and consistency metrics, plus continuous monitoring to detect degradation. We do not cover the cybersecurity part. | Partial |
Formally it requires nothing: it is voluntary. In practice it became the expected standard in the United States and the base that state AI laws build on (Colorado, Texas). Adopting it is also a defense argument if something goes wrong.
It organizes the work into four functions: govern, map the risks, measure them and manage them.
Of the four functions, "Measure" is literally what an evaluation platform does. It is the one most organizations document and fewest execute, because it demands real instrumentation.
Brazil leads the way with bill PL 2338/2023: a risk-based regime, with obligations of risk assessment, bias detection and documentation for high-risk systems, and fines of up to 2% of local turnover. Oversight is sectoral: the Central Bank for financial AI, the ANS for health.
Chile, Mexico, Colombia and Peru have advanced bills, all with obligations around evaluation, data quality and bias detection. And the EU AI Act is becoming the reference: aligning with it opens Europe to you and puts you ahead of local regulation.
Brazil regulates banking, health and insurance first. Right there, an agent that invents a rate or fails to hand off to a doctor is no longer just a quality problem: it is a regulatory one.
They are also the sectors regulation reaches first: the ones that manage money, health or the rights of people.
An agent that invents a rate, a term or a balance generates a complaint and, if a credit decision was involved, a non-discrimination problem. High risk under Annex III of the EU AI Act and under the watch of the Central Bank in Brazil.
What is critical is not just accuracy: it is the safe handoff. That the agent recognizes an alarm symptom and passes to a professional. That is exactly what we measure as human oversight.
Citizen services with fairness, transparency and accountability standards more demanding than in the private sector, and with the obligation to explain every decision.
Answer four questions about your system and find out, as guidance, which category you would fall into and which quality obligations you would have to demonstrate. Under a minute and no data required.
For guidance only, not legal advice. Based on Regulation (EU) 2024/1689.
No. ArtificialQA helps you generate the technical evidence that the certification process requires: quality measurements, an auditable history of every run and exportable reports. Certification is issued by an accredited body after auditing your AI management system, and there are steps (policies, roles, impact assessment) that are outside our scope.
It provides technical quality evidence for several high-risk obligations, with the most direct fit in Art. 14 (human oversight) and Art. 11-12 (documentation and record-keeping). It does not replace the conformity assessment and it does not constitute legal advice: the classification of your system and the complete file are managed by your organization with legal advisors.
No. You connect your bot or your agent by URL or by API and design the tests from the interface. It is built to be operated by quality or compliance teams, not just developers.
With a dedicated evaluator that compares the responses of the system across groups and reports the disparities it detects. We measure bias in the outcomes, which is where the problem becomes observable and auditable, not in the training data.
Every run is kept in an immutable history with a record of what was evaluated and when, and the reports are exportable. ArtificialQA also calibrates its AI evaluators against expert judgment (it verifies that the scores match what a specialist would say) and is model and vendor agnostic: an independent verification carries more weight than the self-assessment of whoever built the system.
In 30 minutes we connect your bot or your agent and show you what gets measured, what gets recorded and how the evidence is exported.
Ideas, guides and best practices on testing and quality for AI.
Notice. This material is informational and does not constitute legal or regulatory advice. The fit described between ArtificialQA and each framework is technical and functional, not a legal opinion. ArtificialQA does not certify compliance with the EU AI Act, ISO/IEC 42001 or any other standard, and does not replace the conformity assessment: it generates technical quality evidence that feeds part of those obligations. Each organization must validate its own with legal advisors. Based on Regulation (EU) 2024/1689, ISO/IEC 42001 and the NIST AI RMF; timelines and details may change, check the current status. Third-party data is cited with its source; product figures are illustrative. July 2026.