Changelog
News, improvements, and fixes for each ArtificialQA release. We publish in reverse chronological order.
2026-05 — Current release
New Mandatory email verification on new account signup.
New Optional two-factor authentication (TOTP) for all users. Compatible with Google Authenticator, Authy, and 1Password.
New AI-powered enhanced reports on the Enterprise plan: automatic executive summary and per-evaluator textual analysis, in Spanish or English.
Improvement Notification when reaching 80% of monthly token quota.
Security External security audit performed with Nextfense; recommendations evaluated and implemented.
2026-04 — Evaluation improvements
Improvement Calibration of all 17 LLM evaluators with representative datasets per domain.
Improvement Automatic PII detection (emails, phone numbers, ID documents, credit cards) in chatbot responses.
Improvement Real-time view during Test Plan execution with progress, current case, and estimated time.
Fix Fixed timeouts on Browser connections over pages with complex login.
2026-03 — AI generation by industry
New AI test case generation, configurable by industry. 15 supported industries (general, customer support, healthcare, finance, ecommerce, travel, telecom, education, legal, hr, saas, insurance, real estate, food, safety).
New Review Queue: human approval queue for generated cases before they enter production.
New Support for conversational (multi-turn) test cases in addition to simple Q&A.
2026-02 — HTTP/API connection
New HTTP/API connection for chatbots with their own endpoint, on Pro and Enterprise plans.
New Support for Bearer Token, API Key (header or query), and Custom Headers.
Improvement Body templating and response mapping configurable from the UI.
2026-01 — PDF-exportable reports
New PDF export of the Evaluation Report (executive summary, per-evaluator performance, failed-case detail, run metadata).
Improvement Immutable per-Run snapshots: input, response, timings, screenshots, and scores stay frozen for auditing.
2025-12 — Initial public launch
New Browser connection (Playwright) for chatbots embedded in webpages.
New Deterministic asserts: exact_match, regex, contains, not_contains, json_schema, numeric_range, response_time, keyword_present, classification.
New 17 LLM evaluators to score responses across different dimensions.
New Per-organization multi-tenant isolation.
New Free, Pro, and Enterprise plans.
Roadmap requests
We're working on or evaluating including:
- Webhooks to notify Run completion.
- Native Slack and Teams integration.
- Native Jira and other ticket-tracker integration.
- Official SDKs (Python, Node.js).
- Single Sign-On (SSO) on Enterprise plans.
If any of these is a blocker for your team, write to us — the roadmap is prioritized with real feedback.