Connections with your AI agent

Why the Connection is a separate object

The test (what to test) and the connection (what to test against) are separate, reusable pieces:

• The same Test Plan can run against different connections without touching the plan — for example, the same TP against your AI agent in staging and production.
• The same connection serves N test plans in the same project.
• When you click Execute, the platform asks you to choose which connection to use for that specific run.

How to create a connection

Go to Configuration → AI Agents → Add Connection.

The form asks for:

• Name — identifiable name (e.g., "AI agent prod", "Support staging").
• Base URL — the endpoint or site of the AI agent.
• Protocol — HTTP or Browser (Playwright). Defines the rest of the form's fields.
• Environment — Production / Staging / Dev. Identification only.
• Description — optional.

Browser protocol (Playwright)

For AI agents embedded in a web page. ArtificialQA runs Chromium in headless mode on its cloud workers and simulates user actions against your site. You don't open or see Chromium — all rendering happens server-side.

Available on every plan (including Free).

Browser Configuration

• Chat URL (required) — the exact URL where the chat is.
• Login URL (optional) — if the chat requires login and lives at a different URL than the chat itself.

Login Steps

JSON array with the sequence of steps to authenticate before reaching the chat. Supported actions: fill, click, wait, navigate. For example: enter username, enter password, click "Sign in".

Post-Navigation Steps

JSON array with steps that run after the chat page loads. Typical example: click a specific tab, open the bot panel, etc.

Element Selectors

CSS selectors for the key chat elements:

• Chat Input — where the user types.
• Send Button — the send button.
• Response Area — where the bot's response appears.
• Response Complete Indicator — an element that only appears when the bot is done answering (e.g., a "Copy" button). Useful for capturing responses via clipboard when the bot streams.

These selectors are optional — if you leave them empty, ArtificialQA uses AI auto-detection to identify them. Only fill them in manually if auto-detection isn't working on your site.

AI Locator Provider

The LLM the platform uses to auto-detect selectors when CSS hints fail. If left empty, the system-default provider is used.

Timeouts

• Response Wait (ms) — how long to wait for the bot's response. Default: 30,000 ms.
• Navigation Timeout (ms) — how long to wait for the chat page to load. Default: 30,000 ms.

Toggles

• Headless mode — the browser runs without a visible window (faster). On by default. Execution happens on our cloud workers, so as a user you don't see a difference — the toggle exists for internal configuration.
• Keep chat session — controls whether the page persists between test cases or reloads.
  • On: keeps the session across multiple cases. Useful when the AI agent stores context and you don't want to lose it.
  • Off: reloads the page before each test case (clean state). Useful when you want each case to start fresh, with no contamination from the previous one.
• Navigation mode: Automatic — Playwright navigates and logs in by itself, following the defined Login Steps. This is the currently supported mode. (A "Manual" mode is planned in the form but not yet enabled in this version.)

HTTP/API protocol (Pro or Enterprise plan)

For AI agents that expose their own endpoint. The cleanest, most efficient option when you have API access. It does not use Login Steps — authentication goes in the request header.

Basic data

• Base URL — the agent endpoint.
• Method — GET or POST.

Authentication

The form offers 4 options:

• Bearer Token — the most common format. You paste the token and the platform sends Authorization: Bearer <token>.
• API Key — key in header or query parameter. You configure the field name and value.
• Custom Headers — for non-standard authentication schemes. You define arbitrary key/value pairs.
• None — for open endpoints with no authentication.

Message Configuration

• Request body template (JSON) — the body of the request you send the AI agent. The platform ships an OpenAI-style template (model + messages[]) that you can adjust.
• Input field path — the JSON path where the user's text is injected (e.g., messages[0].content).
• Output field path — the path where the bot's response is extracted (e.g., choices[0].message.content).

Conversation Mode (for multi-turn cases)

Defines how context is maintained between turns in a conversation:

• Message History — every turn sends the entire conversation history inside the request body. OpenAI-style.
• Conversation ID — the AI agent maintains the session on its side and the platform sends the ID so it remembers the thread.
• None — every turn is sent as independent (useful for stateless AI agents).

If you pick Message History, the form asks for the History field path (where the message array goes in the request body, e.g., messages).

Test Connection

Once the connection is saved, use the Test Connection button to validate it. The platform sends a probe message and shows you what was sent, what was received, and how long it took. If it fails, you'll see the exact error:

• Timeout — the AI agent didn't respond within the configured time.
• 401 / 403 — invalid or expired credentials (HTTP).
• Selector not found (Browser) — selectors don't match in the DOM. Try with auto-detection (leave the fields blank) so the AI identifies them.
• Invalid output path (HTTP) — the response doesn't have the field where you expected the answer.

Multiple environments of the same AI agent

Common pattern: you have your AI agent in Dev, Staging, and Production. You create 3 connections (one per environment) using the Environment field. Then you run the same Test Plan against any of them by picking the connection at Execute time.

Common errors

• "Test Connection fails with 401." Verify the token / API key is correct and not expired.
• "The AI agent answers, but the platform says it didn't find a response." Check the Output field path — the actual response may be in a different field.
• "Selector not found in Browser." The DOM changed, or your site has a layout that auto-detection doesn't catch. Manually fill in the Chat Input / Send Button / Response Area selectors.
• "The connection works on its own but fails during a Run." Probably the AI agent is rate-limiting or your session expired between TCs. Try adjusting the Login Steps or using "Keep chat session: OFF" to refresh between cases.

Next step

Once your connection is configured and validated with Test Connection, you can execute Test Plans using it.