# Agent skill (/docs/agent-skill) Agentiqa's chat experience is **the agent proposes, you curate**: it explores your app, drafts a test plan, and you approve it before anything is saved or run. The **agent skill** brings that same loop to the command line, with **your coding agent** (Claude Code and compatible agents) acting as the coordinator. You stay the QA lead — you review the draft in chat and decide when it gets saved and run. This is the *authoring* side of the CLI. Once a plan is saved, gating it in a pipeline is the [CI lane](/docs/ci)'s job: the plan you author here is exactly what `agentiqa run --plan-id` replays and gates in CI. ## What it is (and what it isn't) [#what-it-is-and-what-it-isnt] The skill is a compact instruction set — a `SKILL.md` plus bundled reference files — that teaches an AI coding agent how to drive Agentiqa: which surface to pick, the `agentiqa` CLI verbs, the plan-authoring loop, the GitHub Action, the JSON envelope, and the exit-code contract. It uses progressive disclosure and delegates to this site's [`/llms-full.txt`](/llms-full.txt) for anything not bundled. It ships as part of the **Agentiqa plugin** ([`Agentiqa/agentiqa-plugin`](https://github.com/Agentiqa/agentiqa-plugin)), which contains two skills: * **`agentiqa-test`** — runs Agentiqa against your app from inside the coding agent (explore, report bugs with reproduction steps and media); * **`agentiqa`** (this skill) — how to drive Agentiqa: the CLI verbs and selectors, the **plan-authoring loop** (explore → review → save → run → read → revise), the GitHub Action, service keys, the JSON envelope, and exit codes. It is **not**: * **an MCP server** — it exposes no tools and runs no process; * **the Agentiqa product** — the skill contains only instructions. The actual testing is done by the tools it points at: the `agentiqa` **CLI** (published on npm) and the `agentiqa/qa-action` **GitHub Action**. ## Install [#install] Claude Code: ``` /plugin marketplace add Agentiqa/agentiqa-plugin /plugin install agentiqa@agentiqa ``` Cursor: `/add-plugin https://github.com/Agentiqa/agentiqa-plugin` · Codex CLI: `codex plugin marketplace add Agentiqa/agentiqa-plugin` The agent loads the skill on demand when you ask it to test an app with Agentiqa, author or run a test plan, or parse an Agentiqa result. No configuration is required. ## Authentication [#authentication] The authoring loop runs against your account, so it needs auth. Two options: * **`agentiqa login`** (interactive) — enough for every verb, `run` included. Pick the project with `agentiqa project use `, `--project`, or `AGENTIQA_PROJECT_ID`; see [Choosing a project](/docs/cli#choosing-a-project). This is the route to use for a project a teammate **shared** with your organization, because only a project's owner can mint a service key for it. * **A project-scoped service key** (unattended / CI) — create one in the web or desktop app (**Project Settings → CLI Service Keys**), then expose it to your coding agent's shell as `AGENTIQA_SERVICE_KEY` — never inline it on the command line or paste it into chat. See [CLI Service Keys](/docs/guides/cli-service-keys). One honest limit: **a service key is scoped to a single project.** Every `plan list` / `get` / `save` and every `run` under that key reads and writes that one project's plans, and `--project` cannot override it. To work across projects under one login, use `agentiqa project use` / `--project` instead of a key. ## On a shared project [#on-a-shared-project] A project [shared with your organization](/docs/guides/team-projects) works here like any other: after `agentiqa login`, the plans you save and the runs you start are the team's, and each carries **your** name in the project's history — the app shows who ran what and who last edited each plan. Plans that sign in use your own credential when you have one, and otherwise a [shared account](/docs/guides/shared-accounts) the team's admin provisioned. ## Environments [#environments] By default the CLI and skill target Agentiqa **production** (`agentiqa.com`) — no configuration needed. If your organization runs a non-default or self-hosted Agentiqa, set `AGENTIQA_API_URL` to that environment's base URL; the hosted engine is derived from it automatically, so you never pass `--engine` unless you self-host the engine itself. ## The loop [#the-loop] Your coding agent runs the same curation loop the chat coordinator does — it just uses the CLI verbs to do it: 1. **Explore** — the agent runs `agentiqa explore` against your target. The engine proposes a draft plan (the `--json` envelope's `testPlan`). 2. **Review** — the agent presents the draft to you *in chat*: the title, and each numbered step with its type and criteria. Nothing is saved yet. 3. **Approve** — you approve the exact plan shown. The agent treats only your explicit approval as consent to save — never the original request, silence, or `--auto-approve` (that flag only clears exploration's runtime checkpoints). 4. **Save** — on approval the agent pipes the plan JSON to `agentiqa plan save`, which mints a `tp_…` id (or edits in place when the JSON already carries one). 5. **Run** — the agent runs the saved plan by id and reports the verdict, summary, and run link. 6. **Read & revise** — `agentiqa runs get` reads the verdict history; to change the plan the agent starts from `agentiqa plan get`, edits the prose and structure, and loops back to review. A realistic session: ```text you > Explore the checkout flow on the app and draft a plan. agent > $ agentiqa explore "Test the checkout flow" \ --url https://example.com --auto-approve --json Draft plan "Checkout flow" — 4 steps: 1. [action] Add an item to the cart 2. [assertion] Cart badge shows 1 item 3. [action] Complete checkout with a test card 4. [assertion] Confirmation page shows an order number Save this plan? (I won't save until you say so.) you > Yes, save it. agent > $ agentiqa plan save --file - --json # approved plan JSON on stdin Saved plan tp_9f3c… "Checkout flow" (0 lint warnings) $ agentiqa run --plan-id tp_9f3c… --json PASSED · Checkout flow · 42s run: https://web.agentiqa.com/projects/proj_…/test-plans-v2/tp_9f3c…/history/run_… you > Show the last few runs. agent > $ agentiqa runs get tp_9f3c… --limit 5 --json tp_9f3c… "Checkout flow" — 3 runs: passed, passed, failed ``` **Criteria stay engine-authored.** The engine writes each step's machine-checkable criteria — the expected value, match type, and grounding. You and the agent curate the prose and structure (title, labels, step wording, ordering) but never hand-write those typed fields. When a criterion's *meaning* needs to change, the agent re-explores rather than editing the assertion by hand. ## The plan id is the shared handle [#the-plan-id-is-the-shared-handle] Plans are **shared account state**, not a per-surface artifact. The `tp_…` id your agent saves opens in the web app immediately, and a plan authored in the web app is `agentiqa plan get`-able by id from your agent. Hand an id in either direction — you never copy a whole plan between surfaces. ## Labels [#labels] Labels group a project's plans so CI can run a subset: `agentiqa run --label-ids a,b` runs every plan carrying any of the given labels. Labels are opaque `lbl_…` ids — your agent discovers them with `agentiqa labels list` (and can mint one with `agentiqa labels create ` when none fits), then attaches one by including it in a plan's `labels` array when it saves. A freshly created plan has no labels, so it runs fine by id but is skipped by a label-selected CI run until you attach the right label. An unknown id comes back as an `UNKNOWN_LABEL_IDS` warning on save — a failed attach to fix and re-save. See [Labels](/docs/guides/labels) for the full picture. ## Handoff to CI [#handoff-to-ci] The plan you author here is the same plan a pipeline gates on. Once it is saved, wire it in with `agentiqa run --plan-id tp_…` (or a label selector) and gate on the exit code — see [CI Integration](/docs/ci) for the exit-code contract and JSON envelope, and the [GitHub Action](/docs/github-action) for the turnkey wrapper. Authoring is interactive and curated; CI is headless and deterministic — same plans, same engine. ## Consuming the docs directly [#consuming-the-docs-directly] Any agent (with or without the skill) can read these docs as plain text: * [`/llms.txt`](/llms.txt) — a machine-readable index of every page. * [`/llms-full.txt`](/llms-full.txt) — the entire docs corpus as one document. * Append `.md` to any docs URL for that page as markdown (e.g. [`/docs/ci.md`](/docs/ci.md)). Because the CLI and Action references are generated from the code and the prose is audited nightly, the skill delegates to `/llms-full.txt` for freshness — when the bundled references and the live docs disagree, the live docs win. # CI Integration (/docs/ci) Whether you use the [GitHub Action](/docs/github-action) or run the [CLI](/docs/cli) directly, `agentiqa run` exposes a **governed, versioned contract** for CI: a fixed exit-code model and a machine-readable JSON envelope. To choose *which* plans a pipeline runs and *how* they execute — a labeled subset, sequential or parallel — see [Labels](/docs/guides/labels) and [Parallel runs](/docs/cli#parallel-vs-sequential). This page covers how to **gate** on and **parse** whatever you run. ## Exit codes [#exit-codes] The CLI exits with a status that gates your job automatically. It's a fixed, governed contract, so you can rely on it: | Code | Meaning | | ---- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `0` | Success — all selected plans passed (or there was nothing to run). | | `1` | **Plan failure** — plans executed, at least one failed. A real product-quality signal. | | `2` | **Usage / configuration error** — bad flags, not authenticated, a selector that matched no plans, **a quota / plan-limit block** (account state — retrying won't help), or a batch with **skipped plans** (see below). Something that had to run could not. | | `3` | **Infra / runtime error** — engine unreachable, an auth failure, or an unexpected error. Nothing reached a verdict, so it's safe for CI to retry. | The `1` vs `3` split is deliberate and stable: `1` is a genuine test failure you should investigate; `3` is a transient/infra problem that is safe to retry. A gate can retry on `3` without masking regressions: **A plan that cannot sign in is skipped, not fatal to the batch (since CLI v1.1.55).** If a plan's sign-in step declares a credential you have not stored, that one plan is skipped and every other plan in the batch still runs. The skipped plans are named at the end of the output and the CLI exits `2` — so the job stays red and you keep the results of everything that could run. A plain retry cannot clear it: store the named credential in Project settings → Project credentials and run again. Earlier versions refused the whole invocation, so one such plan meant the batch tested nothing. **An automated run draws on the same shared accounts as people.** If the project uses [shared accounts](/docs/guides/shared-accounts), each running plan leases one account of the role it signs in as, and returns it when it finishes. A scheduled job and a teammate can therefore run at the same time — as long as the role has enough accounts. When they are all busy, the run is refused up front, naming the role. Give a regularly-scheduled job enough headroom (one account per plan you run concurrently, plus what your team uses) rather than letting the nightly compete with the working day. **A lost connection is not automatically exit `3` (since CLI v1.1.49).** A dropped connection says nothing about whether the run finished — the engine can record the final result at the same instant the connection breaks. When a plan's connection ends before the result arrives, the CLI reads that run's own record and, if a final result was already recorded, reports **that** result (exit `0` or `1`). Only a run with no recorded final state stays retryable (exit `3`), so the retry loop below re-runs plans that genuinely produced nothing — never ones that already finished. Earlier CLI versions classified every persistent disconnect as exit `3`. ```yaml - name: Run test plans (retry only on infra/runtime errors) run: | for attempt in 1 2 3; do npx -y agentiqa@latest run --engine https://engine.agentiqa.com code=$? # 0 = pass, 1 = plan failure (do NOT retry), 2 = usage error (do NOT retry) [ "$code" -ne 3 ] && exit "$code" echo "Infra/runtime error (exit 3) on attempt $attempt — retrying…" sleep 15 done exit 3 ``` ## JSON envelope [#json-envelope] Add `--json` (or set `AG_OUTPUT=json`) to emit exactly **one JSON document on stdout**. Every document carries `"schemaVersion": 1` at the top level; a breaking change to the shape bumps that number. * **All logs go to stderr.** stdout is only the JSON document — no banners, no progress, no ANSI color. `stdout | jq` is always safe. * **The exit code is independent of the envelope** — always branch on the exit code for pass/fail; use the JSON for detail. **Success** (`ok: true`): ```json { "ok": true, "schemaVersion": 1, "outcome": "passed", "plans": [ { "title": "Checkout flow", "outcome": "passed", "durationSec": 42, "exitCode": 0, "runUrl": "https://web.agentiqa.com/projects/proj_…/test-plans-v2/tp_…/history/run_…", "videoUrl": "https://assets.agentiqa.com/e2e-videos/run-…/checkout-flow.mp4" }, { "title": "Login", "outcome": "failed", "durationSec": 18, "exitCode": 1, "summary": "…" } ] } ``` `outcome` is `"passed"` only when every plan passed, otherwise `"failed"`. Each `plans[]` entry carries a per-plan `outcome`, `durationSec`, `exitCode`, and an optional `summary`. When artifacts are captured, an entry may also carry `runUrl`, `videoUrl`, `videoPath`, and `artifactDir` — each present **only** when available (omitted, never `null`, so `schemaVersion` stays `1`). `runUrl` is the deep link to the run in the Agentiqa web app. It is set for every cloud run, and the CLI also prints it to stderr as `[] Run: `. Anyone you send it to opens it with their own Agentiqa login, as a member of the run owner's organization — there are no public, login-free run links. **Failure** (`ok: false`) — emitted for usage errors and thrown infra/runtime errors: ```json { "ok": false, "schemaVersion": 1, "error": { "code": "run_error", "message": "…" } } ``` ## Gate on the exit code, extract detail with jq [#gate-on-the-exit-code-extract-detail-with-jq] ```yaml - name: Run test plans (JSON) run: | npx -y agentiqa@latest run --engine https://engine.agentiqa.com --json > result.json code=$? jq -r '.plans[] | "\(.outcome)\t\(.title)"' result.json || cat result.json exit "$code" ``` Because logs are on stderr, `> result.json` captures only the envelope; the human-readable run log still streams to the console. ## Action outcome buckets [#action-outcome-buckets] The [GitHub Action](/docs/github-action) maps these exits to `outcome` buckets for gating: `0` (with a valid envelope and ≥1 plan) → `passed`; `0` with zero plans or no valid envelope → `config-error`; `1` → `plan-failure`; `2`/unknown → `config-error`; `3` → `infra-error`. `fail-on: plan-failure` (default) fails on plan failures and config errors and swallows retryable infra errors; `fail-on: any` gates on anything nonzero. # Desktop App (/docs/desktop-app) The Agentiqa desktop app is an Electron application with a **built-in headed browser**. It has the complete capability set and lets you watch the agent drive the browser in real time. It can test both public URLs and apps running on `localhost`. ## Install [#install] The desktop app is distributed as a signed build for macOS (and other desktop platforms). Download the latest release from [agentiqa.com](https://agentiqa.com), install it, and sign in with your account. {/* verify: confirm the user-facing desktop download page/link and the supported OS list (build+publish flow is release-desktop-*.yml → releases.agentiqa.com). */} Releases are published to a manifest at `releases.agentiqa.com`; the app checks it and shows an in-app update prompt when a newer build is available. ## Your first test [#your-first-test] 1. **Create a project** with your app's URL (a `http://localhost:…` URL works here). 2. **Ask the Assistant** to test it. The built-in browser opens and the agent begins exploring. 3. **Curate** the scope, plan, and findings checkpoints. The feature set is shared with the web app — the [feature guides](/docs/guides/projects) apply to both surfaces. ## Why desktop [#why-desktop] The desktop app adds capabilities the browser SPA cannot provide because it runs locally: browser-extension / wallet (web3) testing, local file-system access, and native OS notifications. See [Desktop vs Web](/docs/desktop-vs-web) for the full matrix. # Desktop vs Web (/docs/desktop-vs-web) The web app and the desktop app give you the **same features** — projects, the Assistant, test plans, runs, issues, the dashboard, and settings all work the same way in both. The difference comes down to a few capabilities that need to run on your own machine, so they're available in the **desktop app** but not in the browser. | What you can do | Desktop app | Web app | | ----------------------------------------------------------------- | ----------- | ------- | | Test browser-extension / wallet flows (e.g. Metamask, web3 dApps) | Yes | No | | Save run video and artifacts to a folder on your computer | Yes | No | | Get native desktop notifications (e.g. when a long run finishes) | Yes | No | | Native window controls | Yes | No | | Test an app running on `http://localhost` | Yes | No | ## Which one should I use? [#which-one-should-i-use] * **Use the web app** to try Agentiqa with nothing to install and to test a public URL — the browser runs in Agentiqa's cloud. * **Use the desktop app** when you need to test a site on `http://localhost`, test a browser extension or crypto wallet, or save run artifacts to your computer. * **Use the [CLI](/docs/cli)** to run tests locally from a terminal or in CI. Whichever you pick, everything in the [feature guides](/docs/guides/projects) works the same. > **A note on mobile:** Agentiqa tests **web apps**. Mobile-app testing was retired > and is no longer supported. {/* verify: mobileAutomation flag is still true for desktop in PlatformCapabilities but the product dropped mobile 2026-07-01 — kept out of the user table intentionally. */} # Introduction (/docs) Agentiqa is an AI-powered testing platform for **web applications**. An LLM agent navigates a real browser, discovers what matters, executes test plans, and reports issues with screenshots and reproduction steps. ## The model: the agent proposes, you curate [#the-model-the-agent-proposes-you-curate] You provide a URL. The agent provides everything else — what to test, why it matters, how to test it, and what looks broken. Your role is to approve, reject, or redirect at three checkpoints: * **Scope** — the areas the agent discovered, ranked by risk. Default: approve all. * **Plan** — how each area will be tested. Default: approve all. * **Findings** — bugs with evidence and repro steps. The only step that always needs your review. In headless contexts (CLI, CI, scheduled runs) every checkpoint auto-approves and the agent uses its own judgment — same engine, same quality, no human in the loop. ## Where you run it [#where-you-run-it] | Surface | Best for | Guide | | ----------------- | -------------------------------------------------------------- | ------------------------------------ | | **Web app** | Trying Agentiqa in the browser, cloud execution | [Web App](/docs/web-app) | | **Desktop app** | The full experience with a built-in headed browser | [Desktop App](/docs/desktop-app) | | **CLI** | Local runs and scripting (`agentiqa explore` / `agentiqa run`) | [CLI](/docs/cli) | | **Coding agent** | Authoring and running plans from Claude Code, Cursor, or Codex | [Agent skill](/docs/agent-skill) | | **GitHub Action** | Running saved plans in CI and gating on the result | [GitHub Action](/docs/github-action) | The web and desktop apps share the same feature set — the difference is a small set of platform capabilities. See [Desktop vs Web](/docs/desktop-vs-web). ## Next steps [#next-steps] * New here? Start with the [Quickstart](/docs/quickstart). * Automating in CI? Read [CI Integration](/docs/ci) for the exit-code contract and JSON envelope. * For agents: this site serves [`/llms.txt`](/llms.txt) and [`/llms-full.txt`](/llms-full.txt), and every page has a `.md` variant (append `.md` to any docs URL). # Quickstart (/docs/quickstart) Pick the surface that fits how you want to work — they all drive the same agent.