Runs

Trigger test runs, poll for results, and browse run history via the Smoketest API.

A run is created when you trigger a test. This page covers the full lifecycle: triggering, polling for a result, and listing run history.

The run object

JSON

{
  "id": "018ecccc-abcd-7000-8000-abc123456789",
  "workspaceId": "018e1234-abcd-7000-8000-abc123456789",
  "testId": "018eaaaa-abcd-7000-8000-abc123456789",
  "startUrl": "https://preview-acme.vercel.app/login",
  "environmentId": null,
  "environmentName": null,
  "trigger": "api",
  "status": "passed",
  "result": "passed",
  "reasoning": "The user successfully logged in. The dashboard heading 'Welcome back' was visible after sign-in.",
  "errorMessage": null,
  "stepCount": 7,
  "creditsCharged": 1,
  "startedAt": "2026-06-03T10:05:02.000Z",
  "completedAt": "2026-06-03T10:06:14.000Z",
  "createdAt": "2026-06-03T10:05:00.000Z"
}

Field	Type	Description
`id`	UUID	Unique identifier
`workspaceId`	UUID	Workspace this run belongs to
`testId`	UUID	Test that was run
`startUrl`	string \| null	Effective URL the agent opened first for this run
`environmentId`	UUID \| null	Environment used for this run
`environmentName`	string \| null	Environment name at the time of the run
`trigger`	`"on_demand"` \| `"schedule"` \| `"api"` \| `"github"` \| `"webhook"`	What initiated this run. Public API-triggered runs use `"api"`; GitHub App runs return `"github"`; deployment webhook runs return `"webhook"`.
`status`	string	Current status — see table below
`result`	`"passed"` \| `"failed"` \| null	Agent's result; `null` until the run finishes
`reasoning`	string \| null	Agent's explanation of the result
`errorMessage`	string \| null	Populated when `status = "error"`
`stepCount`	integer	Number of browser actions taken
`creditsCharged`	integer	Credits consumed by this run
`startedAt`	ISO 8601 \| null	When the agent started working
`completedAt`	ISO 8601 \| null	When the run finished
`createdAt`	ISO 8601	When the run was enqueued

Terminal run responses also include artifact URL fields when the corresponding artifact exists: videoUrl, traceUrl, and transcriptUrl. These URLs point to authenticated /v1/runs/:id/artifacts/... routes. Fetch them with the same Authorization: Bearer $SMOKETEST_API_KEY header you use for the run API.

Run statuses

Status	Meaning
`pending`	Queued, waiting for an agent to pick it up
`running`	Agent is actively executing the test
`passed`	Agent verified the expected outcome was met
`failed`	Agent determined the outcome was not met
`error`	Run encountered an unexpected error (see `errorMessage`)
`cancelled`	Run was cancelled before completing

Terminal statuses are passed, failed, error, and cancelled.

Trigger a run

Enqueues an immediate run for a test and returns the run's initial state.

POSThttps://api.smoketest.sh/v1/tests/:id/runscope: run

Path parameters

id

UUIDrequired

Test ID

Body

startUrl

stringoptional

Optional run-only start URL override. Use this for preview links or staging deployments without changing the saved test.

Returns 201 Created with the run's initial state. Use the returned id to poll GET /v1/runs/:id until the run reaches a terminal status.

JSON

{
  "id": "018ecccc-abcd-7000-8000-abc123456789",
  "startUrl": "https://preview-acme.vercel.app/",
  "status": "pending",
  "createdAt": "2026-06-03T10:05:00.000Z"
}

Shell

curl -X POST https://api.smoketest.sh/v1/tests/018eaaaa-abcd-7000-8000-abc123456789/run \
  -H "Authorization: Bearer $SMOKETEST_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"startUrl": "https://preview-acme.vercel.app"}'

List runs

Returns up to 50 runs ordered by createdAt descending.

GEThttps://api.smoketest.sh/v1/runsscope: read

Query parameters

testId

UUIDoptional

Filter to runs for a specific test

projectId

UUIDoptional

Filter to runs for all tests in a project

status

stringoptional

Filter by status

sourceId

UUIDoptional

Filter to runs generated by a Test Source

iterationId

UUIDoptional

Filter by the Sprint or Cycle captured on the Attempt

from

stringoptional

Include runs created at or after this ISO 8601 timestamp

to

stringoptional

Include runs created at or before this ISO 8601 timestamp

Shell

# Most recent 50 runs in the workspace
curl "https://api.smoketest.sh/v1/runs" \
  -H "Authorization: Bearer $SMOKETEST_API_KEY"

# Failed runs for a specific test
curl "https://api.smoketest.sh/v1/runs?testId=018eaaaa-abcd-7000-8000-abc123456789&status=failed" \
  -H "Authorization: Bearer $SMOKETEST_API_KEY"

Sample response

JSON

[
  {
    "id": "018ecccc-abcd-7000-8000-abc123456789",
    "workspaceId": "018e1234-abcd-7000-8000-abc123456789",
    "testId": "018eaaaa-abcd-7000-8000-abc123456789",
    "startUrl": "https://preview-acme.vercel.app/login",
    "environmentId": null,
    "environmentName": null,
    "trigger": "api",
    "status": "failed",
    "result": "failed",
    "reasoning": "The login button was not found. The page showed a 503 error.",
    "errorMessage": null,
    "stepCount": 3,
    "creditsCharged": 1,
    "startedAt": "2026-06-03T10:05:02.000Z",
    "completedAt": "2026-06-03T10:05:45.000Z",
    "createdAt": "2026-06-03T10:05:00.000Z"
  }
]

Get a run

GEThttps://api.smoketest.sh/v1/runs/:idscope: read

Path parameters

id

UUIDrequired

Run ID

Shell

curl https://api.smoketest.sh/v1/runs/018ecccc-abcd-7000-8000-abc123456789 \
  -H "Authorization: Bearer $SMOKETEST_API_KEY"

Rerun

Creates a new run for the same test as a previous run. If the previous run used a custom startUrl, the rerun inherits it unless you provide a replacement startUrl.

POSThttps://api.smoketest.sh/v1/runs/:id/rerunscope: run

Shell

curl -X POST https://api.smoketest.sh/v1/runs/018ecccc-abcd-7000-8000-abc123456789/rerun \
  -H "Authorization: Bearer $SMOKETEST_API_KEY"

Cancel a run

Cancels a run that is still pending or running. Terminal runs are unaffected. Requires the run scope.

POSThttps://api.smoketest.sh/v1/runs/:id/cancelscope: run

Path parameters

id

UUIDrequired

Run ID

Returns 200 OK with the updated run.

Shell

curl -X POST https://api.smoketest.sh/v1/runs/018ecccc-abcd-7000-8000-abc123456789/cancel \
  -H "Authorization: Bearer $SMOKETEST_API_KEY"

Stream live events

Streams run progress as Server-Sent Events (text/event-stream) — the same event feed the dashboard live view consumes. Requires the read scope. If the run is already in a terminal state, the stream emits a single terminal event and closes. A ping heartbeat is sent every 15 seconds to keep the connection open.

During finalization, the stream can emit run.phase events such as checking_result, saving_artifacts, and preparing_result. Use the event's message field for user-facing progress while Smoketest verifies the outcome, saves the recording/trace/transcript, records usage, and prepares the terminal result.

GEThttps://api.smoketest.sh/v1/runs/:id/streamscope: read

Path parameters

id

UUIDrequired

Run ID

Shell

curl -N https://api.smoketest.sh/v1/runs/018ecccc-abcd-7000-8000-abc123456789/stream \
  -H "Authorization: Bearer $SMOKETEST_API_KEY"

For most CI use cases, polling GET /v1/runs/:id (below) is simpler and more robust than holding an SSE connection open.

Polling a run

After triggering a run, poll GET /v1/runs/:id until status reaches a terminal state.

Python

import time
import httpx

BASE = "https://api.smoketest.sh"
HEADERS = {
    "Authorization": f"Bearer {SMOKETEST_API_KEY}",
}
TERMINAL = {"passed", "failed", "error", "cancelled"}

client = httpx.Client(base_url=BASE, headers=HEADERS)

# Trigger
triggered = client.post(
    "/v1/tests/018eaaaa-abcd-7000-8000-abc123456789/run"
).raise_for_status().json()

run_id = triggered["id"]
print(f"Queued run: {run_id}")

# Poll every 5 seconds
while True:
    run = client.get(f"/v1/runs/{run_id}").raise_for_status().json()
    print(f"  status: {run['status']}")
    if run["status"] in TERMINAL:
        break
    time.sleep(5)

if run["result"] == "passed":
    print(f"✓ Passed: {run['reasoning']}")
else:
    print(f"✗ {run['status']}: {run['reasoning'] or run['errorMessage']}")
    exit(1)

CI integration

A full example that triggers all tests for a project and fails the CI job if any run does not pass.

.github/workflows/smoketest.yml

name: Smoketest

on:
  deployment_status:

jobs:
  smoketest:
    if: github.event.deployment_status.state == 'success'
    runs-on: ubuntu-latest

    steps:
      - name: Run Smoketest tests
        env:
          SMOKETEST_API_KEY: ${{ secrets.SMOKETEST_API_KEY }}
          SMOKETEST_PROJECT_ID: ${{ vars.SMOKETEST_PROJECT_ID }}
        run: |
          BASE="https://api.smoketest.sh"
          AUTH=(-H "Authorization: Bearer $SMOKETEST_API_KEY")

          # Fetch all tests for the project
          IDS=$(curl -sf "${AUTH[@]}" "$BASE/v1/tests?projectId=$SMOKETEST_PROJECT_ID" | jq -r '.[].id')

          # Trigger all runs
          RUN_IDS=()
          for ID in $IDS; do
            RUN=$(curl -sf -X POST "${AUTH[@]}" "$BASE/v1/tests/$ID/run")
            RUN_IDS+=("$(echo "$RUN" | jq -r '.id')")
          done

          # Poll until every run completes
          FAILED=0
          for RUN_ID in "${RUN_IDS[@]}"; do
            while true; do
              RUN=$(curl -sf "${AUTH[@]}" "$BASE/v1/runs/$RUN_ID")
              STATUS=$(echo "$RUN" | jq -r '.status')
              case "$STATUS" in
                passed) break ;;
                failed|error|cancelled)
                  echo "Run $RUN_ID $STATUS: $(echo "$RUN" | jq -r '.reasoning // .errorMessage')"
                  FAILED=1
                  break
                  ;;
                *) sleep 5 ;;
              esac
            done
          done

          exit $FAILED

On this page