Tests
The difference between generated and reusable Tests and how each is executed.
A Test describes a user journey and its expected outcome in words, with markdown structure and no selectors or locators.
Smoketest has two Test lifecycles:
| Lifecycle | Where it comes from | Editing and triggers |
|---|---|---|
| Generated Test | One Attempt for a Jira or Linear Test Request | Immutable; runs only for its owning Attempt and never activates reusable-Test triggers |
| Reusable Test | Created in the Test Library, API, CLI, MCP, or saved as an independent copy | Editable; can run on demand, on schedules, from GitHub/deployments, and through public interfaces |
Anatomy of a reusable Test
Each test has four fields:
| Field | Purpose |
|---|---|
| Name | A short label for the test. Used in notifications, history, and the sidebar. |
| URL | The page where the agent starts. Relative paths are resolved against the project's base URL. |
| Description | Markdown-backed instructions in words. Tell the agent what to do, step by step. |
| Expected outcome | What a passing run should look like. The agent checks this at the end to decide pass or fail. |
What the agent does with them
When you trigger a run, Smoketest compiles the markdown description into an execution prompt for the AI agent. The agent:
- Opens a real Chromium browser at the given URL
- Reads the compiled description and plans its actions
- Navigates, clicks, types, and scrolls — just like a user would
- At the end, evaluates the page against the expected outcome
- Records a Result: Passed, Failed, or No Result
The agent sees the live DOM and screenshots, not your source code. It works the way a human tester would — visually, not structurally.
The editor highlights @module and /action placeholders, but they are warning-only today. They do not resolve shared prompt modules or execute first-party actions yet.
Intent instead of selectors
A Test is intentionally more flexible than a selector-bound script. It should survive a renamed button or CSS class because the agent follows user intent. The trade-off is that browser execution uses AI compute and credits.
Good test checklist
- One user journey per Test — don't combine unrelated login and checkout goals
- Be specific about interactions: "clicks the Sign in button" not "logs in"
- State the expected outcome explicitly: "and sees the dashboard with the heading Welcome back"
- Use real-looking but safe test credentials, not
admin/admin
Next steps
- Writing good tests — worked examples and patterns
- Test examples by category — copy-paste starters
When setting environmentId, include personaId explicitly (or null for no Persona). The Persona must be connected to the Environment in the same project. This applies to manual and scheduled Tests; automatic ticket Persona selection occurs only for Attempts.