Tests
The difference between generated and reusable Tests and how each is executed.
A Test describes a user journey and its expected outcome in words, with markdown structure and no selectors or locators.
Smoketest has two Test lifecycles:
| Lifecycle | Where it comes from | Editing and triggers |
|---|---|---|
| Generated Test | One Attempt for a Jira or Linear Test Request | Immutable; runs only for its owning Attempt and never activates reusable-Test triggers |
| Reusable Test | Created in the Test Library, API, CLI, MCP, or saved as an independent copy | Editable; can run on demand, on schedules, from GitHub/deployments, and through public interfaces |
Anatomy of a reusable Test
Each test has four fields:
| Field | Purpose |
|---|---|
| Name | A short label for the test. Used in notifications, history, and the sidebar. |
| URL | The page where the agent starts. Relative paths are resolved against the project's base URL. |
| Description | Markdown-backed instructions in words. Tell the agent what to do, step by step. |
| Expected outcome | What a passing run should look like. The agent checks this at the end to decide pass or fail. |
What the agent does with them
When you trigger a run, Smoketest compiles the markdown description into an execution prompt for the AI agent. The agent:
- Opens a real Chromium browser at the given URL
- Reads the compiled description and plans its actions
- Navigates, clicks, types, and scrolls — just like a user would
- At the end, evaluates the page against the expected outcome
- Records a Result: Passed, Failed, or No Result
The agent sees the live DOM and screenshots, not your source code. It works the way a human tester would — visually, not structurally.
The editor highlights @module and /action placeholders, but they are warning-only today. They do not resolve shared prompt modules or execute first-party actions yet.
Intent instead of selectors
A Test is intentionally more flexible than a selector-bound script. It should survive a renamed button or CSS class because the agent follows user intent. The trade-off is that browser execution uses AI compute and credits.
Good test checklist
- One user journey per Test — don't combine unrelated login and checkout goals
- Be specific about interactions: "clicks the Sign in button" not "logs in"
- State the expected outcome explicitly: "and sees the dashboard with the heading Welcome back"
- Use real-looking but safe test credentials, not
admin/admin
Next steps
- Writing good tests — worked examples and patterns
- Test examples by category — copy-paste starters