GitHub
Install the Smoketest GitHub App and run tests as pull request checks or branch-push rules.
The Smoketest GitHub App lets you run selected tests when pull requests change or when branches receive new commits. Each matching trigger rule creates one aggregate GitHub check, waits for the configured deployment readiness condition, links to the Smoketest run detail page after runs start, and finishes when the linked run or runs finish.
Connect GitHub
- Open Project Settings → Automations in Smoketest.
- If your workspace already has GitHub repository access, choose Connect to project beside a repository. You do not need to install the app again.
- Otherwise, click Install GitHub app, choose the GitHub account and repositories, and return to Smoketest.
- Add trigger rules for the connected repositories as needed.
The same repository can connect to multiple Smoketest projects. Each project has its own connection, trigger rules and linked-PR analysis access. Connected refers to the current project; an available repository still needs Connect to project.
Disconnect affects only the current project. It disables that project's repository rules and removes the repository from its deployment webhooks; other projects continue working. Existing check history keeps its original project. Reconnecting does not automatically re-enable disabled rules. Use Manage installation to change the repositories GitHub grants Smoketest access to across the workspace.
Analyze linked PRs for ticket testing
When Jira or Linear tickets link to a PR in a mapped repository, new Attempts automatically include Analyze linked PRs after ticket enrichment. Smoketest reads the PR description, bounded diffs and limited supporting code at captured revisions to find relevant flows, prerequisites and edge cases. The ticket remains the authority for expected behavior; code changes and merged status do not prove deployment or a passing test.
Open PRs, including drafts, and merged PRs can provide context. Closed, unmerged PRs are skipped. This analysis does not require a GitHub trigger rule. It does not post its insights to the ticket.
Expand the Attempt step for PR links, captured revisions, cited insights and coverage limits. Missing repository access, partial code or an analysis failure produces an explanation and testing continues using the available context. Older frozen Attempts are not reanalyzed; start a new Attempt to use newer PR changes.
Supporting code reads require Contents: read permission on the Smoketest GitHub App installation. If GitHub requests approval for updated App permissions, the installation owner must approve them. Missing permission limits enrichment without blocking testing.
Create a trigger rule
After a repository is mapped, create a rule for that repository.
| Setting | What it does |
|---|---|
| Event | Choose Pull request or Push. |
| Target | Choose one test, or choose a project tag to run all tests with that tag. |
| Branch patterns | Filter PR base branches or pushed branches. |
| Preview URL template | Optionally override the test URL for this GitHub-triggered run. When URL readiness is enabled, Smoketest polls the rendered template before starting runs. |
| Start test | Choose whether runs start immediately, after a fixed delay, or when the target URL is reachable. |
| Enabled | Turn the rule on or off without deleting it. |
Pull request rules run for opened, reopened, synchronized, and ready-for-review PRs. Draft PRs are skipped until they become ready for review.
Push rules default to main and master. Leave branch patterns blank to match all branches.
Deployment readiness
GitHub checks stay in progress while Smoketest waits for the selected readiness condition.
| Mode | Default | Use it for |
|---|---|---|
| When URL is reachable | Pull request rules wait 30 seconds, then poll every 15 seconds for up to 10 minutes. | Vercel, Netlify, or other PR preview deployments where the URL becomes available after the webhook. |
| After fixed delay | Push rules wait 120 seconds before starting. | Production branch pushes where the deployment usually takes a predictable amount of time. |
| Immediately | No wait. | Repositories where the target URL is already serving the new code when the webhook arrives. |
URL readiness treats 2xx and 3xx HTTP responses as ready. 4xx, 5xx, and network errors keep waiting until the timeout. If the URL never becomes reachable, Smoketest completes the GitHub check as timed out without starting runs.
For PR previews, set a preview URL template such as https://{branchSlug}--acme-preview.example.com and keep Start test set to When URL is reachable. For production pushes to main or master, use After fixed delay and tune the delay to match your deploy time.
Branch patterns
Branch patterns are comma-separated and support * wildcards.
Examples:
| Pattern | Matches |
|---|---|
main | Only main |
main,master | main or master |
release/* | Branches such as release/2026-06 |
preview/*,staging | Preview branches plus staging |
For pull request rules, patterns match the PR base branch. For push rules, patterns match the pushed branch.
Preview URL templates
Preview URL templates let one saved test run against per-branch or per-PR deployments. Leave the template blank to use the test's saved URL.
When Start test is set to When URL is reachable, Smoketest uses the preview URL template as the readiness URL. If the template is blank, a single-test rule waits on that test's saved URL, and a tag rule waits on all unique saved URLs for the matching tests.
Templates must render to an absolute http or https URL. Supported variables:
| Variable | Meaning |
|---|---|
{owner} | Repository owner |
{repo} | Repository name |
{branch} | Source branch for PRs, pushed branch for pushes |
{branchSlug} | URL-safe branch value |
{baseBranch} | PR base branch, or pushed branch for push events |
{sha} | Full commit SHA |
{shortSha} | First 7 characters of the SHA |
{prNumber} | Pull request number, blank for push events |
Example:
https://{branchSlug}--acme-preview.example.comIf a template renders to an invalid URL, Smoketest completes the GitHub check as action required instead of leaving it pending.
GitHub check results
Each matching rule creates one check named Smoketest / <test name> for test rules or Smoketest / #<tag> for tag rules.
| Outcome | Meaning |
|---|---|
success | All linked Smoketest runs passed. |
failure | At least one linked run failed or errored. |
cancelled | A linked run was cancelled. |
neutral | A tag rule matched no current tests. |
timed_out | A readiness URL never became reachable, or Smoketest did not receive terminal results before the check timeout. |
action_required | Smoketest could not enqueue the run, usually because billing, sharedStep validation, or preview URL validation blocked it. |
Open the check details link to see the repository, commit, rule target, and linked Smoketest runs.