--- title: GitHub description: Install the Smoketest GitHub App and run tests as pull request checks or branch-push rules. --- # GitHub The Smoketest GitHub App lets you run selected tests when pull requests change or when branches receive new commits. Each matching trigger rule creates one aggregate GitHub check, waits for the configured deployment readiness condition, links to the Smoketest run detail page after runs start, and finishes when the linked run or runs finish. ## Connect GitHub 1. Open **Project Settings → Automations** in Smoketest. 2. If your workspace already has GitHub repository access, choose **Connect to project** beside a repository. You do not need to install the app again. 3. Otherwise, click **Install GitHub app**, choose the GitHub account and repositories, and return to Smoketest. 4. Add trigger rules for the connected repositories as needed. The same repository can connect to multiple Smoketest projects. Each project has its own connection, trigger rules and linked-PR analysis access. **Connected** refers to the current project; an available repository still needs **Connect to project**. **Disconnect** affects only the current project. It disables that project's repository rules and removes the repository from its deployment webhooks; other projects continue working. Existing check history keeps its original project. Reconnecting does not automatically re-enable disabled rules. Use **Manage installation** to change the repositories GitHub grants Smoketest access to across the workspace. ## Analyze linked PRs for ticket testing When Jira or Linear tickets link to a PR in a mapped repository, new Attempts automatically include **Analyze linked PRs** after ticket enrichment. Smoketest reads the PR description, bounded diffs and limited supporting code at captured revisions to find relevant flows, prerequisites and edge cases. The ticket remains the authority for expected behavior; code changes and merged status do not prove deployment or a passing test. Open PRs, including drafts, and merged PRs can provide context. Closed, unmerged PRs are skipped. This analysis does not require a GitHub trigger rule. It does not post its insights to the ticket. Expand the Attempt step for PR links, captured revisions, cited insights and coverage limits. Missing repository access, partial code or an analysis failure produces an explanation and testing continues using the available context. Older frozen Attempts are not reanalyzed; start a new Attempt to use newer PR changes. Supporting code reads require **Contents: read** permission on the Smoketest GitHub App installation. If GitHub requests approval for updated App permissions, the installation owner must approve them. Missing permission limits enrichment without blocking testing. ## Create a trigger rule After a repository is mapped, create a rule for that repository. | Setting | What it does | | --- | --- | | Event | Choose **Pull request** or **Push**. | | Target | Choose one test, or choose a project tag to run all tests with that tag. | | Branch patterns | Filter PR base branches or pushed branches. | | Preview URL template | Optionally override the test URL for this GitHub-triggered run. When URL readiness is enabled, Smoketest polls the rendered template before starting runs. | | Start test | Choose whether runs start immediately, after a fixed delay, or when the target URL is reachable. | | Enabled | Turn the rule on or off without deleting it. | Pull request rules run for opened, reopened, synchronized, and ready-for-review PRs. Draft PRs are skipped until they become ready for review. Push rules default to `main` and `master`. Leave branch patterns blank to match all branches. ## Deployment readiness GitHub checks stay **in progress** while Smoketest waits for the selected readiness condition. | Mode | Default | Use it for | | --- | --- | --- | | When URL is reachable | Pull request rules wait 30 seconds, then poll every 15 seconds for up to 10 minutes. | Vercel, Netlify, or other PR preview deployments where the URL becomes available after the webhook. | | After fixed delay | Push rules wait 120 seconds before starting. | Production branch pushes where the deployment usually takes a predictable amount of time. | | Immediately | No wait. | Repositories where the target URL is already serving the new code when the webhook arrives. | URL readiness treats `2xx` and `3xx` HTTP responses as ready. `4xx`, `5xx`, and network errors keep waiting until the timeout. If the URL never becomes reachable, Smoketest completes the GitHub check as **timed out** without starting runs. For PR previews, set a preview URL template such as `https://{branchSlug}--acme-preview.example.com` and keep **Start test** set to **When URL is reachable**. For production pushes to `main` or `master`, use **After fixed delay** and tune the delay to match your deploy time. ## Branch patterns Branch patterns are comma-separated and support `*` wildcards. Examples: | Pattern | Matches | | --- | --- | | `main` | Only `main` | | `main,master` | `main` or `master` | | `release/*` | Branches such as `release/2026-06` | | `preview/*,staging` | Preview branches plus `staging` | For pull request rules, patterns match the PR base branch. For push rules, patterns match the pushed branch. ## Preview URL templates Preview URL templates let one saved test run against per-branch or per-PR deployments. Leave the template blank to use the test's saved URL. When **Start test** is set to **When URL is reachable**, Smoketest uses the preview URL template as the readiness URL. If the template is blank, a single-test rule waits on that test's saved URL, and a tag rule waits on all unique saved URLs for the matching tests. Templates must render to an absolute `http` or `https` URL. Supported variables: | Variable | Meaning | | --- | --- | | `{owner}` | Repository owner | | `{repo}` | Repository name | | `{branch}` | Source branch for PRs, pushed branch for pushes | | `{branchSlug}` | URL-safe branch value | | `{baseBranch}` | PR base branch, or pushed branch for push events | | `{sha}` | Full commit SHA | | `{shortSha}` | First 7 characters of the SHA | | `{prNumber}` | Pull request number, blank for push events | Example: ```text https://{branchSlug}--acme-preview.example.com ``` If a template renders to an invalid URL, Smoketest completes the GitHub check as **action required** instead of leaving it pending. ## GitHub check results Each matching rule creates one check named `Smoketest / ` for test rules or `Smoketest / #` for tag rules. | Outcome | Meaning | | --- | --- | | `success` | All linked Smoketest runs passed. | | `failure` | At least one linked run failed or errored. | | `cancelled` | A linked run was cancelled. | | `neutral` | A tag rule matched no current tests. | | `timed_out` | A readiness URL never became reachable, or Smoketest did not receive terminal results before the check timeout. | | `action_required` | Smoketest could not enqueue the run, usually because billing, sharedStep validation, or preview URL validation blocked it. | Open the check details link to see the repository, commit, rule target, and linked Smoketest runs.