Common Mistakes
Anti-patterns that produce flaky or inconclusive results — and how to rewrite them.
These are the most frequent test writing mistakes that lead to inconclusive results or unreliable runs.
Too vague
The agent has to interpret intent from your words. Vague instructions leave too much room for misinterpretation.
User logs in and checks their account.User navigates to /login.
Enters "[email protected]" in the email field.
Enters "TestPass123!" in the password field.
Clicks the "Sign in" button.
Expected outcome: The dashboard loads with the heading "Welcome back" visible.Multiple tests in one test
Testing too many things at once makes failures hard to diagnose and increases the chance of a timeout.
User logs in, updates their profile picture, posts a comment,
then logs out and verifies they can't access the dashboard.Split into four separate tests:
- Login test
- Profile picture update
- Comment posting
- Logout + protected route redirect
Non-observable expected outcome
The agent can only verify what's visible on the page. It can't check your database, read emails, or verify API calls.
Expected outcome: The user's email is saved to the database.Expected outcome: A success toast appears with the message
"Profile updated successfully." The new email is shown in
the profile section.Relying on specific CSS selectors or element IDs
Tests should describe what a user sees, not how the DOM is structured. Referencing internal identifiers makes tests fragile — they break when the UI changes.
Click the element with id="submit-btn-v2".Click the "Submit" button at the bottom of the form.Assuming too much pre-state
If the test depends on specific data being present (a user account, a published post, a product in stock), make sure that state actually exists in your test environment or describe how to create it first.
Tests run against the URL you provide — they don't set up database fixtures or seed data. If you need a user account to log in with, that account must already exist.
Missing expected outcome
A test without an explicit expected outcome forces the agent to guess what success means. Runs with vague outcomes are more likely to return inconclusive.
User opens the homepage and navigates around.User opens the homepage.
Clicks "Features" in the navigation bar.
Expected outcome: The features page loads.
The heading "Everything you need" is visible.