Scenarios
A scenario defines what you want to test — a single, focused behavior or flow.
- Project-scoped: Every scenario belongs to a project
- Contains checkpoints: An ordered list of milestones the agent must achieve
- Draft vs Published: Scenarios start as drafts. Publish them when they’re ready for formal test runs
- Preconditions: Assumptions about the game state before the test starts (e.g., “player is on level 212”, “account has 500 coins”)
- Variables: Placeholder values (like
{{username}}or{{bet_amount}}) that get substituted at runtime, so you can reuse the same scenario with different configurations - Reusable: Once published, a scenario can be run many times across different builds and devices
Checkpoints
A checkpoint is a single verifiable milestone the agent must reach during the test.
Goal (Required)
What should the agent achieve? A clear, outcome-focused statement.- “Close the welcome popup”
- “Build all items except last on Islands screen”
- “Win a poker hand”
Instructions (Optional)
How should the agent achieve it? Step-by-step guidance for complex tasks. Only add these when the path isn’t obvious.- “Enter the Islands screen and build items until the last item is left. Stop when the build button becomes inactive.”
- “Watch the completion animation, collect rewards, tap Continue, close the popup, and change bet to maximum.”
Checkpoint Statuses
During a run, each checkpoint moves through these statuses:- Pending — Not yet reached. This is the default when the scenario starts
- Completed — The agent successfully reached this checkpoint. Includes a timestamp, screenshot, and duration
- Failed — The agent was unable to reach this checkpoint. Includes a reason:
- Timeout: The agent didn’t reach the goal within the time limit
- Skipped: A later checkpoint was reached first (required checkpoints only)
- Agent explanation: The agent’s own reasoning for why it couldn’t proceed
Variables
Variables let you write flexible scenarios that adapt at runtime. Instead of hard-coding values like “Bet 100 coins”, you write “Bet{{bet_amount}} coins”. In the Run screen, you choose the actual value that will be substituted in. This means one scenario can cover multiple configurations without duplication.
Project Variables
You can define project-level variables with default values that apply across all scenarios. Variable keys must follow the formatUPPER_CASE_WITH_UNDERSCORES (e.g., BET_AMOUNT, PLAYER_LEVEL). When starting a run, you can override any project variable’s default value for that specific run.
Preconditions
Preconditions describe the assumed game state before the test begins. They tell the agent (and the reader) what to expect when the scenario starts.- “Player is logged in and on the main menu”
- “Account has at least 500 coins”
- “Tutorial has been completed”
Tips for Writing Good Scenarios
- One behavior per scenario — Don’t try to test everything in one scenario. Keep each one focused
- Start with the goal — Write the checkpoint goal first. Only add instructions if the path isn’t obvious
- Be specific in goals — “Close the daily bonus popup” is better than “Handle the popup”
- Let the agent explore — Don’t over-specify. If the agent can figure out the next step, skip the instructions
- Keep timeouts realistic — A simple tap needs 1–2 minutes. A multi-step flow might need 5–10 minutes
Quick Reference
| Concept | What it is | Example |
|---|---|---|
| Scenario | A structured test plan for one behavior | ”Complete First Island” |
| Checkpoint | A milestone the agent must reach | ”Build all items except last” |
| Goal | What the agent should achieve (required) | “Close the welcome popup” |
| Instructions | How to achieve the goal (optional) | “Tap the X button in the top-right corner” |
| Variable | A placeholder substituted at runtime | {{bet_amount}} |
| Precondition | Assumed game state before the test | ”Player is on the main menu” |