exploratory-charter · git:20260810.36b8bbd · 2026-08-10 · sha256 3e428d63be24030c
exploratory-charter git:20260810.36b8bbdA
Immutable. This exact content is served forever at /api/v1/blob/3e428d63be24030c.
---
description: Write, run, and debrief exploratory testing charters against Katalon True Platform/TestOps when there is no script to follow. Use when you need to turn a vague area into a charter (mission, areas, oracles, timebox), run a timeboxed unscripted session, log what you find as session notes, judge which findings are real defects, and promote what was learned into atomic manual cases plus the folder or suite that holds them. This is unscripted discovery and the debrief after it; when a written requirement is already in hand and the job is designing cases from it, use create-test-cases instead. Written for the manual tester who has an hour, a feature, and no cases to run against it.
alwaysApply: false
---
<!-- GENERATED by scripts/build-adapters.mjs from skills/. Do not edit by hand. -->
# Katalon Exploratory Charter
Use this skill for the **unscripted** part of the design stage: a timeboxed session that learns something the written requirement does not say, then converts what it learned into scripted coverage. A charter is not a test case and a session is not a test run. The value is the *learning plus the promotion*, so a session that produces no notes and no promoted cases has failed even if nothing crashed.
Treat the human as a tool. Ask when the area, the timebox, the environment, or the risk appetite cannot be resolved safely. Never invent an oracle.
## Availability Boundary
**There is no exploratory session object in the platform.** Say that before promising to "run a session in Katalon".
- **Available via MCP:** resolve scope (`list_projects`, `list_repositories`), read the oracles you already have (`find_requirements`, `read_requirement`, `fetch_requirement_data`), check whether an area is already covered before spending the timebox on it (`find_test_cases`, `find_test_cases_by_requirement`), and land the outcome (`manage_test_folder`, `create_test_case`, `link_requirements_to_test_case`, `manage_test_suite`, `read_test_suite`).
- **Not available, and no workaround exists:** a charter / session / session-sheet entity, a session timer or duration field, attachment upload for screenshots or video or HAR files, a tool that sets a manual test result to failed, custom fields or tags to mark a case as exploratory in origin, and creating a requirement for behavior the charter discovered. Do not reach for `create_charter`, `upload_attachment`, `set_test_result_status`, or `manage_tags` - none of them exists.
- **The defect trap.** `create_defect` requires a known **failed test result ID**. An unscripted session produces no test result, so a charter finding cannot be filed through MCP as it stands. In order of preference: (1) file it in Jira or Azure DevOps directly, outside the MCP; (2) execute the reproduction case and mark it failed in the TestOps UI, then file from that run. A third path is MCP-legal but conditional - promote the finding to a case, `create_manual_test_run`, `create_manual_ai_session`, and if the AI run genuinely fails, `create_defect` with that result ID. It only works when Run with AI actually reproduces the bug, so never offer it as the default.
- **The Katalon MCP cannot see the application.** It reads and writes platform objects only. Every observation in a session comes from a browser you drive yourself, so use Browser or Playwright for the product and the MCP for the artifacts.
- **Where the charter lives.** Author the charter and the session sheet as a local artifact the user keeps. Optionally mirror the charter into the platform as one test case in the charter folder, mission in Description and session log in Steps. Say which you did.
Read `references/charter-and-session.md` before writing a charter. Consult `true-platform-testing/references/unavailable-capabilities.md` when the user asks whether Katalon can run exploratory sessions for them.
## Session Workflow
```text
+------------------+ +------------------+ +------------------+
| Frame the area | --> | Write charter | --> | Run the timebox |
| scope + coverage | | mission/oracles | | notes as you go |
+------------------+ +------------------+ +------------------+
|
v
+------------------+ +------------------+ +------------------+
| Promote to cases | <-- | Debrief | <-- | Classify findings|
| hand to authoring| | timebox account | | oracle broken |
+------------------+ +------------------+ +------------------+
```
## Steps and tool rules
1. **Resolve scope and existing coverage.** `list_projects` -> `list_repositories`. Then `find_test_cases` on the area keywords and `find_test_cases_by_requirement` when a requirement key is known. **Coverage is an input to the charter, not an afterthought**: an area that already has twenty cases is a poor use of the timebox, and the gaps are where the charter should point.
2. **Write the charter before touching the application.** Six fields, all required: mission, areas, oracles, timebox, out of scope, done when. Show it to the user and get one confirmation. A charter without oracles is a browsing session; refuse to start until at least one oracle is named.
3. **Choose the tours that fit the mission.** Pick two or three from `references/oracles-and-tours.md`. Name them in the charter so the session is reproducible by another tester.
4. **Run the timebox and take notes as you go, not afterwards.** Every note is timestamped and typed - `SETUP`, `OBS`, `Q`, `BUG`, `RISK`, `IDEA`. A `BUG` note is worthless without the steps that produced it, so write the repro before moving on. Use Browser or Playwright for the application itself; the Katalon MCP cannot see the AUT UI.
5. **Stop when the timebox ends, not when you run out of ideas.** Record the timebox accounting - minutes on charter, minutes on bug investigation and writeup, minutes on setup. A session that spent 40 of 60 minutes on setup is a finding about the environment; report it as one.
6. **Classify every finding against the oracle it broke.** Product defect, test-data or environment issue, question for the product owner, or design risk. A finding with no broken oracle is an observation, not a defect - say so plainly rather than inflating it.
7. **Debrief.** Report what the mission was, what was covered, what was not reached, the findings by class, the timebox accounting, and the charter you would run next. The next charter is part of the deliverable.
8. **Promote, then hand off.** Turn confirmed findings into a promotion list - title, priority, requirement link if one exists, and the one-line expected result. `manage_test_folder` to create or reuse the charter folder, then **hand authoring to `create-test-cases`**, which owns duplicate checking, the atomic-case rules, and import. Do not reimplement those rules here. Add the promoted cases to the relevant suite with `manage_test_suite` and confirm with `read_test_suite`.
## Hard rules
- **No oracle, no charter.** At least one named, checkable oracle before the timebox starts. *It feels wrong* is not an oracle.
- **The timebox ends the session.** Not the idea supply. An overrun charter is two charters that were never split.
- **A `BUG` note carries its repro or it is an `OBS`.** Write the reproduction before moving to the next observation, while the state is still on screen.
- **A finding with no broken oracle is not a defect.** Report it as a question or a risk. Inflating observations into defects is how a session loses its credibility.
- **Never explore production without explicit confirmation.** Confirm the environment from `read_auts` or from the user before the first action.
- **A session with no promoted cases and no notes has failed.** Nothing crashing is not a result.
## Naming and platform constraints
- Test case and folder **names** accept only letters, numbers, spaces, and `( ) . , _ -`. Charter titles love colons, and a colon will be rejected. Use `CH-01 Explore guest checkout with expired card`, never `CH-01: Explore ...`. Folder paths may use `/`.
- There is no tag or custom field via MCP, so the only durable marker of exploratory origin is the naming convention plus the folder. Keep both: folder `Exploratory / <charter id> <short mission>`, and prefix promoted case titles with the charter id.
- Prefer `update_test_case` over delete-and-recreate when a later session revises a promoted case. Deletion can fail server-side and updating in place keeps IDs, links, and history.
## Worked example
**User asks:** *We shipped guest checkout last week with no cases for it. Run an hour of exploratory testing on the payment step in Cellphone Shop and tell me what to write up.*
**Step 1 - scope and coverage.**
```text
list_projects -> Cellphone Shop (id 4021), Demo Bank (id 3980)
list_repositories -> Katalon Cloud (id 118) [exactly one, so use it]
find_requirements "CEL" -> CEL-6 Guest checkout (synced from Jira)
find_test_cases_by_requirement CEL-6
-> 4 cases, all on the cart step, none on payment
```
Coverage gap confirmed, so the payment step is worth the timebox.
**Step 2 - the charter, confirmed with the user before starting.**
```text
CHARTER CH-01
Mission Explore guest checkout payment with declined, expired, and
mistyped cards, to discover error-handling and cart-state defects
Areas Payment form, order confirmation, cart persistence, email receipt
Oracles CEL-6 acceptance lines 3 and 4 (an unsuccessful payment keeps
the cart intact and shows a retry message)
Consistency with the logged-in checkout flow, which is covered
Consistency with purpose (no order record means no charge)
The payment provider's published test-card behavior
Tours Error-message tour, then money tour
Timebox 60 minutes, one tester, staging
Out of scope Saved cards, subscriptions, refunds, mobile web
Done when Every declined-card path has a recorded outcome and every
finding is classified
```
**Step 4 - session sheet excerpt.**
```text
00:00 SETUP Staging reachable, cart seeded with 1 phone, provider test keys present
00:07 OBS CVV field accepts 5 digits without complaint, submit still enabled
00:09 BUG-1 Expired card 01/24 -> spinner 41s -> blank page. Back button shows an
EMPTY cart. No order in Orders. No charge on the provider dashboard.
Repro 3/3. Oracle broken - CEL-6 line 3, cart must survive a decline
00:24 Q-1 Is a 5-digit CVV meant to be accepted, or is the provider silently
truncating? No acceptance line covers CVV length
00:31 BUG-2 Declined card shows the raw provider string "card_declined" instead
of a user message. Oracle broken - CEL-6 line 4
00:38 RISK-1 Retrying after a decline re-submits the same idempotency key. Could
double-charge under a slow network. Not reproduced in the timebox
00:52 OBS Receipt email never arrives for guest orders. Out of scope, logged
01:00 STOP
```
**Step 5 - timebox accounting.** 44 minutes on charter, 11 on bug investigation and writeup, 5 on setup.
**Step 6 - findings.**
| ID | Class | Oracle broken | Severity | Promote |
|---|---|---|---|---|
| BUG-1 | product defect | CEL-6 line 3 | P0 | yes, 2 cases |
| BUG-2 | product defect | CEL-6 line 4 | P1 | yes, 1 case |
| Q-1 | question for the product owner | none, the rule does not exist | n/a | no, ask first |
| RISK-1 | design risk, unreproduced | none yet | n/a | no, next charter |
**Step 8 - promotion list handed to `create-test-cases`.**
```text
CH-01 Guest checkout keeps the cart after an expired card is declined P0 CEL-6
CH-01 Guest checkout shows a retry message after a decline P0 CEL-6
CH-01 Guest checkout shows a readable message not a provider code P1 CEL-6
```
```text
find_test_cases "checkout expired card" -> 0 matches [dedupe before create]
manage_test_folder create "Exploratory / CH-01 Guest checkout payment"
-> folder id 9134
-> hand to create-test-cases for create_test_case x3 + link_requirements_to_test_case
manage_test_suite add the 3 cases to "CEL-6 Regression"
read_test_suite -> 7 cases, 3 new [confirmed]
```
**Expected report back to the user.** Name every one of these; a report missing the boundary line is incomplete:
```text
Charter CH-01, guest checkout payment, 60 minutes on staging. Timebox: 44 charter / 11 writeup / 5 setup.
Findings: 2 product defects (BUG-1 P0 cart lost on decline, BUG-2 P1 raw provider string),
1 open question (CVV length, no acceptance line covers it), 1 unreproduced risk (duplicate idempotency key).
Promoted: 3 cases under folder 9134 "Exploratory / CH-01 Guest checkout payment", added to the CEL-6
Regression suite. read_test_suite confirms 7 cases, 3 new.
Not reached: email receipt path, out of scope by the charter.
Next charter: CH-02, retry and idempotency under a slow network, 45 minutes.
Boundary: the two defects were NOT filed through MCP. create_defect needs a failed test result ID and this
session produced none. File them in Jira, or run the three promoted cases and file from the failed run.
```
## Prompt recipes
- `Run a 60 minute exploratory charter on guest checkout payment in Cellphone Shop and give me the findings.`
- `Write a charter for the new search filters. I have no requirement, just the feature.`
- `Debrief my session notes below and tell me which findings should become test cases.`
- `We have an hour before the release. Which area is worth exploring given current coverage?`
- `Which of these findings are real defects and which are just observations?`
## Hand-offs
- Promoted cases to author, dedupe, and import -> `create-test-cases`.
- The data a promoted case needs before anyone can run it -> `test-data`.
- Running the promoted cases, by hand or with AI -> `execute-test`.
- **A red run, not a session.** If the user already has failed test results, this is not the skill - `analyze-failures` owns failures with a result ID. This skill owns findings that have no result ID at all.
- A charter programme across a sprint or release -> `test-plan`.
- Coverage gaps the session exposed across the repository -> `test-management`.
Read `references/charter-and-session.md` before writing a charter, and `references/oracles-and-tours.md` before choosing how to explore.
---
## Bundled references
_The reference material the skill points to is inlined below so this file is self-contained._
### references/charter-and-session.md
# Charter and Session
## The six-field charter template
Copy this verbatim. All six fields are required. The hard rule first: **no oracle, no charter.**
```text
CHARTER <id, e.g. CH-01>
Mission <what you are exploring, and what information you want out of it>
Areas <the surfaces in scope, named concretely>
Oracles <the rules a finding could break, each one checkable>
Tours <the two or three tours you will use>
Timebox <45, 60, or 90 minutes, how many testers, which environment>
Out of scope <the adjacent things you will not chase>
Done when <the observable condition that ends the session>
```
| Field | The one rule |
|---|---|
| Mission | A target plus an information goal. If it does not say what you want to *learn*, it is not a mission. |
| Areas | Named surfaces, not the whole feature. Four is a session, twelve is a release. |
| Oracles | Each one must be checkable by a second person. Point at an acceptance line, a covered flow, a published behavior, or a stated purpose. |
| Tours | Named in advance so another tester can reproduce the session, not just the bugs. |
| Timebox | Includes the environment. A charter that does not say where it runs will be run in the wrong place. |
| Out of scope | Written down so a tempting detour becomes a note instead of a lost 20 minutes. |
| Done when | An observable condition. "When I have looked at it enough" is not one. |
## Good versus bad missions
The difference is always the same: a target plus an information goal.
| Bad | Good | What changed |
|---|---|---|
| Test checkout. | Explore guest checkout payment with declined, expired, and mistyped cards, to discover error-handling and cart-state defects. | Named the surface, named the inputs, named what you want to learn. |
| Have a look at the new search. | Explore search filters with empty, single, and conflicting selections, to discover result-count and reset defects. | Replaced "have a look" with the conditions that could break it. |
| Check the mobile layout works. | Explore the order history screen at 320px and on a slow network, to discover truncation, overflow, and loading-state defects. | Gave the vague quality bar concrete conditions and an information goal. |
## The session sheet
One line per note. Timestamped, typed, written as you go and never reconstructed afterwards.
| Type | Means | Rule |
|---|---|---|
| `SETUP` | Getting the environment ready | Counts against the timebox. Log it honestly. |
| `OBS` | Something observed, no rule broken | The default. Most notes are this. |
| `Q` | A question no oracle answers | Names the missing rule, and who should answer. |
| `BUG` | An oracle was broken | **Carries its repro, or it is an `OBS`.** |
| `RISK` | A plausible failure not reproduced in the timebox | Says what would confirm it. |
| `IDEA` | A charter worth running later | Feeds the next-charter field of the debrief. |
Rules that make the sheet worth keeping:
- **Timestamp every line** as `MM:SS` from the session start. The timestamps are what produce the accounting below.
- **Write the repro before moving on.** The state is on screen now and gone in 90 seconds. Steps, the data used, and how many times out of how many it reproduced.
- **Name the broken oracle inside the `BUG` note**, not later. A `BUG` that cannot name its oracle is an `OBS` that got excited.
- **Log out-of-scope discoveries as `OBS` and keep going.** That is what the out-of-scope field is for.
- **Evidence capture is on the tester.** The MCP has no attachment upload, so screenshots and recordings go in the ALM ticket, not the platform.
## Timebox accounting
Three buckets, reported every time:
| Bucket | What counts |
|---|---|
| On charter | Actually exploring the mission |
| Bug investigation and writeup | Reproducing, narrowing, and writing findings |
| Setup | Environment, accounts, data, tooling |
Read the shape, do not just report the numbers:
- **Setup over 25 percent is an environment finding.** Report it as one. The next tester will pay the same tax.
- **Bug writeup over 50 percent means the session found one thing** and should have been split into an investigation and a fresh charter.
- **On charter under 50 percent means the session did not really happen.** Say so rather than presenting the survivors as a full sweep.
## The debrief checklist
Six questions. Each has a weak answer to avoid.
| Question | Weak answer looks like |
|---|---|
| What was the mission? | Restating the feature name instead of the information goal. |
| What was covered? | "Checkout" instead of the specific paths actually walked. |
| What was not reached? | Silence. Unreached areas are the most useful line in the debrief. |
| What was found, by class? | An undifferentiated bug list with no oracle named per item. |
| Where did the time go? | No accounting, or accounting with no reading of its shape. |
| What charter would you run next? | "More of the same." The next charter is part of the deliverable. |
## The promotion rubric
A finding becomes a scripted case when **all three** hold:
1. It is **reproducible**, with steps that work from a stated starting state.
2. It maps to a **rule someone will defend**, an oracle a product owner would stand behind.
3. It **would not be caught by existing coverage**, checked with `find_test_cases` before proposing it.
Otherwise it stays where it is:
- **No oracle covers it** -> it stays a question. Ask the product owner; do not invent the rule and write a case that enforces your invention.
- **Not reproducible inside the timebox** -> it stays a risk. Propose it as the next charter with what would confirm it.
- **Already covered** -> it is a duplicate. Say so and move on.
**The handoff contract to `create-test-cases`** is four fields per promoted finding and nothing else:
```text
<charter id> <title, no colon> <priority> <requirement key or none>
Expected: <one line>
```
`create-test-cases` owns duplicate checking, the atomic-case rules, and import. Do not pre-write steps here and do not reimplement those rules.
## Charter sizing
- **45 minutes** for a narrow surface with a known oracle.
- **60 minutes** is the default and the one to reach for when unsure.
- **90 minutes** only with a second tester or a heavy setup cost, and only when the mission genuinely cannot be split.
- **A charter that needs more than 90 minutes is two charters.** Split it on the mission, not on the clock.
### references/oracles-and-tours.md
# Oracles and Tours
An oracle is how you decide something is wrong. A tour is how you decide where to look. The charter needs both: oracles make findings defensible, tours make the timebox productive.
## The oracle taxonomy
Seven kinds. Name which one you are using in the charter, because that is what a finding gets judged against later.
| Oracle | The question it asks | Katalon-flavoured example |
|---|---|---|
| **Consistency within the product** | Do two parts of the product disagree? | The logged-in checkout keeps the cart after a decline; the guest checkout empties it. |
| **Consistency with history** | Did this behave differently before? | Order search returned partial matches last release and now requires an exact ID. |
| **Consistency with a comparable product** | Does a peer product set the expectation? | Every storefront in the market keeps the cart on a payment failure. |
| **Consistency with claims** | Does a written statement say otherwise? | `read_requirement` on CEL-6 line 3 states the cart survives an unsuccessful payment. |
| **Consistency with user expectations** | Would a reasonable user be surprised? | A 41 second spinner with no progress indicator on the payment step. |
| **Consistency with purpose** | Does it defeat what the feature is for? | No order record exists but the provider dashboard shows a charge. |
| **Consistency with standards or statutes** | Is there a rule outside the product? | The payment provider's published test-card behavior, or a WCAG contrast minimum. |
The strongest findings break **two or more** at once. BUG-1 in the skill's worked example breaks claims (CEL-6 line 3), within-product consistency (the logged-in flow), and purpose (cart state lost) together, which is why it is a P0 and not an argument.
## Naming an oracle so it is checkable
The test: **could a second person verify the oracle without you in the room?**
| Not an oracle | An oracle |
|---|---|
| It feels wrong. | The logged-in flow keeps the cart after a decline and the guest flow does not. |
| This is bad UX. | The error shows the raw provider string `card_declined` rather than a user message, against CEL-6 line 4. |
| It seems slow. | The payment step takes 41 seconds with no progress indicator, against the 5 second budget in the requirement. |
| Users will hate this. | Every competitor keeps the cart on failure, and support ticket CS-882 says ours does not. |
Point at one of four things, always:
1. **An acceptance line** you can quote, from `read_requirement`.
2. **A covered flow** you can name, from `find_test_cases`.
3. **A published behavior** you can link, such as provider docs or a standard.
4. **A stated purpose** the feature would defeat.
If you cannot point at any of the four, you have a `Q`, not a `BUG`. Log it and ask.
## Tours
A tour is a lens that constrains where you look. Pick two or three, never all of them.
| Tour | Walk it by | It pays when |
|---|---|---|
| **Feature tour** | Visiting every capability once, shallowly | The feature is new and nobody has mapped it |
| **Money tour** | Following the paths that take or move money | Checkout, billing, subscriptions, refunds |
| **Landmark tour** | Hopping between the key screens in varied orders | Navigation and state are suspect |
| **Data tour** | Feeding empty, huge, unicode, and malformed values | Forms, imports, search |
| **Configuration tour** | Changing settings, then re-walking a flow | Feature flags, roles, tenant settings |
| **Error-message tour** | Deliberately provoking every failure path | Validation and error handling are the risk |
| **Back-button tour** | Browser back, forward, refresh, and deep links at every step | Multi-step flows and wizards |
| **Interruption tour** | Killing the network, the tab, or the session mid-flow | Long transactions, payments, uploads |
| **Anti-social tour** | Doing the least reasonable legal thing at each step | The happy path is already covered |
## Choosing tours from the mission
| Mission shape | Tours that pay |
|---|---|
| A new feature nobody has tested | Feature tour, then data tour |
| A rewrite of something that worked | Consistency-driven landmark tour, then back-button tour |
| A cluster of related bugs | Error-message tour, then interruption tour |
| A pre-release sweep with an hour | Money tour, then feature tour |
| A payment or checkout surface | Error-message tour, then money tour, then interruption tour |
| A form-heavy or search surface | Data tour, then anti-social tour |
| A permissions or multi-tenant change | Configuration tour, then landmark tour |
Two or three. A charter naming six tours has not chosen; it has listed.
## The boundary, restated
- **The Katalon MCP cannot see the application.** Drive the product with Browser or Playwright. The MCP reads requirements and writes cases, folders, and suites, and that is all it does here.
- **Evidence capture is on the tester.** There is no attachment upload tool, so screenshots, recordings, and HAR files belong in the ALM ticket, not in the platform. Reference them from the finding by ticket ID.
- **There is no exploratory session object**, so the charter and the session sheet are local artifacts unless you deliberately mirror the charter into a test case. Say which you did.