analyze-failures git:20260807.4fb0cddA

Immutable. This exact content is served forever at /api/v1/blob/8d47d18cab23ed7f.

---
description: Triage Katalon True Platform/TestOps test failures and file defects. Use when you need to investigate failed test results, classify each failure as product defect vs automation defect vs environment/data issue, cluster failures by common signature, find likely root cause from execution data, and optionally create ALM-linked defects for real product bugs. This is failure diagnosis and defect filing; for the overall ship/no-ship release call use release-analyze, and for repairing the tests themselves use test-maintenance. Written for the manual tester and the automation tester looking at a red run and needing to know whether the application broke or the test did.
alwaysApply: false
---

<!-- GENERATED by scripts/build-adapters.mjs from skills/. Do not edit by hand. -->

# Katalon Analyze Failures

Use this skill for the **failure-analysis** part of the report/analysis stage: turn a set of failed results into a diagnosis and, when warranted, filed defects. The core value is **classification** — separating real product bugs from automation and environment noise.

## Availability Boundary

- **Available via MCP:** read results (`read_test_result`, `read_execution_test_results`, `find_test_results`, `read_execution`), defect context (`fetch_defect_data`), ALM discovery + filing (`find_alm_integration_projects`, `create_defect`).
- **Not directly available:** AI root-cause summarization and automation-error-pattern analytics are TestOps/Studio product features, not MCP calls — narrate their availability, do not claim to call them. `create_defect` requires a **known failed test result ID** and ALM integration details; there is no ID-less defect creation.

## Triage Workflow

```text
+---------------------+     +----------------------+     +----------------------+
| Collect failures    | --> | Classify each        | --> | Cluster by signature |
| read results        |     | product/auto/env     |     |                      |
+---------------------+     +----------------------+     +----------------------+
                                                                   |
                                                                   v
                                                         +----------------------+
                                                         | File defects (asked) |
                                                         +----------------------+
```

## Steps and tool rules

1. **Collect the failures.** `find_test_results` (recent/specific) or `read_execution_test_results` for a run; `read_test_result` per failed case for detail.
2. **Classify each failure** into one bucket:
   - **Product defect** — the application behaved wrong (assertion on real behavior failed, unexpected error/state). Candidate for a filed defect.
   - **Automation defect** — the test is wrong (bad locator, timing, stale data, broken step). Route to `test-maintenance`.
   - **Environment / data** — infra, account, network, fixture, or AUT-state issue. Route to re-run after fix.
3. **Cluster by signature.** Group failures with the same error message / step / object so one root cause is not filed as N defects.
4. **Check existing defects.** `fetch_defect_data` to avoid duplicate filings.
5. **File defects only when asked and only for product defects.** `find_alm_integration_projects` -> `create_defect` with the failed result ID. Ask before creating unless the user explicitly requested defect filing.
6. **Report.** Per cluster: classification, likely cause, affected cases, and action (file / repair / re-run).

## Prompt recipes

- `Triage the failures in execution 8842: which are product bugs vs flaky tests vs environment?`
- `Cluster today's failed results by root cause and tell me what to file.`
- `File defects for the confirmed product bugs in the checkout suite and link them to the failed results.`

## Hand-offs

- Automation defects / flaky -> `test-maintenance`.
- Ship decision from the failure picture -> `release-analyze`.
- Coverage gap exposed by a failure -> `test-plan`.

Read `references/failure-triage.md` before classifying. Consult the orchestrator's `references/unavailable-capabilities.md` for defect-filing boundaries.

---

## Bundled references

_The reference material the skill points to is inlined below so this file is self-contained._


### references/failure-triage.md

# Failure triage reference

## Classification signals

| Bucket | Typical signals | Action |
|---|---|---|
| Product defect | assertion on real app behavior failed; unexpected app error/state; reproducible across runs and environments | file defect (with failed result ID) |
| Automation defect | element-not-found / locator error; timing / wait failure; stale test data; step no longer matches UI; passes on re-run without app change | route to `test-maintenance` |
| Environment / data | infra/network error; account/permission; missing fixture; wrong AUT state; TestCloud/agent issue | fix environment, then re-run |

Reproducibility is the strongest signal: a failure that repeats across environments and runs leans product; one that vanishes on re-run leans automation/environment.

## Clustering

Group by the tuple `(failing step, error message, object/locator)`. One cluster = one candidate root cause = at most one defect. Report the cluster size so a single bug is not filed N times.

## Defect filing rules

- Only for **product defects**, only when the user asked (or explicitly pre-approved failure filing).
- Requires a **failed test result ID** — get it from `read_test_result` / `read_execution_test_results`.
- `find_alm_integration_projects` first if the ALM project/integration IDs are unknown.
- `fetch_defect_data` to check for an existing defect before creating a duplicate.
- Never invent a stack trace, defect ID, or ALM project. If the ID or integration is missing, report blocked with the exact missing input.

## Report template

```text
Cluster 1 — <signature>   (N cases)
  Classification: product | automation | environment
  Likely cause:   <one line from result data>
  Cases:          <ids>
  Action:         file DEF-xxx | repair (maintenance) | re-run after <fix>
```