playwright-execute · git:20260725.11dd3e7 · 2026-07-25 · sha256 1f8f32238eca7c27

playwright-execute git:20260725.11dd3e7A

Immutable. This exact content is served forever at /api/v1/blob/1f8f32238eca7c27.

---
description: Run Playwright tests or suites and upload the resulting report to Katalon True Platform. Use when you need to execute Playwright scripts, package scripts, spec files, projects, or suites, configure or verify @katalon/playwright-reporter, upload Playwright reports with Katalon CLI/reporter commands, and verify uploaded Katalon Platform runs or reports.
alwaysApply: false
---

# Katalon Playwright Execute

Use this skill when the user asks to execute a Playwright script/suite and send results to Katalon True Platform. The workflow is not complete until local execution finishes, report artifacts are collected, and upload is either verified or clearly blocked.

## Execution Workflow

## Required Context

Resolve or ask for only missing required values:

- Playwright target: spec file, grep, project, package script, suite command, or user-provided command.
- Repository/root directory that contains `package.json` and Playwright config.
- Katalon Platform project ID. Use Katalon MCP `list_projects` when available.
- Katalon API key from a secure source such as `KATALON_API_KEY`; check whether it is already available before asking.
- Platform base URL only when non-default or when the project uses QA/prod custom domains.
- Whether the agent may modify Playwright config/package files if the Katalon reporter is missing.

Treat the human as a tool: ask concise questions when target, credentials, project, or permission cannot be resolved safely. Never ask the user to paste secrets into normal chat when a secure secret mechanism or environment variable can be used.

## Preflight

Before running:

- Inspect the repository for `package.json`, `playwright.config.*`, `@playwright/test`, existing reporters, package manager lockfiles, and scripts.
- Verify Node.js is version 18 or newer when possible.
- Check for `KATALON_API_KEY` in the current process environment or the repo's approved secret mechanism before composing any upload command.
- If `KATALON_API_KEY` is missing, stop before execution/upload and ask the user to provide it through a secure environment variable or secret mechanism.
- Run dependency installation only when the repo convention is clear and the user has allowed dependency changes.
- Confirm `@katalon/playwright-reporter` is installed or can be installed.
- Verify a deterministic report output directory, preferably `./reports` or the repo's existing Katalon report directory.
- Build commands with secrets redacted in user-facing output.

Read `references/playwright-katalon-reporting.md` before editing config or composing upload commands.

## Configure Reporter

Prefer the existing project convention. If the Katalon reporter is absent and the user allowed edits:

- Add `@katalon/playwright-reporter` as a dev dependency.
- Add `getGlobalSetupPath()` from `@katalon/playwright-reporter` to Playwright `globalSetup` unless the existing config already has a global setup that must be preserved.
- Add the Katalon reporter with `outputDir`.
- Configure platform upload through environment variables, not hard-coded values.
- Preserve existing reporters such as `list`, `html`, `junit`, or custom reporters.

Do not overwrite unrelated Playwright projects, retries, timeouts, use settings, web servers, or CI-specific behavior.

## Run And Upload

Use one of these modes:

- Upload during execution: set `KATALON_API_KEY` and `KATALON_PROJECT_ID`, then run the selected Playwright command.
- Separate upload: run Playwright to generate report files, then run `playwright-reporter upload`.
- Existing report upload: skip execution only when the user explicitly provides an existing Playwright reporter output directory.

Prefer upload during execution when the project already has the Katalon reporter configured. Prefer separate upload when the user asks to run first, upload later, or when troubleshooting import failures.

If the Playwright command fails after generating report artifacts, still attempt upload when the reporter output exists unless the user asked to skip upload on failure. Test failures are valid report data.

## Platform Verification

After upload:

- Use Katalon MCP tools to list/read recent executions, test results, or quality data when available.
- Match by project ID, run time, test names, and framework metadata.
- Resolve the Platform result URL from reporter output, upload output, MCP execution data, or the known project/execution path.
- Always provide the best available Platform result URL so the user can open the uploaded run/report.
- If MCP cannot verify, inspect CLI output for upload success and report that Platform verification is blocked.
- If upload is asynchronous/importing, poll when a read/list tool exists; otherwise report the importing state.

## Response Contract

Report:

- Playwright command run and exit status, with secrets redacted.
- Katalon upload mode: during execution, separate upload, or existing report upload.
- Local report output directory and key artifacts.
- Platform upload status: verified, importing, failed, or not verifiable.
- Platform report/run URL for the user to open when returned or discoverable; if no exact URL is available, provide the closest project reports/executions URL and state why it is not exact.
- Passed, failed, skipped, and timed-out counts when available.
- Failed tests with concise reasons.
- Any required follow-up: missing API key, missing project ID, install blocked, config permission missing, upload failed, or MCP verification unavailable.

Do not claim success if Playwright ran locally but the Katalon upload did not happen or cannot be verified.

---

## Bundled references

_The reference material the skill points to is inlined below so this file is self-contained._

### references/playwright-katalon-reporting.md

# Playwright Katalon Reporting

## Official References

- Playwright Integration: https://docs.katalon.com/katalon-platform/integrations/testing-framework/playwright-integration
- Katalon API Key: https://docs.katalon.com/katalon-platform/administer/profile/katalon-api-key-in-katalon-testops
- Katalon CLI Integration: https://docs.katalon.com/katalon-platform/integrations/ci-cd/katalon-cli-integration

## Dependencies

Required:

- Node.js 18+
- `@playwright/test` 1.0+
- `@katalon/playwright-reporter`
- `KATALON_API_KEY` for upload
- `KATALON_PROJECT_ID` for upload

Install with the repo package manager:

```bash
npm install --save-dev @katalon/playwright-reporter
pnpm add -D @katalon/playwright-reporter
yarn add -D @katalon/playwright-reporter
```

Use the package manager already used by the repo. Do not install dependencies globally unless the repo already does that.

## API Key Check

Before running or uploading, check whether the API key is already available:

```bash
test -n "$KATALON_API_KEY" && echo "KATALON_API_KEY is set" || echo "KATALON_API_KEY is missing"
```

Do not print the key value. If missing, ask the user to provide it through a secure environment variable or secret mechanism, for example by setting `KATALON_API_KEY` in the shell/session. Stop before upload until the key is available.

## Playwright Config Pattern

Add the reporter without removing existing reporters:

```ts
import { defineConfig } from '@playwright/test';
import { getGlobalSetupPath } from '@katalon/playwright-reporter';

export default defineConfig({
globalSetup: getGlobalSetupPath(),
reporter: [
['list'],
[
'@katalon/playwright-reporter',
{
outputDir: './reports',
...(process.env.KATALON_API_KEY && {
platform: {
apiKey: process.env.KATALON_API_KEY,
projectId: process.env.KATALON_PROJECT_ID,
baseUrl: process.env.KATALON_BASE_URL,
authUrl: process.env.KATALON_AUTH_URL,
},
}),
},
],
],
});
```

When an existing `globalSetup` exists, preserve it. If combining setup is non-trivial, ask before editing and prefer separate upload after execution.

## Run Commands

Run the user's target command when provided. Otherwise use the narrowest Playwright command:

```bash
npx playwright test
npx playwright test tests/example.spec.ts
npx playwright test --grep "TC-07"
npx playwright test --project chromium
```

Upload during execution:

```bash
KATALON_API_KEY="$KATALON_API_KEY" \
KATALON_PROJECT_ID="<project_id>" \
KATALON_BASE_URL="<platform_url>" \
npx playwright test <target>
```

Omit `KATALON_BASE_URL` when the default Katalon endpoint is correct.

## Separate Upload

Use this when the report already exists or when run and upload are separate:

```bash
npx playwright test <target>
playwright-reporter upload \
--output ./reports \
--api-key "$KATALON_API_KEY" \
--project-id "<project_id>" \
--base-url "<platform_url>"
```

Omit `--base-url` when the default endpoint is correct.

## Secret Handling

- Never print `KATALON_API_KEY` or auth tokens.
- Do not write API keys to `playwright.config.*`, `package.json`, logs, reports, or skill files.
- Prefer environment variables, shell secret injection, or the platform's secure secret mechanism.
- Redact secret-like values before summarizing command output.

## Result URL

After upload, return a URL the user can open:

- Prefer an exact URL returned by `@katalon/playwright-reporter` or `playwright-reporter upload`.
- If MCP can find the execution/report, use that execution/report path.
- If only `project_id` is known, provide the closest Platform project reports or executions URL and mark it as a fallback.
- For custom Platform domains, build links from `KATALON_BASE_URL`; otherwise use the default Katalon Platform base URL expected by the reporter.

Do not finish with only "uploaded successfully"; include the result URL or explicitly state why no URL could be resolved.

## Common Failure Handling

- Missing dependency: install `@katalon/playwright-reporter` only with permission to modify dependencies.
- Missing API key: stop before upload and ask for a secure secret source.
- Missing project ID: use Katalon MCP `list_projects`; ask only if multiple possible projects remain.
- Playwright test failures: still upload reports if artifacts exist.
- Reporter output missing: inspect Playwright config and stdout; do not run separate upload without an output directory.
- Upload returns success but MCP cannot find a run: report upload output and state that Platform verification is not available from current tools.