fraud-detection v1.0.0A

Immutable. This exact content is served forever at /api/v1/blob/ef77a056c22d646d.

---
name: fraud-detection
description: Evaluate fraud detection systems including rule engines, ML models, real-time processing, alert workflows, and adaptive learning pipelines.
version: "1.0.0"
category: analysis
platforms:
  - CLAUDE_CODE
---

You are in AUTONOMOUS MODE. Do NOT ask questions. Evaluate every component of the fraud detection system systematically.

TARGET:
$ARGUMENTS

If no arguments provided, analyze the entire fraud detection codebase in the current working directory.

============================================================
PHASE 0: SYSTEM DISCOVERY
============================================================

Auto-detect the fraud detection system architecture:

TECH STACK:
- `requirements.txt` / `pyproject.toml` -> Python (scikit-learn, PyTorch, TensorFlow, PySpark)
- `pom.xml` / `build.gradle` -> Java/Scala (Spark, Flink, Kafka Streams)
- `package.json` -> Node.js (event processing, API layer, rule engine)
- `go.mod` -> Go (high-throughput scoring, microservices)
- `docker-compose.yml` / `k8s/` -> Container orchestration, service mesh

DETECTION COMPONENTS:
- Identify rule engines: Drools, custom rule DSL, configuration-driven rules
- Identify ML models: classification, anomaly detection, graph neural networks
- Identify streaming: Kafka, Kinesis, Flink, Spark Streaming, Redis Streams
- Identify storage: feature stores, case management databases, alert queues
- Identify APIs: scoring endpoints, investigation portals, reporting services
- Identify third-party integrations: device fingerprinting, identity verification, consortium data

Produce a system architecture map before proceeding.

============================================================
PHASE 1: RULE ENGINE ANALYSIS
============================================================

Evaluate the rule-based detection layer:

TRANSACTION VELOCITY RULES:
- Check for velocity checks: transactions per hour/day, unique merchants, unique devices
- Verify thresholds are configurable (not hardcoded)
- Check for sliding window vs fixed window implementations
- Verify time zone handling in velocity calculations
- Check for account-level vs card-level vs device-level velocity

AMOUNT THRESHOLD RULES:
- Check for single transaction amount limits
- Verify cumulative amount thresholds over time windows
- Check for round-number detection (common in money laundering)
- Verify currency conversion handling in multi-currency systems
- Check for structuring detection (transactions just below reporting thresholds)

GEOGRAPHIC ANOMALY RULES:
- Check for impossible travel detection (two transactions far apart in short time)
- Verify geolocation accuracy and fallback handling
- Check for high-risk country/region flagging
- Verify cross-border transaction rules
- Check IP geolocation vs card billing address comparison

DEVICE AND BEHAVIORAL RULES:
- Check for device fingerprinting integration
- Verify new device detection and risk scoring
- Check for session anomaly detection (unusual navigation, rapid clicks)
- Verify account takeover indicators (password change + large transaction)
- Check for bot detection rules

RULE MANAGEMENT:
- Verify rules can be updated without code deployment
- Check for rule versioning and audit trail
- Verify rule conflict detection (contradictory rules)
- Check for rule performance metrics (hit rate, false positive rate per rule)
- Verify rule testing/simulation capability before production deployment

For each finding: file path, rule component, severity, description, recommendation.

============================================================
PHASE 2: ML MODEL EVALUATION
============================================================

Evaluate the machine learning detection layer:

ANOMALY DETECTION:
- Identify anomaly detection algorithms (isolation forest, autoencoder, one-class SVM)
- Check training data composition (ratio of fraud to legitimate transactions)
- Verify unsupervised models are calibrated against known fraud patterns
- Check for seasonal and temporal pattern handling
- Verify anomaly score thresholds are optimized and documented

SUPERVISED CLASSIFICATION:
- Identify classification models (gradient boosting, random forest, neural networks)
- Check for class imbalance handling (SMOTE, undersampling, cost-sensitive learning)
- Verify model performance metrics: precision, recall, F1, AUC-PR (not just AUC-ROC)
- Check that precision-recall tradeoff is optimized for business objectives
- Verify holdout and temporal validation (no data leakage from future transactions)

FEATURE ENGINEERING:
- List all features used in ML models
- Check for aggregated features (rolling averages, velocity features, deviation from norm)
- Verify feature computation is consistent between training and real-time scoring
- Check for feature freshness — stale features in real-time scoring pipeline
- Verify feature importance analysis is performed and documented
- Check for feature store consistency (offline vs online feature values match)

MODEL ENSEMBLE:
- Check if multiple models are combined (stacking, voting, cascading)
- Verify ensemble logic is documented and tested
- Check model score aggregation methodology
- Verify individual model contribution is tracked

============================================================
PHASE 3: REAL-TIME PROCESSING EVALUATION
============================================================

Evaluate the real-time detection pipeline:

EVENT STREAMING:
- Identify streaming platform (Kafka, Kinesis, Flink, Pulsar)
- Check for message ordering guarantees within partitions
- Verify at-least-once or exactly-once processing semantics
- Check for dead letter queue handling on processing failures
- Verify backpressure handling under high load

LATENCY REQUIREMENTS:
- Measure or estimate end-to-end scoring latency
- Check for timeout handling in synchronous scoring paths
- Verify pre-auth vs post-auth scoring placement
- Check for latency SLA definitions and monitoring
- Verify graceful degradation when latency exceeds thresholds (default allow vs deny)

ALERT ROUTING:
- Check for alert priority/severity classification
- Verify routing rules direct alerts to appropriate queues or teams
- Check for real-time alert delivery (WebSocket, push notification, SMS)
- Verify alert deduplication logic (same fraud event does not spawn multiple alerts)
- Check for alert suppression rules (known good patterns, whitelisted entities)

SCALABILITY:
- Check for horizontal scaling capability under transaction spikes
- Verify partitioning strategy handles skewed data (high-volume merchants)
- Check for resource limits and auto-scaling configuration
- Verify performance under peak load scenarios

============================================================
PHASE 4: FALSE POSITIVE MANAGEMENT
============================================================

Evaluate false positive handling and feedback loops:

INVESTIGATION QUEUES:
- Check for priority-based queue management
- Verify queue assignment logic (round-robin, skill-based, workload-balanced)
- Check for SLA tracking on investigation time
- Verify case notes and evidence attachment capability
- Check for escalation workflows for complex cases

FEEDBACK LOOPS:
- Verify analyst decisions feed back into model training pipeline
- Check for labeled data quality controls (inter-annotator agreement)
- Verify feedback loop latency (how quickly decisions improve detection)
- Check for confirmation bias mitigation in labeling
- Verify ground truth data collection for cases resolved later

AUTO-RESOLUTION:
- Check for automatic resolution rules on low-risk alerts
- Verify auto-resolution criteria are documented and auditable
- Check that auto-resolved cases are sampled for quality review
- Verify auto-resolution does not suppress true positives

FALSE POSITIVE RATE TRACKING:
- Check for false positive rate monitoring by rule, model, and segment
- Verify customer friction metrics (legitimate transactions blocked)
- Check for customer experience impact measurement
- Verify false positive reduction targets and progress tracking

============================================================
PHASE 5: ADAPTIVE LEARNING AND ROBUSTNESS
============================================================

Evaluate the system's ability to adapt to evolving fraud:

MODEL RETRAINING:
- Check for scheduled retraining pipeline
- Verify retraining uses recent data (not just historical)
- Check for automated retraining triggers (performance degradation)
- Verify champion-challenger deployment for new models
- Check for retraining data quality validation

CONCEPT DRIFT DETECTION:
- Check for feature distribution monitoring (PSI, KL divergence)
- Verify model prediction distribution monitoring
- Check for automated drift alerts
- Verify drift response procedures are documented

ADVERSARIAL ROBUSTNESS:
- Check for adversarial testing of ML models
- Verify rule engine cannot be reverse-engineered from API responses
- Check for rate limiting on scoring endpoints to prevent probing
- Verify that model scores and thresholds are not exposed in API responses
- Check for detection of systematic probing patterns

NEW FRAUD PATTERN DETECTION:
- Check for unsupervised anomaly detection complementing supervised models
- Verify graph analysis for fraud ring detection
- Check for trend analysis on emerging fraud patterns
- Verify mechanism for rapid rule deployment for new attack vectors

============================================================
PHASE 6: REPORTING AND REGULATORY COMPLIANCE
============================================================

Evaluate regulatory reporting and case management:

SAR FILING:
- Check for Suspicious Activity Report (SAR) generation workflow
- Verify SAR narrative generation includes required elements
- Check for SAR filing deadline tracking (30-day requirement)
- Verify SAR data retention (5-year minimum)
- Check for continuing activity reporting on ongoing suspicious behavior

CTR REPORTING:
- Check for Currency Transaction Report generation (transactions > $10,000)
- Verify aggregate transaction calculation for CTR threshold
- Check for structuring detection alerts

CASE MANAGEMENT:
- Verify investigation case lifecycle tracking
- Check for evidence chain of custody
- Verify case documentation requirements enforcement
- Check for regulatory examination audit trail
- Verify case outcome analytics and reporting

REGULATORY REPORTING:
- Check for FinCEN reporting integration
- Verify regulatory report accuracy validation
- Check for report filing confirmation tracking
- Verify compliance with jurisdiction-specific requirements

============================================================
OUTPUT
============================================================

## Fraud Detection System Analysis Report

**System:** [name/description]
**Stack:** [detected technologies]
**Transaction Volume:** [estimated if detectable from configs]

### Summary

| Category | Status | Findings | Critical |
|----------|--------|----------|----------|
| Rule Engine | [PASS/WARN/FAIL] | N | N |
| ML Models | [PASS/WARN/FAIL] | N | N |
| Real-Time Processing | [PASS/WARN/FAIL] | N | N |
| False Positive Mgmt | [PASS/WARN/FAIL] | N | N |
| Adaptive Learning | [PASS/WARN/FAIL] | N | N |
| Reporting/Compliance | [PASS/WARN/FAIL] | N | N |

### Detection Coverage Matrix

| Fraud Type | Rule Coverage | ML Coverage | Detection Stage | Gap |
|------------|--------------|-------------|-----------------|-----|
| Card-not-present | | | | |
| Account takeover | | | | |
| Identity theft | | | | |
| Structuring | | | | |
| Friendly fraud | | | | |

### Detailed Findings

For each category with WARN or FAIL:

#### [Category Name]

| # | Severity | File | Description | Impact | Recommendation |
|---|----------|------|-------------|--------|----------------|

### Performance Assessment
- **Rule engine hit rate:** [findings]
- **ML model metrics:** [AUC, precision, recall summary]
- **False positive rate:** [findings]
- **Latency profile:** [findings]

### Adversarial Resilience
- **Probing resistance:** [assessment]
- **Rule reverse-engineering risk:** [assessment]
- **Model evasion risk:** [assessment]

### Remediation Priority
[Ordered list by detection gap severity and regulatory risk]

============================================================
NEXT STEPS
============================================================

After reviewing the analysis:
- "Run `/credit-risk` to analyze credit risk scoring models in the pipeline."
- "Run `/financial-compliance` to review KYC/AML regulatory compliance."
- "Run `/owasp` to audit the detection API and investigation portal for security."
- "Run `/arch-review` to evaluate system architecture for scalability and reliability."

============================================================
DO NOT
============================================================

- Do NOT modify any detection rules or model weights — this is an analysis skill.
- Do NOT trigger or test alerts against production systems.
- Do NOT access or display actual transaction data or customer PII.
- Do NOT expose detection thresholds, model scores, or rule logic in the output that could aid fraud.
- Do NOT skip the regulatory compliance phase even for internal-only systems.
- Do NOT assume ML model performance without checking validation methodology.
- Do NOT conflate correlation with causation in feature analysis — note confidence levels.