git:20260718.ab29907 to git:20260724.d4db756

---

name: soc2-evidence-collector

description: "Build-an-X for SOC 2 Type II evidence collection - per-Trust-Services-Criterion test artifacts (Common Criteria CC1.1 - CC9.2; plus Availability A1, Confidentiality C1, Processing Integrity PI1, Privacy P1 - P9 if in scope); auto-collection from CI logs + audit trails + access logs + change-management records; alignment with Vanta / Drata / Secureframe evidence shapes; observation-period sampling. Use when the team is preparing for SOC 2 Type II audit and needs continuous evidence collection automation."

---

# soc2-evidence-collector

## Overview

SOC 2 (Service Organization Control 2) is the AICPA-defined audit

framework for SaaS providers. Per AICPA Trust Services Criteria

(TSC):

| Category | TSC sections | Required? |

|---|---|---|

| Common Criteria | CC1 - CC9 (35 sub-criteria) | Always required |

| Availability | A1 | Optional (recommended for SaaS uptime claims) |

| Confidentiality | C1 | Optional (typical for B2B SaaS) |

| Processing Integrity | PI1 | Optional (common for transaction-processing SaaS) |

| Privacy | P1 - P9 | Optional (common when handling PII at scale) |

**Type I** = point-in-time design assessment.

**Type II** = continuous-evidence assessment over an observation

period (3 - 12 months).

Type II requires evidence collection across the period - auditors

sample, but every control should have continuous evidence available.

This is a **build-an-X workflow** - the per-criterion evidence

collection script, not a standalone tool. Pair with Vanta / Drata /

Secureframe (commercial GRC platforms) for evidence storage +

auditor-facing dashboards.

## When to use

- Pre-audit: Type II observation period started; need continuous

evidence.

- Pre-pre-audit: identifying which controls + evidence are

testable vs require manual attestation.

- Post-finding remediation: a previous audit flagged an evidence

gap.

- Adopting a GRC platform (Vanta/Drata/Secureframe) and need

evidence-feed configuration.

## Step 1 - Identify in-scope criteria

Most SaaS engagements include CC + Availability + Confidentiality.

Privacy criteria add when GDPR/CCPA also in scope. Processing

Integrity adds for fintech / data-processing SaaS.

| Criterion category | Typical scope decision |

|---|---|

| CC1 Control Environment | Always |

| CC2 Communication & Information | Always |

| CC3 Risk Assessment | Always |

| CC4 Monitoring | Always |

| CC5 Control Activities | Always |

| CC6 Logical & Physical Access | Always |

| CC7 System Operations | Always |

| CC8 Change Management | Always |

| CC9 Risk Mitigation | Always |

| A1 Availability | If uptime SLA committed |

| C1 Confidentiality | Typical for B2B SaaS |

| PI1 Processing Integrity | If data-processing accuracy matters |

| P1 - P9 Privacy | If handling PII at scale |

## Step 2 - Auto-collect evidence per criterion

Map each control to one or more automatable evidence sources:

| Control | Evidence source | Collector pattern |

|---|---|---|

| CC6.1 Logical access | IDP audit logs (Okta/Auth0/Keycloak) | Daily export of user-access events |

| CC6.2 Access provisioning | Onboarding workflow logs | Per-hire ticket + access-grant audit |

| CC6.3 Access deprovisioning | Offboarding workflow logs | Per-departure ticket + access-revoke audit |

| CC7.1 Threat detection | SIEM (Datadog, Splunk) alert logs | Continuous alert-history feed |

| CC7.2 System monitoring | APM (Datadog, New Relic) uptime data | Daily uptime report |

| CC8.1 Change management | Git PR history + CI deploy logs | Per-PR audit (reviewer attribution) |

| A1.1 Availability monitoring | SLO dashboards | Monthly availability report |

| C1.1 Encryption at rest | Cloud KMS audit logs | Quarterly attestation |

| C1.2 Encryption in transit | TLS config audit | Quarterly attestation |

Example collector script:

```python

# evidence/cc6_1_logical_access.py

import okta_client, datetime, json

def collect_cc6_1_evidence(start_date, end_date):

"""Per CC6.1: collect user-access audit events for the period."""

events = okta_client.get_audit_events(

type='user.session.start',

start_date=start_date,

end_date=end_date,

)

evidence = {

'control_id': 'CC6.1',

'period_start': start_date.isoformat(),

'period_end': end_date.isoformat(),

'evidence_type': 'user_access_logs',

'sample_size': len(events),

'events': events[:100], # auditor sample

'collected_at': datetime.datetime.utcnow().isoformat(),

'collector': 'soc2-evidence-collector v1.0',

}

with open(f'evidence/cc6_1_{start_date.date()}_{end_date.date()}.json', 'w') as f:

json.dump(evidence, f, indent=2)

```

## Step 3 - Per-control test patterns

Beyond raw evidence collection, write tests that verify the control

operates correctly:

```python

def test_cc6_3_offboarded_user_has_no_active_sessions():

"""CC6.3: deprovisioned users must lose all access immediately."""

user = User.objects.get(email='alice@example.com')

deprovision(user)

# Verify across all systems:

assert not okta_client.user_has_active_sessions(user)

assert not aws_iam.user_exists(user.aws_username)

assert not github_org.is_member(user)

assert not slack.is_member(user)

# Audit log records the deprovisioning event:

assert AuditLog.objects.filter(

actor='hr-system',

action='deprovision',

subject=user.email,

).exists()

```

These tests run in CI; their pass/fail history is itself evidence

for the auditor.

## Step 4 - GRC platform alignment

**Default: Vanta** - the broadest native-integration coverage (AWS / Okta / GitHub / GSuite / etc.) means the auto-collected evidence (Step 2) only needs to fill gaps the integrations don't cover. Use the alternatives when Vanta doesn't fit:

| Platform | Use when |

|---|---|

| Vanta (default) | Standard SaaS stack with mainstream identity / cloud / source-control providers |

| Drata | Multi-framework engagement (SOC 2 + ISO 27001 + HIPAA) where Drata's templates lead |

| Secureframe | Budget-constrained engagement where Vanta's pricing is prohibitive |

Across all three, evidence ingest format is platform-specific but the auto-collected JSON (Step 2) feeds the platform's manual-upload UI when no native integration exists for your tooling.

## Step 5 - Observation period sampling

Type II auditors typically request:

- Population list (all instances of a control event during the

period - e.g., all PRs merged, all access-grants)

- Sample (auditor selects 25 - 40 random instances)

- Per-sample evidence (the specific logs, tickets, approvals)

Your evidence collector should support both:

- Population queries (`SELECT * FROM audit_log WHERE date BETWEEN ...`)

- Per-instance evidence retrieval (full context for one event)

## Step 6 - Continuous-monitoring controls

Some controls are continuous (e.g., CC7.1 threat detection) - the

evidence is an alert-history feed, not point-in-time samples.

Pattern: daily collector cron job that:

1. Queries the source system for the previous day's events

2. Stores in append-only evidence storage (S3 with versioning,

immutable)

3. Records collector-run metadata (when, what, how many records)

Continuity gaps in collector runs are themselves audit findings -

make collector failures alert-worthy.

## Step 7 - End-to-end recipe

For each in-scope criterion:

1. ✅ Map criterion to evidence source(s)

2. ✅ Implement automated collector (Step 2)

3. ✅ Write per-control test (Step 3)

4. ✅ Wire evidence into GRC platform (Step 4)

5. ✅ Verify continuous-collection has no gaps (Step 6)

6. ✅ Run a mock auditor query (request a sample; verify response is complete + timely)

## Anti-patterns

| Anti-pattern | Why it fails | Fix |

|---|---|---|

| Manual evidence collection only | Doesn't scale across observation period; misses sampling intervals | Automated collector (Step 2) |

| Trust the auditor will only sample what we expect | Audit fails on unexpected sample request | Continuous full-population collection (Step 6) |

| Evidence stored in mutable storage | Tampering risk; audit invalidated | Append-only / immutable storage (Step 6) |

| Test pass-history not preserved | Loses control-effectiveness evidence | Persist test results for the period |

| Skip mock-audit dry runs | First real audit reveals gaps | Mock-audit before observation period (Step 7) |

## Limitations

- This is a build-an-X workflow. Tests use the team's existing

test framework + cloud APIs.

- SOC 2 has many criteria; this skill is a starting framework, not

a complete control library.

- Trust Services Criteria evolve (current TSP 2017, revised 2022);

pin version per audit engagement.

- GRC platforms are commercial; OSS alternatives (e.g.,

comply-soc2 / Drata-compatible scripts) exist but are

less polished.

- This skill doesn't replace a SOC 2 readiness consultant.

## References

- aicpa.org/topic/audit-assurance/audit-and-assurance-greaterthan-suitable-trust-services-criteria - AICPA Trust Services Criteria (paywalled; free abstract)

- vanta.com/solutions/soc-2 - Vanta SOC 2 product

- drata.com/grc-central/soc-2 - Drata SOC 2 reference

- secureframe.com/hub/soc-2 - Secureframe SOC 2 hub

sister test-pattern catalogs