claude-skills-reference/ra-qm-team/soc2-compliance/references/evidence_collection_guide.md

SOC 2 Evidence Collection Guide

Practical guide for collecting, organizing, and maintaining audit evidence for SOC 2 Type I and Type II engagements. Covers evidence types, automation strategies, and documentation requirements.

---

Evidence Fundamentals

What Auditors Look For

1. Existence — The control is documented and exists
2. Design effectiveness — The control is designed to address the TSC criterion (Type I + Type II)
3. Operating effectiveness — The control operates consistently over the observation period (Type II only)

Evidence Quality Criteria

Criterion	Description
Relevant	Directly demonstrates the control's operation
Reliable	Generated by systems or independent parties (not self-reported)
Timely	Falls within the audit/observation period
Sufficient	Enough samples to demonstrate consistency
Complete	Covers the full population or a representative sample

Evidence Types

Type	Description	Examples
Inquiry	Verbal or written descriptions from personnel	Interview notes, written responses
Observation	Auditor witnesses control in operation	Process walkthroughs, live demonstrations
Inspection	Review of documents, records, or configurations	Policy documents, system screenshots, logs
Re-performance	Auditor re-executes the control to verify results	Access review validation, configuration checks

---

Evidence by Control Area

Access Management

Control	Type I Evidence	Type II Evidence
Access provisioning	Provisioning policy, role matrix	Sample provisioning tickets with approvals (full period)
Access removal	Termination checklist, deprovisioning SOP	Sample termination events with access removal timestamps
Access reviews	Review policy, review template	Completed quarterly access review reports with sign-offs
MFA enforcement	MFA policy, configuration screenshot	MFA enrollment report showing 100% coverage
Privileged access	Privileged access policy, admin list	Quarterly privileged access reviews, admin activity logs

Change Management

Control	Type I Evidence	Type II Evidence
Change authorization	Change management policy, workflow description	Sample change tickets with approvals, peer reviews
Testing requirements	Testing policy, test plan template	Test results for sampled changes, QA sign-offs
Emergency changes	Emergency change procedure	Emergency change tickets with post-hoc approvals
Deployment process	CI/CD documentation, deployment runbook	Deployment logs, rollback records
Code review	Code review policy	Pull request histories showing reviewer approvals

Incident Response

Control	Type I Evidence	Type II Evidence
IR plan	Incident response plan document	Plan review/update records, version history
IR testing	Tabletop exercise schedule	Tabletop exercise reports, lessons learned
Incident handling	Triage procedures, classification criteria	Incident tickets with timestamps, escalation records
Postmortems	Postmortem template, review process	Completed postmortem documents, follow-up actions
Communication	Communication plan, stakeholder list	Notification records, status page updates

Vulnerability Management

Control	Type I Evidence	Type II Evidence
Scanning	Scanning schedule, tool configuration	Scan reports covering the full period (weekly/monthly)
Remediation SLAs	Remediation policy with SLA definitions	Remediation tracking showing SLA compliance rates
Patch management	Patching policy, schedule	Patch records, before/after scan comparisons
Penetration testing	Pentest policy, scope definition	Pentest reports (annual), remediation records

Encryption and Data Protection

Control	Type I Evidence	Type II Evidence
Encryption at rest	Encryption policy, configuration docs	Configuration screenshots, encryption audit reports
Encryption in transit	TLS policy, minimum version requirements	TLS scan results, certificate inventory
Key management	Key management policy, rotation schedule	Key rotation logs, access records for key stores
DLP	DLP policy, tool configuration	DLP alert logs, incident records, exception approvals

Backup and Recovery

Control	Type I Evidence	Type II Evidence
Backup procedures	Backup policy, schedule, retention rules	Backup success/failure logs (daily), retention compliance
DR planning	DR plan, recovery procedures	DR plan review records, update history
DR testing	DR test schedule, test plan	DR test reports with RTO/RPO measurements
BCP	BCP document, communication tree	BCP review records, test results

Monitoring and Logging

Control	Type I Evidence	Type II Evidence
SIEM/logging	Logging policy, SIEM configuration	Log retention evidence, alert samples, dashboard screenshots
Alert management	Alert rules, escalation procedures	Alert trigger samples, response records
Uptime monitoring	Monitoring tool configuration, SLA definitions	Uptime reports covering the full period
Anomaly detection	Detection rules, baseline configuration	Detection events, investigation records

Policy and Governance

Control	Type I Evidence	Type II Evidence
Security policies	Policy library, version control	Policy acknowledgment records, annual review evidence
Security training	Training program description, content	Training completion records (all employees)
Risk assessment	Risk assessment methodology	Annual risk assessment report, risk register updates
Board oversight	Committee charter, reporting schedule	Board meeting minutes, security reports to leadership

Vendor Management

Control	Type I Evidence	Type II Evidence
Vendor inventory	Vendor register, classification criteria	Current vendor register with risk tiers
Vendor assessment	Assessment questionnaire, criteria	Completed assessments, vendor SOC reports collected
Contractual controls	DPA template, security requirements	Signed DPAs, contract review records
Ongoing monitoring	Monitoring schedule, reassessment triggers	Reassessment records, monitoring reports

---

Evidence Automation

Automated Evidence Sources

Evidence	Automation Approach	Tools
Access reviews	Scheduled IAM exports, automated review workflows	Okta, Azure AD, AWS IAM + Jira/ServiceNow
Configuration compliance	Infrastructure-as-code, policy-as-code scanning	Terraform, OPA, AWS Config, Azure Policy
Vulnerability scans	Scheduled scanning with report auto-generation	Nessus, Qualys, Snyk, Dependabot
Change management	Git-based audit trails (commits, PRs, approvals)	GitHub, GitLab, Bitbucket
Uptime monitoring	Continuous synthetic monitoring with SLA dashboards	Datadog, New Relic, PagerDuty, Pingdom
Backup verification	Automated backup validation and restore tests	AWS Backup, Veeam, custom scripts
Training completion	LMS with automated tracking and reminders	KnowBe4, Curricula, custom LMS
Policy acknowledgment	Digital signature workflows with tracking	DocuSign, HelloSign, internal tools

Evidence Collection Script Pattern

1. Define evidence requirements per control
2. Map each requirement to a data source (API, log, screenshot)
3. Schedule automated collection (daily/weekly/monthly)
4. Store evidence with timestamps in a central repository
5. Generate collection status dashboard
6. Alert on missing or overdue evidence

Evidence Repository Structure

evidence/
├── {year}-{audit-period}/
│   ├── access-management/
│   │   ├── quarterly-access-review-Q1.pdf
│   │   ├── quarterly-access-review-Q2.pdf
│   │   ├── mfa-enrollment-report-2025-03.png
│   │   └── provisioning-samples/
│   ├── change-management/
│   │   ├── change-ticket-samples/
│   │   └── deployment-logs/
│   ├── incident-response/
│   │   ├── ir-plan-v3.2.pdf
│   │   ├── tabletop-exercise-2025-06.pdf
│   │   └── incident-tickets/
│   ├── vulnerability-management/
│   │   ├── scan-reports/
│   │   └── pentest-report-2025.pdf
│   ├── policies/
│   │   ├── information-security-policy-v4.pdf
│   │   └── acknowledgment-records/
│   └── vendor-management/
│       ├── vendor-register.csv
│       └── vendor-assessments/

---

Sampling Methodology

Auditors use sampling to test operating effectiveness. Understanding the methodology helps you prepare the right volume of evidence.

Sample Sizes by Control Frequency

Control Frequency	Population Size (per period)	Typical Sample Size
Annual	1	1 (all items)
Quarterly	4	2-4
Monthly	6-12	2-5
Weekly	26-52	5-15
Daily	180-365	20-40
Continuous/per-event	Varies	25-60

Key Sampling Rules

1. Higher frequency = larger sample — more occurrences mean more samples needed
2. Automated controls — typically only 1 sample needed if the system is validated
3. Exceptions must be explained — any deviation in a sample requires documentation
4. Population completeness — you must provide the full population for the auditor to select from

---

Type I vs Type II Evidence Differences

Aspect	Type I	Type II
Time scope	Single point in time	Entire observation period (3-12 months)
Volume	Lower — policies and configurations	Higher — ongoing logs, tickets, reports
Focus	"Is the control designed properly?"	"Did the control operate effectively?"
Exceptions	N/A	Must document and explain every exception
Owner sign-off	Policy approval records	Ongoing review sign-offs throughout the period

---

Common Evidence Pitfalls

Pitfall	Impact	Prevention
Screenshots without timestamps	Auditor cannot verify timing	Always include system clock or date stamps
Policies without version control	Cannot prove current vs outdated	Use document management with version tracking
Access reviews without sign-off	Cannot prove review was completed	Require digital approval/sign-off on every review
Gaps in monitoring data	Suggests control was not operating	Ensure logging continuity; document any outages
Evidence from wrong period	Does not cover the observation window	Verify date ranges before submission
Redacted evidence without explanation	Auditor may question completeness	Provide redaction rationale and methodology
Self-generated evidence only	Lower reliability in auditor's assessment	Include system-generated and third-party evidence
Missing exception documentation	Auditor flags as control failure	Document every exception with root cause and remediation