security-auditing

What this skill does

# Security Auditing Skill

---
name: security-auditing
version: 1.0.0
domain: security/compliance
risk_level: HIGH
languages: [python, go, typescript]
frameworks: [structlog, opentelemetry, falco]
requires_security_review: true
compliance: [GDPR, HIPAA, PCI-DSS, SOC2, ISO27001]
last_updated: 2025-01-15
---

> **MANDATORY READING PROTOCOL**: Before implementing audit logging, read `references/advanced-patterns.md` for tamper-evident patterns and `references/threat-model.md` for log integrity attacks.

## 1. Overview

### 1.1 Purpose and Scope

This skill provides security auditing and compliance capabilities:

- **Tamper-Evident Logging**: Cryptographically signed audit trails
- **SIEM Integration**: Forward events to security monitoring systems
- **Vulnerability Assessment**: Automated security scanning and reporting
- **Compliance Reporting**: Generate audit reports for regulations

### 1.2 Risk Assessment

**Risk Level**: HIGH

**Justification**:
- Audit logs are evidence in incident investigations
- Log tampering hides attacker activity
- Compliance violations result in legal penalties
- Missing logs = blind spots in security monitoring

**Attack Surface**:
- Log injection attacks
- Log tampering/deletion
- SIEM misconfiguration
- Sensitive data in logs (PII leakage)
- Log storage exhaustion

## 2. Core Responsibilities

### 2.1 Primary Functions

1. **Generate tamper-evident audit logs** for security events
2. **Forward events to SIEM** for correlation and alerting
3. **Assess vulnerabilities** through automated scanning
4. **Produce compliance reports** for regulatory requirements
5. **Detect anomalies** in user behavior and system activity

### 2.2 Core Principles

- **TDD First**: Write tests for security checks before implementation
- **Performance Aware**: Use incremental scanning and caching for efficiency
- **NEVER** log sensitive data (passwords, PII, secrets)
- **NEVER** trust log data without integrity verification
- **ALWAYS** use structured logging (JSON)
- **ALWAYS** include correlation IDs for request tracing
- **ALWAYS** protect logs from unauthorized modification

## 3. Technology Stack

| Component | Recommended | Purpose |
|-----------|-------------|---------|
| Structured Logging | `structlog` (Python) | JSON log generation |
| Log Aggregation | Elasticsearch, Loki | Centralized storage |
| SIEM | Splunk, QRadar, Sentinel | Security monitoring |
| Integrity | Signed logs, WORM storage | Tamper evidence |
| Compliance | OpenSCAP, Prowler, Trivy | Assessment tools |

## 4. Implementation Patterns

### 4.1 Tamper-Evident Audit Logging (Summary)

```python
import hashlib
import hmac
import json
from datetime import datetime, timezone

class TamperEvidentLogger:
    """Audit logger with cryptographic integrity protection."""

    def __init__(self, signing_key: bytes, output_path: str):
        self._key = signing_key
        self._path = output_path
        self._sequence = 0
        self._previous_hash = b'\x00' * 32

    def log(self, event: str, actor: str = None, **context) -> dict:
        """Log a tamper-evident audit entry."""
        self._sequence += 1

        entry = {
            'timestamp': datetime.now(timezone.utc).isoformat(),
            'sequence': self._sequence,
            'event': event,
            'actor': actor,
            'context': context,
            'previous_hash': self._previous_hash.hex(),
        }

        # Calculate and sign
        entry_bytes = json.dumps(entry, sort_keys=True).encode()
        entry['hash'] = hashlib.sha256(entry_bytes).hexdigest()
        entry['signature'] = hmac.new(
            self._key, entry_bytes, hashlib.sha256
        ).hexdigest()

        self._previous_hash = bytes.fromhex(entry['hash'])

        with open(self._path, 'a') as f:
            f.write(json.dumps(entry) + '\n')

        return entry
```

**📚 For complete implementation** (verification, chain validation):
- See `references/advanced-patterns.md`

### 4.2 Structured Security Logging

```python
import structlog

logger = structlog.get_logger()

class SecurityAuditLogger:
    """Security-focused audit logging."""

    @staticmethod
    def log_authentication(user_id: str, success: bool, method: str, ip: str):
        """Log authentication attempt."""
        logger.info(
            "auth.attempt",
            user_id=user_id,  # Never log email for privacy
            success=success,
            method=method,
            ip_address=ip
        )

    @staticmethod
    def log_authorization(user_id: str, resource: str, action: str, allowed: bool):
        """Log authorization decision."""
        logger.info(
            "authz.decision",
            user_id=user_id,
            resource=resource,
            action=action,
            allowed=allowed
        )

    @staticmethod
    def log_data_access(user_id: str, resource_type: str, resource_id: str, action: str):
        """Log data access for compliance."""
        logger.info(
            "data.access",
            user_id=user_id,
            resource_type=resource_type,
            resource_id=resource_id,
            action=action
        )
```

**📚 For complete patterns** (decorators, context managers, SIEM integration):
- See `references/security-examples.md`

### 4.3 SIEM Integration (CEF Format)

```python
class SIEMForwarder:
    def _to_cef(self, event: dict) -> str:
        """Convert event to CEF format for SIEM ingestion."""
        severity = self._map_severity(event.get('level', 'INFO'))
        return (f"CEF:0|JARVIS|SecurityAudit|1.0|{event.get('event', 'unknown')}|"
                f"{event.get('event', 'Unknown Event')}|{severity}|"
                f"src={event.get('ip_address', '')} suser={event.get('user_id', '')}")
```

**📚 For full SIEM implementation**: See `references/security-examples.md#siem-integration`

### 4.4 Vulnerability Assessment

```python
from dataclasses import dataclass
from typing import List

@dataclass
class Vulnerability:
    id: str
    severity: str
    package: str
    fixed_version: str

class VulnerabilityScanner:
    def scan_dependencies(self, path: str) -> List[Vulnerability]:
        """Scan dependencies using pip-audit, trivy for containers."""
        pass
```

**📚 For complete scanner**: See `references/advanced-patterns.md#vulnerability-assessment`

## 5. Implementation Workflow (TDD)

### Step 1: Write Failing Test First

```python
import pytest
from security_auditing import TamperEvidentLogger, SecurityAuditLogger

class TestTamperEvidentLogger:
    def test_log_entry_contains_required_fields(self, tmp_path):
        """Each log entry must have timestamp, sequence, hash, signature."""
        logger = TamperEvidentLogger(b'test-key', str(tmp_path / 'audit.log'))
        entry = logger.log("user.login", actor="user123")
        assert all(k in entry for k in ['timestamp', 'sequence', 'hash', 'signature'])

    def test_chain_integrity_detects_tampering(self, tmp_path):
        """Tampered logs must be detected via chain validation."""
        log_path = tmp_path / 'audit.log'
        logger = TamperEvidentLogger(b'test-key', str(log_path))
        logger.log("event1", actor="user1")

        # Tamper with log file
        tampered = log_path.read_text().replace('"event1"', '"TAMPERED"')
        log_path.write_text(tampered)

        valid, errors = logger.verify_chain()
        assert not valid and len(errors) > 0

    def test_no_pii_in_log_output(self, tmp_path):
        """PII patterns must not appear in logs."""
        import re
        log_path = tmp_path / 'audit.log'
        logger = SecurityAuditLogger(str(log_path))
        logger.log_authentication(user_id="user123", success=True, method="password", ip="192.168.1.1")
        content = log_path.read_text()
        assert not re.search(r'[\w\.-]+@[\w\.-]+', content)  # No emails
```

### Step 2: Implement Minimum to Pass

```python
# Implement only what's needed to pass the tests
class TamperEvidentLogger:
    def __init__(se