audit

What this skill does

# confide:audit — corpus-scale, stats-only PII audit

Measure how much PII lives across a whole folder of sessions, without ever exposing any
of it. The audit runs the layered LOCAL detector stack from `shared/confide_core.py`
(regex → Natasha → local LLM) over each file and emits **only aggregates**. This mirrors
the `real_session_eval` privacy contract: read text only in-process, emit counts.

## Privacy invariants (do not violate)
- **Local-only.** No cloud APIs. Raw transcript text never leaves the machine.
- **Stats-only output.** The report (markdown + json + optional HTML) contains ONLY
  counts and rates — never a transcript substring, never a detected PII value.
- **No filenames.** Per-file rows are keyed by anonymized ids `own-00`, `own-01`, …
  The original path/name is never written or printed. On an unreadable file, only the
  index + exception class name is recorded.
- **Safe to surface.** Because it is counts-only, the aggregate report can be shared
  with a cloud agent or pasted into a chat. The PII stays on the machine.

## What it reports
- `n_files`, total / mean / min / max document chars
- `spans_by_type` (PERSON, EMAIL, PHONE, DATE, …) and `spans_by_layer` (regex / natasha / llm)
- `overall_redaction_rate` plus the **per-session** redaction-rate distribution
  (min / median / mean / max)
- a **coarse residual proxy**: spans still detectable after redaction — ~0 on a clean RED
  corpus, a leakage signal on a GREEN corpus.

## Run it
Point it at a folder (recurses, processes every `.md`/`.txt`; skips confide's own
`*.green.md` / `*.stats.json` outputs):

```bash
python3 skills/audit/scripts/audit.py FOLDER
```

Options:
- `--list paths.txt` — also/instead audit absolute paths listed one per line.
- `--layers regex,natasha,llm` — choose detection layers (default from config).
  Use `--layers regex` for a fully offline, deterministic pass (no models/network).
- `--out report.md` — report path; a `report.json` sibling is written alongside.
- `--html` — also write a Tufte-ish dashboard (`report.html`, counts only).

Writes the markdown + json report (and optional HTML) and prints the aggregate summary —
all counts only.

## RED vs GREEN
- **RED (raw) corpus:** sizes the PII problem before any redaction.
- **GREEN (redacted) corpus:** the residual proxy and remaining `spans_by_type` tell you
  whether redaction is holding at scale.

## After running
1. Report the aggregate summary (file count, span totals by type/layer, redaction-rate
   distribution, residual proxy) — never paste PII.
2. If residual is non-trivial on a GREEN corpus, point the user at **confide:anon** to
   re-redact and **confide:red** to probe re-identification risk.

## Setup
Layer availability (Natasha, local LLM via Ollama) comes from config — run
**confide:setup** if they aren't installed. `--layers regex` always works offline.