doc-completeness-audit

What this skill does

# Documentation Completeness Audit

Determine whether a documentation set covers everything it should by building an
inventory of what *needs* documenting and comparing it to what *exists*. The output
is a prioritized gap report — not new documentation.

## When to Use

- After shipping a feature — verify docs cover the new surface area
- Before a release — ensure no undocumented public APIs, CLI flags, or config options
- When users or new hires report "I couldn't find docs for X"
- Periodic health check on doc coverage
- After running `doc-maintenance` (structural) and `doc-claim-validator` (accuracy) to go wider

## Quick Reference

| Resource | Purpose | Load when |
|----------|---------|-----------|
| `references/coverage-model.md` | Defines what "complete" means per doc type | Always (Phase 1) |

---

## Workflow Overview

```
Phase 1: Inventory   → Build the "should exist" list from code and config
Phase 2: Map         → Match inventory items to existing documentation
Phase 3: Classify    → Score each gap by audience impact
Phase 4: Report      → Produce the prioritized gap report
```

---

## Phase 1: Build the Inventory

Construct a list of everything that should be documented. Use four sources, checking
all of them:

### Source 1: Public Code Surface

Run the bundled inventory script to extract documentable surface area deterministically:

```bash
python3 skills/doc-completeness-audit/scripts/inventory.py --root . --json > inventory.json

# Or human-readable:
python3 skills/doc-completeness-audit/scripts/inventory.py --root .

# Run specific detectors only:
python3 skills/doc-completeness-audit/scripts/inventory.py --root . --detectors env_vars,cli_commands
```

The script scans source files across Python, JavaScript/TypeScript, Rust, Go, Ruby, Java,
and shell, extracting six categories:

| Detector | What it extracts |
|----------|-----------------|
| `env_vars` | Environment variable references (`os.environ`, `process.env`, `env::var`, etc.) |
| `cli_commands` | CLI commands and flags (argparse, click, clap, cobra, commander) |
| `config_keys` | Configuration key access in config-related files |
| `http_endpoints` | HTTP route definitions (Flask, FastAPI, Express, Actix, Axum, net/http) |
| `public_exports` | Public module exports (`__init__.py`, `export`, `pub fn`, Go capitalized funcs) |
| `error_types` | Custom error/exception class definitions |
| Event types, webhooks, callbacks | Every event name and payload shape |

Dispatch an Explore agent to scan for these signals. Provide it with the project's
primary language and entry points.

### Source 2: User-Facing Features

Identify features a user interacts with:

- TUI screens, views, keybindings
- CLI workflows (multi-step operations)
- Integration points (hooks, plugins, extensions)
- Authentication/authorization flows
- Error messages that imply user action

### Source 3: Operational Surface

Identify what operators and maintainers need:

- Installation and setup procedures
- Upgrade and migration paths
- Backup and restore procedures
- Troubleshooting common errors
- Environment requirements and dependencies
- CI/CD integration points

### Source 4: Existing Docs Cross-References

Check existing docs for promises of documentation that doesn't exist:

- "See [link]" references to pages that don't exist
- "Coming soon" or "TODO" markers
- Table of contents entries without corresponding pages
- Navigation entries without targets

### Source 5: Architectural / Operational / Migration Topic Discovery (sonnet)

The first four sources catch *code-detectable* surface (env vars, CLI flags,
endpoints, exported APIs, broken cross-references). They miss topics that
exist as architectural patterns, user flows, ops procedures, or migration
paths but don't surface as a single greppable symbol. Examples:

- Architectural patterns the system implements (CQRS, event sourcing,
  saga) — should be documented but won't show up in inventory.py
- User flows implicit across UI surfaces — "how to share a project" may
  span multiple components and isn't a single CLI command
- Migration paths between versions — typically tribal knowledge until
  someone needs them
- Operational runbooks (incidents, rollbacks, capacity events)
- Recovery procedures and disaster scenarios

Dispatch one `general-purpose` + `sonnet` agent for topic discovery:

```
subagent_type: "general-purpose"
model: "sonnet"
description: "Architectural/operational topic discovery"
```

Prompt: read README, top-level docs, and a sample of code (architecture
files, integration boundaries, deployment configs, major feature
directories). Identify topics that *should* be documented but aren't
captured by the code-surface inventory. For each topic, name:

- `topic` — what needs documenting (one phrase)
- `evidence` — what in the codebase implies this topic exists (path:line citations)
- `audience` — who would read this (operators, contributors, advanced users)
- `type` — reference, tutorial, guide, explanation, runbook
- `confidence` — high (clear evidence), medium (inferred), low (speculative)

Append the agent's output to the inventory list before Phase 2.

**Output:** A structured inventory list. Each item has:
- `topic` — what needs documenting
- `source` — where the requirement was discovered (code path, config key, user flow, sonnet inference)
- `audience` — who needs this (end user, developer, operator)
- `type` — what kind of doc it needs (reference, tutorial, guide, explanation, runbook)
- `confidence` — high (deterministic) | medium | low (sonnet-inferred speculative)

---

## Phase 2: Map to Existing Documentation (per-docfile sonnet dispatch)

For each inventory item, determine whether it's documented and how well.
"Adequate coverage" requires reading surrounding context — a grep hit
doesn't tell you whether the topic is truly explained vs. just mentioned in
passing. Orchestrator-side execution would require reading every doc *N*
times (once per inventory item), which strains the context window.

### Dispatch strategy

Two-phase mapping:

1. **Bulk grep pass (orchestrator)** — for each inventory item, grep docs
   for the topic name. Build a candidate match map: which docs mention
   each topic.
2. **Per-docfile sonnet pass** — for each docfile that surfaced as a
   candidate match for any inventory item, dispatch one `general-purpose` +
   `sonnet` agent. The agent receives the doc + the list of inventory
   items that grep'd to this doc, and judges each as Documented / Shallow /
   Misplaced.

This keeps total agent calls ≈ N candidate docfiles (not N inventory items
× M docs). For a typical project with 100 inventory items and 50 docs, the
candidate map usually has 30–50 docs needing review.

### Per-docfile prompt template

```
subagent_type: "general-purpose"
model: "sonnet"
description: "Coverage mapping for <docfile>"
```

Prompt:
```
Read the doc at <DOCFILE_PATH>. The following inventory items grep-matched
this doc — judge each:

<INVENTORY_ITEMS_FOR_THIS_DOC>

For each item, classify as one of:
- Documented: dedicated section or page provides adequate coverage
- Shallow: mentioned but insufficient (missing examples, edge cases,
  parameter listings; flag-in-table without explanation)
- Misplaced: covered, but in the wrong doc type for the audience (API
  reference embedded in a tutorial; user-facing topic in dev-only docs)
- No real match: grep matched but the doc doesn't actually cover the topic
  (incidental mention, different concept with the same word)

Output as YAML:

doc_path: <path>
items_reviewed: N
classifications:
  - item: <topic>
    classification: Documented | Shallow | Misplaced | No real match
    section: <heading or line range where the topic is covered>
    evidence: <quote or paraphrase of the relevant content>
    gap: <if Shallow, what's missing; if Misplaced, where it should live>
```

### Items with no candidate match

Inventory items that grep'd 0 docs go directly to the "Missing" bucket
without