octocode-news

What this skill does

# What's New — Tech Research Agent

**Goal**: Lock scope, sweep RSS + cataloged sources in parallel, research gaps, assemble a validated JSON report and an HTML report, then open the HTML in the default browser.

## Quick Input

Use defaults silently when the user did not provide a value. Only ask if the request is genuinely ambiguous.

1. **Domains**: A=AI, B=DevTools, C=Web/JS, D=Security, E=Repos. Default: all
2. **Window**: `24h` / `7d` / `14d` / `30d`. Default: `7d`
3. **Depth**: `brief` / `deep` / `comprehensive`. Default: `deep`

## Non-Negotiables

1. Run both discovery scripts before manual browsing.
2. Treat `references/sources.md` as the baseline catalog, not a suggestion.
3. Use official/product/project sources first; secondary sources validate or widen coverage.
4. Read the full canonical page before writing a kept item summary. RSS snippets are discovery-only.
5. When a source is a daily digest hub, open the dated daily post for the reporting window rather than summarizing the landing page.
6. Every kept item must include:
   - `summary` based on full-page content (120+ chars)
   - `whyImportant` (or legacy `whyInteresting`) explaining why the story matters now
   - `references` (at least one)
   - `contentEvidence.method = "full-page"` with `chars >= 200`
7. `topItems` are the hero stories — must not be duplicated inside `sections[].items`.
8. Record every checked, blocked, stale, or skipped source in `sourcesChecked` with `status`, `found`, and `notes`. Never silently drop a failed source.
9. Keep original item fields in JSON; add concise display fields (`shortTitle`, `shortDescription`) rather than replacing `title` or `summary`.
10. Low-heat stories still need useful editorial framing via `shortTitle` and `shortDescription`.
11. Deduplicate across RSS, manual browsing, and GitHub — same story from multiple sources counts once. Keep the richest version.
12. Finish only when all section JSONs exist, `meta.json` exists, repo verification (step 5) is done, HTML exists, and browser open was attempted.
13. When more than one domain is in scope, research each selected section/domain with its own subagent.
14. Run section subagents in parallel after discovery. Each subagent writes its own `{id}.json`. The coordinator owns dedupe, topItem selection, `meta.json`, and final HTML build.
15. Every subagent writes its section JSON file (`~/tmp/{ts}-sections/{id}.json`) following the `SectionPayload` schema and returns `coverageSummary`, `dedupeHints`, and `blockedOrStale` to the coordinator.
16. Subagents must not write `meta.json`, the validated JSON, or HTML files directly. Only the coordinator writes `meta.json` and runs `build-report`.

## Workflow

### 1) Lock Scope

Generate `{ts}` once as `YYYYMMDD-HHmmss` (e.g. `20260404-143000`) and reuse it for every output file path. Use defaults when missing: `domains=all`, `window=7d`, `depth=deep`.

If dependencies are missing: run `yarn install` from the monorepo root (this resolves all workspaces including skills).

### 2) Discovery (parallel)

Run both scripts in parallel — they are independent and both read `references/sources.md`:

**RSS fetch + health** — candidate pool, volume by domain, and per-feed health status in one pass:

```bash
yarn --cwd skills/octocode-news fetch-rss \
  --window {window} \
  --include-health \
  --json-out ~/tmp/{ts}-whats-new-rss.json
```

**Source catalog** — full coverage checklist (websites, RSS, repos, custom resources per domain):

```bash
yarn --cwd skills/octocode-news catalog-sources \
  --json-out ~/tmp/{ts}-whats-new-catalog.json
```

After both finish, you have:

- `rss.json` → `summary` shows volume by domain, `items` gives the candidate pool, and each `feeds[]` entry includes a `health` object (`ok`, `reason`, `validFeed`, freshness data) — broken/stale/empty feeds are flagged inline
- `catalog.json` → every website, RSS feed, repo, and custom resource to track in `sourcesChecked`

Note: `check-rss` still exists as a standalone script for targeted feed audits, but `--include-health` on `fetch-rss` covers the same ground in a single network pass.

### 3) Research & Write Per-Section JSON (parallel by section)

Create the output directory first:

```bash
mkdir -p ~/tmp/{ts}-sections/
```

**Resume protocol**: Before spawning subagents, check for existing section JSON files in `~/tmp/{ts}-sections/`. If a valid `{id}.json` already exists (has items or is a quiet stub), skip that domain — it was completed by a previous run. This allows recovery from coordinator crashes without re-doing finished domains.

Domains: `ai`, `devtools`, `web`, `security`, `repos`, optional `cross` (when a story clearly spans sections).

**Single-domain shortcut**: When only one domain is selected, skip subagent overhead — run the research pass inline as the coordinator. Write the section JSON file directly and proceed to Step 4.

**Quiet stubs for unrequested domains**: When `domains` is not `all`, write a quiet stub for each unselected primary domain so the HTML report renders every section:

```json
{ "id": "web", "name": "Web Platform", "icon": "W", "iconClass": "si-web", "quiet": true, "quietMsg": "Web platform was not in scope for this run.", "items": [], "sourcesChecked": [] }
```

When multiple domains are selected, spawn one subagent per domain. Each subagent runs the same four-step pass:

1. **Triage RSS candidates** — scan the RSS candidate pool from step 2 and mark clear top stories for full-page reads.
2. **Check non-RSS sources** — visit cataloged websites that do not expose RSS. Skip feeds flagged broken/stale by the health check.
3. **Validate repo/release claims** — use Octocode GitHub tools for release notes, merged PRs, changelogs, and repo context.
4. **Log everything** — record checked, blocked, stale, empty, and skipped sources. These become `sourcesChecked` entries.

Each subagent writes `~/tmp/{ts}-sections/{id}.json` following the `SectionPayload` schema:

```json
{
  "id": "ai",
  "name": "AI",
  "icon": "A",
  "iconClass": "si-ai",
  "quiet": false,
  "items": [ ... ],
  "sourcesChecked": [ ... ]
}
```

Required fields: `id`, `name`, `icon`, `iconClass`, `quiet`, `items`, `sourcesChecked`.
Set `quiet: true` + `quietMsg` + empty `items` when a domain has no notable items.

Each `items[]` entry follows the `Item` schema (domain, type, title, summary, heat, references, contentEvidence, etc.).
Each `sourcesChecked[]` entry follows the `SourceCheck` schema.

Each subagent also returns to the coordinator:
- `coverageSummary` — what was covered, what was quiet, what needs follow-up
- `dedupeHints[]` — URLs, repos, release pages, or products that may overlap another domain
- `blockedOrStale[]` — sources or candidate stories that need coordinator review

The coordinator waits for all selected subagents, then reads their JSON files for cross-domain dedup, topItem selection, and meta assembly.

Prefer Octocode for GitHub data, local scripts for catalog/RSS work, and direct web fetching for non-GitHub sources.

**Blocked-page fallback**: When a canonical URL cannot be fetched (challenge, paywall, timeout), use the RSS snippet + source metadata as `contentEvidence.method = "rss-snippet"` with `chars` set to actual snippet length. Log the reason in `sourcesChecked`. Do not silently drop the item — a well-sourced RSS summary with a clear `whyImportant` is better than nothing.

**Source tiers at brief depth**: When `depth=brief`, focus on P1 sources from `sources.md` only. Skip P2 sources unless P1 coverage is clearly thin (<3 items for a domain).

**Subagent stopping condition**: stop when the domain slice has met its depth floor or exhausted the cataloged sources for that domain, every checked/blocked/skipped source is logged, and the section JSON file is written.

**Subagent context-limit recovery**: If a subagent is approaching its context limit before finishing all sources, it must immediately write its section JSON with whatever items it has collected, mark r