gateway

What this skill does

# Gateway Operations

The gateway daemon is the always-on pi session that receives events from Inngest functions, Telegram, and webhooks. It's the system's notification and communication layer.

**Rule: Always use `joelclaw gateway` CLI. Never use `launchctl`, `curl`, or log file grep directly.**

## CLI Commands

```bash
joelclaw gateway status    # Daemon availability, runtime mode, session pressure, Redis health
joelclaw gateway restart   # Roll daemon, clean Redis, fresh session
joelclaw gateway enable    # Re-enable launch agent + start daemon
joelclaw gateway test      # Push test event, verify delivery (Redis bridge path)
joelclaw gateway push --type <type> [--payload JSON]  # Push an event to all sessions
joelclaw gateway events    # Peek at pending events per session
joelclaw gateway drain     # Clear all event queues
joelclaw gateway stream    # NDJSON stream of all gateway events (ADR-0058)
joelclaw gateway channel {list|status|enable|disable}  # Enable/disable runtime channels
joelclaw gateway behavior {add|list|promote|remove|apply|stats}  # ADR-0211 behavior control plane
```

`joelclaw gateway restart` is the canonical restart. It kills the process, cleans Redis state, re-enables `com.joel.gateway` if launchd disabled it, waits for launchd to respawn, and verifies the new session. `joelclaw gateway enable` is the direct recovery path when launchd drift disabled the service. Never use `launchctl bootout/bootstrap` directly.

## Channel enable/disable

Use the CLI. Do not hand-edit `~/.joelclaw/scripts/gateway-start.sh` unless the CLI is broken.

```bash
joelclaw gateway channel list
joelclaw gateway channel disable discord --restart
joelclaw gateway channel enable discord --restart
joelclaw gateway channel status discord
```

The CLI blanks/restores the channel env assignments in `~/.joelclaw/scripts/gateway-start.sh`, writes a `/tmp/joelclaw/gateway-start.sh.*` backup before changes, and can restart the gateway when `--restart` is present. Telegram disable requires `--force` because it is a primary operator channel. Do not revoke or delete stored secrets for a temporary disable.

Expected disabled Discord state: component `disabled`; channel `configured:false`, `started:false`, `ready:false`, `botUserId:null`; health entry `status:"disabled"`.

## Quick Triage

Substrate precheck first (avoid chasing secondary gateway symptoms):

```bash
colima status --json
kubectl get nodes -o wide
kubectl get pods -n joelclaw redis-0 inngest-0
```

If Colima is down or node/core pods are not healthy, recover substrate before gateway operations.

Run in order, stop at first failure:

```bash
joelclaw gateway status    # 1. Is it alive? Sessions registered?
joelclaw gateway test      # 2. Can events flow end-to-end?
joelclaw gateway events    # 3. Are events piling up? (backpressure)
joelclaw gateway restart   # 4. If stuck, restart
```

If `joelclaw gateway status` shows pending > 0 on sessions, the agent is mid-stream or stuck. If it persists after a minute, restart.

## Redis-degraded mode (ADR-0214)

`joelclaw gateway status` now distinguishes:

- `mode: normal` — Redis bridge healthy
- `mode: redis_degraded` — daemon/channels/session available, but Redis-backed capabilities are degraded

When `mode=redis_degraded`:

- direct human conversation can still work
- Redis-backed commands/inspections are only partially trustworthy
- `joelclaw gateway test` validates the Redis bridge path, so expect `joelclaw gateway diagnose` to skip that layer intentionally
- use `joelclaw gateway diagnose` to see the degraded capability list and session pressure fields

Do not report `redis_degraded` as “gateway down” unless process/session health is also failing.

## Gateway context refresh scoping (ADR-0204)

Gateway rolling context refresh is useful only when it stays scoped.

Hard rule:
- do **not** treat automated digests, gateway event wrappers, recovery summaries, or terse acknowledgements as recall seeds
- if there is no real conversational topic, skip refresh rather than querying generic global memory

Why:
- a bad hidden `context-refresh` injection poisons the live gateway session even when `joelclaw gateway status` still reports healthy
- this showed up as unrelated voice/livekit notes bleeding into the gateway transcript

If Joel says the gateway session feels "fucked" while health checks look green, inspect the gateway session transcript for hidden `context-refresh` / `gateway-recovery` / `memory-recall` messages before trusting the CLI summary.

## Session pressure visibility (ADR-0218 rank 3 slice)

`joelclaw gateway status` / `joelclaw gateway diagnose` now expose session-pressure specifics instead of just a coarse health word:

- context usage % + next action
- next threshold summary (`compact at 65% ...` / `rotate at 75% ...` / `rotate immediately`)
- last compaction age + session age
- thread counts (`active` / `warm` / `total`)
- fallback state + activation count + consecutive prompt failures
- pressure reasons (`context_usage`, `context_ceiling`, `compaction_gap`, `session_age`)
- last alert health/time + cooldown state

The daemon emits OTEL under `daemon.session-pressure` (`session_pressure.alert.suppressed|failed`). Session-pressure states do **not** page Telegram; rotation/compaction pressure stays in status/diagnose/OTEL because it is gateway maintenance, not Joel action.

Idle maintenance is autonomous for time-based pressure:
- watchdog evaluates idle session pressure even when no turn is active
- overdue compaction gaps trigger autonomous compaction instead of waiting for the next human/event turn
- age-triggered rotation can also happen from the watchdog path; because Pi removed `AgentSession.newSession()`, gateway writes `/tmp/joelclaw/gateway.force-new-session.json` and exits cleanly so launchd restarts into a fresh `SessionManager`, then injects the compression summary as hidden context before the next inbound turn
- those watchdog-triggered runs emit the same `daemon.maintenance.started|completed|failed` telemetry as turn-bound maintenance

## Interruptibility and supersession (ADR-0196 / ADR-0218 rank 4 slice)

For direct human turns across Telegram, Discord, iMessage, and Slack invoke paths, the latest message now wins.

Runtime contract:

- new human turns get a short `1.5s` batching window before dispatch
- batching is per source, so rapid follow-ups collapse into one queued prompt
- if that source is already active, gateway supersedes immediately instead of waiting on the timer
- stale queued prompts from that source are dropped
- durable queue replay must not self-drop the freshest human message; supersession only applies when a genuinely newer same-source message exists
- daemon requests `session.abort()` on the stale turn
- stale response text is suppressed instead of being delivered late
- `joelclaw gateway status` exposes `supersession` plus `supersession.batching`
- `joelclaw gateway diagnose` adds an `interruptibility` layer with supersession and batching details

Passive intel / background event routes are excluded from this path.

## Operator ack/timeout tracing (ADR-0218 rank 5 slice)

Telegram operator actions now get trace ids and explicit lifecycle tracking.

Current covered paths:
- `cmd:*` command-menu callbacks
- `worktree:*` callbacks
- `pitch:*` ADR pitch callbacks
- default Telegram callback actions and external callback-route handoffs
- direct Telegram slash commands registered through the command handler
- native Telegram `/stop`, `/esc`, and `/kill` commands

Gateway now tracks `kind=callback|command` plus ack/dispatched/completed/failed/timed_out state for those paths, exposes canonical `operatorTracing` in `joelclaw gateway status` (with `callbackTracing` kept as a compatibility alias), and adds an `operator-tracing` layer in `joelclaw gateway diagnose`.

Queued Telegram agent commands now keep their trace id through downstream gateway execution, complete on the real turn completion path, and fail on pr