metric-extraction

Included with Lifetime

$97 forever

Parse METRIC output lines, infer units, and track primary vs secondary metrics. Use when processing experiment output from autoresearch.sh.

Data & Analytics

What this skill does


# Metric Extraction

Parses structured output from `autoresearch.sh` to extract primary and secondary metrics.

## Output Format

Each metric is a single line matching:

```
METRIC <name>=<value>
```

- **Name**: word characters (`a-z`, `A-Z`, `0-9`, `_`), dots (`.`), or `µ`. Examples: `total_µs`, `compile_ms`, `cache.hits`
- **Value**: any token parseable as a finite number. `NaN`, `Infinity`, and non-numeric values are silently ignored.
- One metric per line. Lines not starting with `METRIC` are ignored (but may contain useful diagnostics).
- If a name appears multiple times, the **last occurrence wins**.

## Primary vs Secondary

- **Primary metric**: The one whose name matches what was declared at session init (the optimization target). This is what determines `keep` vs `discard`.
- **Secondary metrics**: All other `METRIC` lines. Tracked for tradeoff monitoring but don't affect keep/discard decisions.

If the primary metric is missing from output, treat the run as a **crash** — the benchmark didn't produce the expected data.

## Unit Inference

Infer units from metric name suffixes for display and context:

| Suffix | Unit |
|--------|------|
| `µs` | µs (microseconds) |
| `_ms` | ms (milliseconds) |
| `_s` or `_sec` | s (seconds) |
| `_kb` | kb (kilobytes) |
| `_mb` | mb (megabytes) |
| (none matched) | (unitless) |

Units are informational — they don't affect computation.

## Tracking Across Runs

Maintain a list of known secondary metrics discovered across the session. When a new metric name appears in output that hasn't been seen before, register it with its inferred unit. This allows consistent reporting even when scripts evolve during the loop.

## Recording in JSONL

When logging an experiment, record metrics as:

```json
{
  "metric": 14600,
  "metrics": {
    "compile_µs": 4100,
    "render_µs": 9500,
    "cache.hits": 42
  }
}
```

- `metric`: the primary metric's numeric value (top-level for easy querying)
- `metrics`: object of all secondary metric name→value pairs

## Designing Informative Output

The `autoresearch.sh` script should output whatever helps the agent make better decisions:

- **Phase timings** when the workload has distinct stages
- **Error counts or categories** when correctness can fail in different ways
- **Memory/cache diagnostics** when relevant to the optimization
- **Domain-specific signals** that help localize regressions

The script can be updated during the loop as you learn what signal matters. Add instrumentation when you need more data to decide where to focus next.

Files: 1

Size: 2.7 KB

Complexity: 11/100

Category: Data & Analytics

Source: https://github.com/pbdeuchler/llm-plugins/tree/main/plugins/autoresearch/skills/metric-extraction

Related in Data & Analytics

clawarr-suite

Included

Comprehensive management for self-hosted media stacks (Sonarr, Radarr, Lidarr, Readarr, Prowlarr, Bazarr, Overseerr, Plex, Tautulli, SABnzbd, Recyclarr, Unpackerr, Notifiarr, Maintainerr, Kometa, FlareSolverr). Deep library exploration, analytics, dashboard generation, content management, request handling, subtitle management, indexer control, download monitoring, quality profile sync, library cleanup automation, notification routing, collection/overlay management, and media tracker integration (Trakt, Letterboxd, Simkl).

Data & Analyticsscripts

querying-soql

Included

SOQL query generation, optimization, and analysis with 100-point scoring. Use this skill when the user needs SOQL/SOSL authoring or optimization: natural-language-to-query generation, relationship queries, aggregates, query-plan analysis, and performance or safety improvements for Salesforce queries. TRIGGER when: user writes, optimizes, or debugs SOQL/SOSL queries, touches .soql files, or asks about relationship queries, aggregates, or query performance. DO NOT TRIGGER when: bulk data operations (use handling-sf-data), Apex DML logic (use generating-apex), or report/dashboard queries.

Data & Analyticsscripts

app-store-optimization

Included

App Store Optimization (ASO) toolkit for researching keywords, analyzing competitor rankings, generating metadata suggestions, and improving app visibility on Apple App Store and Google Play Store. Use when the user asks about ASO, app store rankings, app metadata, app titles and descriptions, app store listings, app visibility, or mobile app marketing on iOS or Android. Supports keyword research and scoring, competitor keyword analysis, metadata optimization, A/B test planning, launch checklists, and tracking ranking changes.

Data & Analyticsscripts

habit-flow

Included

AI-powered atomic habit tracker with natural language logging, streak tracking, smart reminders, and coaching. Use for creating habits, logging completions naturally ("I meditated today"), viewing progress, and getting personalized coaching.

Data & Analyticsscripts

app-store-optimization

Included

Data & Analyticsscripts

visualizing-data

Included

Builds dashboards, reports, and data-driven interfaces requiring charts, graphs, or visual analytics. Provides systematic framework for selecting appropriate visualizations based on data characteristics and analytical purpose. Includes 24+ visualization types organized by purpose (trends, comparisons, distributions, relationships, flows, hierarchies, geospatial), accessibility patterns (WCAG 2.1 AA compliance), colorblind-safe palettes, and performance optimization strategies. Use when creating visualizations, choosing chart types, displaying data graphically, or designing data interfaces.

Data & Analyticsscripts