promql-validator

What this skill does

## How This Skill Works

This skill performs multi-level validation and provides interactive query planning:

1. **Syntax Validation**: Checks for syntactically correct PromQL expressions
2. **Semantic Validation**: Ensures queries make logical sense (e.g., rate() on counters, not gauges)
3. **Anti-Pattern Detection**: Identifies common mistakes and inefficient patterns
4. **Optimization Suggestions**: Recommends performance improvements
5. **Query Explanation**: Translates PromQL to plain English
6. **Interactive Planning**: Helps users clarify intent and refine queries

## Workflow

When a user provides a PromQL query, follow this workflow:

### Working Directory Requirement

Run validation commands from the repository root so relative paths resolve correctly:

```bash
cd "$(git rev-parse --show-toplevel)"
```

If running from another location, use absolute paths to `scripts/` files.

### Step 1: Validate Syntax

Run the syntax validation script to check for basic correctness:

```bash
python3 devops-skills-plugin/skills/promql-validator/scripts/validate_syntax.py "<query>"
```

Output parsing notes:
- Exit `0`: syntax valid
- Exit non-zero: syntax failure; include stderr and pinpoint token/position
- Prefer quoting the smallest failing fragment, then provide corrected query

The script will check for:
- Valid metric names and label matchers
- Correct operator usage
- Proper function syntax
- Valid time durations and ranges
- Balanced brackets and quotes
- Correct use of modifiers (offset, @)

### Step 2: Check Best Practices

Run the best practices checker to detect anti-patterns and optimization opportunities:

```bash
python3 devops-skills-plugin/skills/promql-validator/scripts/check_best_practices.py "<query>"
```

Output parsing notes:
- Treat script sections as independent findings (cardinality, metric-type misuse, regex misuse, etc.)
- If script output is empty but query is complex, add a manual sanity pass and mark it as `manual-review`
- Preserve script wording for finding labels, then add remediation in plain English

The script will identify:
- High cardinality queries without label filters
- Inefficient regex matchers that could be exact matches
- Missing rate()/increase() on counter metrics
- rate() used on gauge metrics
- Averaging pre-calculated quantiles
- Subqueries with excessive time ranges
- irate() over long time ranges
- Opportunities to add more specific label filters
- Complex queries that should use recording rules

### Step 3: Explain the Query

Parse and explain what the query does in plain English:
- What metrics are being queried
- What type of metrics they are (counter, gauge, histogram, summary)
- What functions are applied and why
- What the query calculates
- What labels will be in the output
- What the expected result structure looks like

**Required Output Details** (always include these explicitly):

```
**Output Labels**: [list labels that will be in the result, or "None (fully aggregated to scalar)"]
**Expected Result Structure**: [instant vector / range vector / scalar] with [N series / single value]
```

Example:
```
**Output Labels**: job, instance
**Expected Result Structure**: Instant vector with one series per job/instance combination
```

### Line-Number Citation Method (Required)

When citing examples/docs in recommendations, include file path + 1-based line numbers:

```text
examples/good_queries.promql:42
docs/best_practices.md:88
```

Rules:
- Cite the most relevant single line (or start line if multi-line snippet)
- Keep citations tight; do not cite full files
- If line numbers are unavailable, state `line number unavailable` and provide file path

### Step 4: Interactive Query Planning (Phase 1 - STOP AND WAIT)

Ask the user clarifying questions to verify the query matches their intent:

1. **Understand the Goal**: "What are you trying to monitor or measure?"
   - Request rate, error rate, latency, resource usage, etc.

2. **Verify Metric Type**: "Is this a counter (always increasing), gauge (can go up/down), histogram, or summary?"
   - This affects which functions to use

3. **Clarify Time Range**: "What time window do you need?"
   - Instant value, rate over time, historical analysis

4. **Confirm Aggregation**: "Do you need to aggregate data across labels? If so, which labels?"
   - by (job), by (instance), without (pod), etc.

5. **Check Output Intent**: "Are you using this for alerting, dashboarding, or ad-hoc analysis?"
   - Affects optimization priorities

> **IMPORTANT: Two-Phase Dialogue**
>
> After presenting Steps 1-4 results (Syntax, Best Practices, Query Explanation, and Intent Questions):
>
> **⏸️ STOP HERE AND WAIT FOR USER RESPONSE**
>
> Do NOT proceed to Steps 5-7 until the user answers the clarifying questions.
> This ensures the subsequent recommendations are tailored to the user's actual intent.

### Step 5: Compare Intent vs Implementation (Phase 2 - After User Response)

**Only proceed to this step after the user has answered the clarifying questions from Step 4.**

After understanding the user's intent:
- Explain what the current query actually does
- Highlight any mismatches between intent and implementation
- Suggest corrections if the query doesn't match the goal
- Offer alternative approaches if applicable

When relevant, mention known limitations:
- Note when metric type detection is heuristic-based (e.g., "The script inferred this is a gauge based on the `_bytes` suffix. Please confirm if this is correct.")
- Acknowledge when high-cardinality warnings might be false positives (e.g., "This warning may not apply if you're using a recording rule or know your cardinality is low.")

### Step 6: Offer Optimizations

Based on validation results:
- Suggest more efficient query patterns
- Recommend recording rules for complex/repeated queries
- Propose better label matchers to reduce cardinality
- Advise on appropriate time ranges

**Reference Examples**: When suggesting corrections, cite relevant examples using this format:

```
As shown in `examples/bad_queries.promql` (lines 91-97):
❌ BAD: `avg(http_request_duration_seconds{quantile="0.95"})`
✅ GOOD: Use histogram_quantile() with histogram buckets
```

Citation sources:
- `examples/good_queries.promql` - for well-formed patterns
- `examples/optimization_examples.promql` - for before/after comparisons
- `examples/bad_queries.promql` - for showing what to avoid
- `docs/best_practices.md` - for detailed explanations
- `docs/anti_patterns.md` - for anti-pattern deep dives

**Citation Format**: `file_path (lines X-Y)` with the relevant code snippet quoted

### Step 7: Let User Plan/Refine

Give the user control:
- Ask if they want to modify the query
- Offer to help rewrite it for better performance
- Provide multiple alternatives if applicable
- Explain trade-offs between different approaches

## Key Validation Rules

### Syntax Rules

1. **Metric Names**: Must match `[a-zA-Z_:][a-zA-Z0-9_:]*` or use UTF-8 quoting syntax (Prometheus 3.0+):
   - Quoted form: `{"my.metric.with.dots"}`
   - Using __name__ label: `{__name__="my.metric.with.dots"}`
2. **Label Matchers**: `=` (equal), `!=` (not equal), `=~` (regex match), `!~` (regex not match)
3. **Time Durations**: `[0-9]+(ms|s|m|h|d|w|y)` - e.g., `5m`, `1h`, `7d`
4. **Range Vectors**: `metric_name[duration]` - e.g., `http_requests_total[5m]`
5. **Offset Modifier**: `offset <duration>` - e.g., `metric_name offset 5m`
6. **@ Modifier**: `@ <timestamp>` or `@ start()` / `@ end()`

### Semantic Rules

1. **rate() and irate()**: Should only be used with counter metrics (metrics ending in `_total`, `_count`, `_sum`, or `_bucket`)
2. **Counters**: Should typically use `rate()` or `increase()`, not raw values
3. **Gauges**: Should not use `rate()` or `increase()`
4. **Histograms**: Use `histogram_quantile()` with `le` label and `rate()` on `_bucket` metrics
5. **Summaries**: Don't average quantiles; calculate from `_sum` and `_count`
6. **Aggregations**: Use `by()` or `without()` to con