functionality-audit

What this skill does

# Functionality Audit

This skill validates that code genuinely works as intended through systematic testing in isolated sandbox environments. Rather than assuming code is correct because it looks right or passes cursory checks, the functionality audit actually executes code with realistic inputs, verifies outputs match expectations, and debugs any issues discovered. This transforms theoretical correctness into verified functionality.

## When to Use This Skill

Use the functionality-audit skill after generating or modifying code to verify it works before deployment, when code appears correct but behavior seems off, after integrating code from multiple sources, or before production releases as final validation. The skill is essential when correctness is critical, when code complexity makes visual inspection insufficient, or when debugging existing code that fails intermittently or mysteriously.

## The Importance of Execution Verification

Code that looks correct is not the same as code that works correctly. Syntax can be valid while logic is flawed. Functions can exist while their implementations are broken. Integration points can appear properly connected while producing incorrect results. Execution verification eliminates the gap between apparent correctness and actual functionality.

**Syntax Correctness Versus Semantic Correctness**: Code can parse and compile without errors while implementing the wrong algorithm, making incorrect assumptions about input data, handling edge cases improperly, or producing subtly wrong outputs. Syntax checking validates form but not substance. Semantic correctness requires execution with realistic inputs and careful verification of outputs.

**Static Analysis Limitations**: Static analysis tools check for code smells, common pitfalls, and structural issues, but they cannot verify that code implements its intended behavior correctly. A function might have perfect style and no linting warnings while computing the wrong result. Static analysis is valuable but insufficient for functionality validation.

**The Testing Gap**: Even code with tests may have bugs if tests are incomplete, shallow, or incorrectly specify expected behavior. Tests themselves can be wrong, passing while the code they test is broken. Comprehensive functionality auditing goes beyond running existing tests to creating new test cases that probe behavior systematically and verify outputs carefully.

**Integration Complexity**: Code that works in isolation often fails when integrated with other components because of incorrect interface assumptions, incompatible data formats, race conditions in concurrent code, or dependencies on execution order. Execution verification in realistic environments exposes these integration issues before they cause production failures.

## Sandbox Testing Methodology

The functionality audit uses isolated sandbox environments to test code safely without affecting production systems or development environments.

### Sandbox Creation

Create isolated test environments that replicate production conditions as closely as possible while remaining safe for experimentation. This involves provisioning fresh virtual environments with required dependencies, setting up isolated databases or mock data stores, configuring environment variables and system settings, and ensuring network isolation to prevent unintended external interactions.

Sandboxes should be reproducible so tests run consistently across different executions. Use containerization, virtual machines, or isolated Python environments depending on the technology stack. The goal is a clean slate for each test run that mimics production without production risks.

### Test Case Generation

Generate comprehensive test cases that exercise code through multiple paths including normal operation with typical valid inputs, boundary conditions that test the edges of acceptable inputs, error cases with invalid or malformed inputs, edge cases that expose off-by-one errors or overflow conditions, and stress tests with large datasets or high concurrency if performance matters.

Test cases should be systematically designed to achieve high code coverage while focusing on the most likely sources of bugs. For example, any code that parses external input should be tested with malformed input. Any code that performs calculations should be tested with boundary values. Any code that loops should be tested with empty, single-item, and large collections.

### Execution Monitoring

Execute code in the sandbox while carefully monitoring behavior including standard output and error streams for unexpected messages, return values and exceptions to verify correct results, execution time and resource usage for performance issues, and file system or database state changes for side effects. Comprehensive monitoring reveals not just whether code runs but whether it runs correctly.

Use instrumentation and logging liberally during testing. Add temporary print statements or debug logs to expose intermediate values. Use debugger breakpoints to step through complex logic. The goal is complete transparency into what the code is actually doing during execution.

### Output Verification

Verify that code produces correct outputs by comparing actual results against expected results using precise equality for deterministic functions, range checking or statistical measures for functions with acceptable variance, structural validation for complex outputs like JSON or objects, and assertion checking for invariants that should always hold.

Output verification requires clear specifications of what correct behavior looks like. For functions with complex outputs, define validation logic that checks all important properties rather than just superficial aspects. Subtle bugs often hide in outputs that are mostly right but wrong in small ways.

### Failure Analysis

When tests fail, perform systematic analysis to understand why. Examine the exact failure mode and error messages, trace execution backwards from the failure point, identify the first point where actual behavior diverges from expected behavior, and determine the root cause rather than just symptoms. Effective debugging starts with precise understanding of what went wrong and why.

Avoid jumping to conclusions or applying fixes before understanding root causes. A hasty fix for a symptom often leaves the underlying bug in place or creates new problems. Take time to understand failures deeply before attempting repairs.

## Systematic Debugging Workflow

When functionality audits reveal bugs, follow this systematic debugging workflow to fix issues reliably without introducing new problems.

### Step 1: Reproduce the Bug Reliably

Before attempting any fix, ensure the bug can be reproduced consistently. Create a minimal test case that reliably triggers the failure. Strip away unnecessary complexity to isolate the bug. Document the exact sequence of inputs or conditions needed to reproduce the issue. Reliable reproduction is essential for verifying that fixes actually work.

If a bug cannot be reproduced reliably, it may be a race condition, memory corruption, or other non-deterministic issue requiring specialized debugging techniques like stress testing under various conditions or memory sanitizers.

### Step 2: Understand the Bug's Cause

Investigate why the bug occurs by tracing through the code execution path that leads to the failure, examining variable values and state at key points, identifying incorrect assumptions or logic errors, and understanding what the code should do versus what it actually does. Root cause understanding prevents superficial fixes that mask problems rather than solving them.

Use debugging tools like breakpoints, watch expressions, and stack traces to inspect program state. Add logging to expose hidden behavior. Draw diagrams of data flow if it helps clarify complex logic. The investment in understanding pays off in better fixes.

### S