test-property-based

What this skill does

# Property-Based Testing with Hypothesis

## Quick Start

Property-based testing automatically generates hundreds of test cases to validate invariants:

```python
from hypothesis import given, strategies as st

# Instead of writing many example tests...
# def test_sort_1(): assert sorted([3,1,2]) == [1,2,3]
# def test_sort_2(): assert sorted([]) == []
# ... (20 more examples)

# Write ONE property test that covers ALL cases
@given(st.lists(st.integers()))
def test_sort_idempotent(lst):
    """Property: Sorting twice gives same result as once."""
    once_sorted = sorted(lst)
    twice_sorted = sorted(once_sorted)
    assert once_sorted == twice_sorted
```

**Hypothesis automatically generates 100+ test cases** including edge cases you'd never think of:
empty lists, single elements, duplicates, large lists, negative numbers, etc.

## Table of Contents

1. [When to Use This Skill](#when-to-use-this-skill)
2. [What This Skill Does](#what-this-skill-does)
3. [Core Concepts](#core-concepts)
   - [Strategies](#strategies)
   - [The @given Decorator](#the-given-decorator)
   - [Shrinking](#shrinking)
   - [Custom Strategies](#custom-strategies)
4. [Step-by-Step Workflow](#step-by-step-workflow)
5. [Common Property Patterns](#common-property-patterns)
6. [Integration with pytest](#integration-with-pytest)
7. [Async Property Testing](#async-property-testing)
8. [Pydantic Model Testing](#pydantic-model-testing)
9. [Configuration](#configuration)
10. [Supporting Files](#supporting-files)
11. [Expected Outcomes](#expected-outcomes)
12. [Requirements](#requirements)
13. [Red Flags to Avoid](#red-flags-to-avoid)

## When to Use This Skill

### Explicit Triggers
Use this skill when users mention:
- "property test"
- "hypothesis test"
- "generate test cases"
- "fuzz testing"
- "invariant testing"
- "roundtrip test"
- "stateful testing"
- "edge case testing"
- "test with random data"

### Implicit Triggers
Use when you observe:
- Manual writing of many similar example tests
- Testing parsing/serialization (perfect for roundtrip properties)
- Validating configuration classes (especially Pydantic models)
- Testing algorithms with mathematical properties
- Protocol message handling (IPC, API requests/responses)
- State machine behavior

### Debugging Triggers
Use when:
- Edge case bugs slip through example-based tests
- Need more comprehensive input coverage
- Test suite misses corner cases
- Validating refactored code behavior

## What This Skill Does

This skill guides you through:

1. **Installing Hypothesis** - Add to project dependencies
2. **Writing property tests** - Transform example tests into property-based tests
3. **Choosing strategies** - Select appropriate data generators
4. **Creating custom strategies** - Build domain-specific generators
5. **Async integration** - Combine with pytest-asyncio
6. **Pydantic integration** - Test Pydantic models automatically
7. **Configuration** - Set up profiles for dev/CI/thorough testing
8. **Stateful testing** - Test state machines and complex workflows

**Philosophy:** Instead of "here are 5 examples that should work", write "here's a property that should ALWAYS hold" and let Hypothesis find edge cases.

## Core Concepts

### Strategies

Strategies describe the type of data Hypothesis should generate:

```python
from hypothesis import strategies as st

# Basic types
st.integers()                           # All integers
st.integers(min_value=0, max_value=100) # Constrained range
st.floats(allow_nan=False)              # Floats without NaN
st.text()                               # Unicode strings
st.text(alphabet="abc", min_size=1)     # Limited alphabet
st.binary()                             # Bytes

# Collections
st.lists(st.integers())                 # Lists of integers
st.dictionaries(st.text(), st.integers()) # Dict[str, int]
st.sets(st.text(), min_size=1)          # Non-empty sets
st.tuples(st.text(), st.integers())     # (str, int) tuples

# Special
st.one_of(st.integers(), st.text())     # Union types
st.none()                               # None values
st.uuids()                              # UUID objects
st.datetimes()                          # datetime objects
```

**See [references/strategies-reference.md](references/strategies-reference.md) for complete strategy catalog.**

### The @given Decorator

The `@given` decorator runs your test function with generated data:

```python
from hypothesis import given, strategies as st

@given(st.integers(), st.integers())
def test_addition_commutative(a, b):
    """Addition should be commutative."""
    assert a + b == b + a

@given(st.lists(st.integers()))
def test_sort_preserves_length(lst):
    """Sorting preserves list length."""
    assert len(sorted(lst)) == len(lst)
```

**Default behavior:** Runs 100 examples (configurable via settings).

### Shrinking

When Hypothesis finds a failing test, it **automatically minimizes** the input:

```python
@given(st.lists(st.integers()))
def test_sum_positive(lst):
    assert sum(lst) >= 0  # Fails for negative numbers

# Hypothesis reports: lst=[-1]
# NOT lst=[-9999, -42, -1, -8888] (the random case it found)
```

**This is invaluable for debugging** - you get the minimal failing case, not a complex random one.

### Custom Strategies

For complex domain objects, build custom strategies with `@composite`:

```python
from hypothesis import strategies as st
from hypothesis.strategies import composite

@composite
def valid_emails(draw):
    """Generate valid email addresses."""
    username = draw(st.text(alphabet=st.characters(
        whitelist_categories=('Ll', 'Lu', 'Nd'),
        min_codepoint=ord('a')
    ), min_size=1, max_size=20))

    domain = draw(st.text(alphabet=st.characters(
        whitelist_categories=('Ll',),
        min_codepoint=ord('a')
    ), min_size=1, max_size=15))

    tld = draw(st.sampled_from(['com', 'org', 'net', 'io']))

    return f"{username}@{domain}.{tld}"

@given(valid_emails())
def test_email_parsing(email):
    """Test parsing of valid email addresses."""
    assert '@' in email
    assert '.' in email.split('@')[1]
```

**See [references/patterns-catalog.md](references/patterns-catalog.md) for more custom strategy patterns.**

## Step-by-Step Workflow

### Step 1: Install Hypothesis

```bash
# Add to project dependencies
uv add --dev hypothesis

# Verify installation
python -c "import hypothesis; print(hypothesis.__version__)"
```

### Step 2: Identify Properties to Test

Look for:
- **Invariants** - Things that should always be true
- **Roundtrips** - Serialize → Deserialize → Should equal original
- **Idempotency** - Operation twice = operation once
- **Commutativity** - Order doesn't matter
- **Consistency** - Related operations agree

**Example:** Testing a JSON serializer:
- Property: `parse(serialize(obj)) == obj` (roundtrip)
- Property: `serialize(obj)` returns valid JSON string
- Property: All serialized objects are parseable

### Step 3: Choose Strategies

Map your data types to Hypothesis strategies:

```python
# Simple types
int → st.integers()
str → st.text()
bool → st.booleans()

# Collections
List[int] → st.lists(st.integers())
Dict[str, int] → st.dictionaries(st.text(), st.integers())
Optional[str] → st.one_of(st.text(), st.none())

# Domain models (Pydantic)
MyModel → builds(MyModel)
```

### Step 4: Write Property Test

```python
from hypothesis import given, strategies as st

@given(st.dictionaries(st.text(), st.text()))
def test_json_roundtrip(data):
    """Property: All dicts should roundtrip through JSON."""
    import json
    serialized = json.dumps(data)
    parsed = json.loads(serialized)
    assert parsed == data
```

### Step 5: Run and Observe

```bash
# Run property test
pytest tests/test_properties.py -v

# Show statistics
pytest --hypothesis-show-statistics

# Reproduce specific failure
pytest --hypothesis-seed=12345
```

### Step 6: Refine if Needed

If test generates invalid inputs:
- Add constraints to strategy
- Use `assume()` to filter (sparingly