Test Generation in EK9

EK9 can generate edge-case tests and mutation variants for your code using the compiler's own symbol table. Unlike external tools that require separate configuration, EK9's test generator has direct access to your types, methods, operators, and constructors — it knows exactly what to test.

Looking for compiler fuzzing? If you want to test the compiler itself, see Compiler Fuzzing instead. This page documents test generation for your code.

• The Problem - Why automated test generation matters
• Two Modes Overview - Test generation vs mutation testing
• Mode 1: Test Generation - Generate edge-case @Test functions
• Mode 2: Mutation Testing - Generate mutation variants
• The AI-Compiler Feedback Loop - Strategic workflow
• CLI Reference - Complete flag table
• Comparison with Industry Tools - EK9 vs PIT, Hypothesis, AFL

The Problem

Tests pass, coverage looks adequate, but edge cases remain untested. Boundary values, empty collections, unset optionals, type extremes — these are the inputs that cause production failures. Developers write tests for the expected cases; the unexpected cases are where bugs hide.

Industry tools like PIT/Pitest (mutation testing), Hypothesis/QuickCheck (property-based testing), and AFL/libFuzzer (coverage-guided fuzzing) address this, but each requires external setup, configuration, and integration. EK9 collapses this into the compiler itself: the generator has the symbol table, knows your types, and produces compilable EK9 test files directly.

Two Modes Overview

Both modes compile-check every candidate through the PRE_IR_CHECKS phase before outputting. Only candidates that compile successfully are included — no false positives.

Test Generation

Usage

The second argument is the output file path — the generated tests are written there. If the output file already exists, EK9 will refuse to overwrite it and emit an error. This prevents accidentally destroying curated test files. Use -overwrite to explicitly allow replacement:

What It Analyses

The generator compiles your source through PRE_IR_CHECKS, then harvests the symbol table to find testable surfaces:

• Constructors — Default and parameterised constructors for all non-abstract types
• Methods — Instance methods with their parameter types
• Operators — Defined operators (+, <, ?, $, etc.)
• Functions — Module-level functions with parameter types

For each parameter position, the generator injects values from a type-specific edge-case value pool. Every type includes an unset value (e.g., String()) to exercise EK9's tri-state semantics — verifying that your code handles the unset state correctly:

Edge-Case Value Pools

Output Format

The output file (specified as the second argument) is a complete, compilable .ek9 file containing @Test functions:

#!ek9
defines module mylib.fuzz.tests
  references
    mylib

  defines program

    @Test
    testFuzz_Account_construct_0_001()
      stdout <- Stdout()
      instance <- Account(0)
      assert instance?
      stdout.println(`Account(0): ${$instance}`)

    @Test
    testFuzz_Account_construct_2147483647_002()
      stdout <- Stdout()
      instance <- Account(2147483647)
      assert instance?
      stdout.println(`Account(MAX_INT): ${$instance}`)

    @Test
    testFuzz_Account_deposit_negOne_003()
      stdout <- Stdout()
      instance <- Account(100)
      instance.deposit(-1)
      stdout.println(`deposit(-1): ${$instance}`)

What to do with the output: Review the generated tests. Keep the ones that expose interesting edge cases. Delete the rest. Run the survivors with ek9 -t mylib_tests.ek9. If the output file already exists, an error is emitted — use -overwrite to replace it.

Console Output

EK9 Test Generation: analyzing mylib.ek9
Seed: 42, Max candidates: 200
Found 3 types, 2 functions
Generated 187 test candidates
Survivors: 142 / 187 (75.9%)
Output written to: mylib_tests.ek9

Mutation Testing

Usage

The second argument is the output directory — mutation variants are written there. If the output directory already exists, EK9 will refuse to overwrite it and emit an error. This prevents accidentally destroying curated mutation sets. Use -overwrite to explicitly allow replacement:

How It Works

Mutation testing assesses your test quality by asking: if I introduce a small bug, do your tests catch it? The generator creates variants of your source code, each with a single mutation. If your tests still pass against a mutant, that mutation represents an undetected fault — a gap in your test coverage.

Each mutation is a single-point change to one line of your source. This ensures that any test failure can be traced to exactly one change.

Mutation Categories

The mutation categories are adapted for EK9's semantics. There are no break, continue, or return mutations because those constructs do not exist in EK9.

Output Format

Mutations are written to a directory with one .ek9 file per surviving mutant, plus a manifest.txt summary:

mutations/
  manifest.txt
  mutation_000_ARITH_SWAP_L42.ek9
  mutation_001_CMP_FLIP_L53.ek9
  mutation_002_BOOL_LOGIC_L67.ek9
  mutation_003_BOUNDARY_SHIFT_L89.ek9
  ...

Reading the Manifest

# Mutation manifest for myapp.ek9
# Generated: 2026-03-02 14:30:00
# Candidates: 100, Survivors: 73 (73.0%), Seed: 123
#
# File                                    Category          Line  Original   Mutated
mutation_000_ARITH_SWAP_L42.ek9          ARITH_SWAP        42    +          -
mutation_001_CMP_FLIP_L53.ek9            CMP_FLIP          53    <          <=
mutation_002_BOOL_LOGIC_L67.ek9          BOOL_LOGIC        67    true       false
mutation_003_BOUNDARY_SHIFT_L89.ek9      BOUNDARY_SHIFT    89    100        99

Interpreting Results

Run your existing tests against each mutant. A mutant that your tests kill (cause to fail) is detected — good. A mutant that survives (tests still pass) indicates a gap:

• ARITH_SWAP survivors — Tests don't verify arithmetic correctness
• CMP_FLIP survivors — Tests don't check boundary conditions
• BOOL_LOGIC survivors — Tests don't cover both branches
• GUARD_MUTATION survivors — Tests don't exercise the unset/set distinction
• METHOD_REMOVAL survivors — Tests don't verify side effects

The mutation score is the percentage of mutants killed. Higher is better. A score below 60% suggests significant gaps in test coverage that line-based coverage metrics may not reveal.

The AI-Compiler Feedback Loop

Test generation and mutation testing are most powerful when combined with AI-assisted development. The strategic workflow:

1. Write code — Implement your feature with initial tests
2. Generate volume — Run -fuzztest to produce hundreds of edge-case candidates (seconds)
3. Compile-check — The compiler filters invalid candidates automatically
4. AI curates — Use an AI assistant to review survivors, identify the most valuable tests, and discard noise
5. Mutate — Run -fuzzmutate to generate mutation variants
6. Assess — Run tests against mutants to find gaps
7. Strengthen — Write targeted tests for surviving mutants

This creates a force multiplier: the compiler generates volume (cheap, fast), the AI curates intelligence (selective, contextual), and the developer makes final decisions (domain knowledge). The token cost is approximately 12x lower than having AI generate all tests from scratch, because the compiler handles the combinatorial explosion.

CLI Reference

Comparison with Industry Tools

The key advantage of EK9's approach is zero configuration. The compiler already has your symbol table, type information, and method signatures. External tools must rediscover this information through instrumentation or configuration.

See Also

• Compiler Fuzzing - Grammar-based fuzzing of the compiler itself
• Testing - Test types, assertions, test runner commands
• Code Coverage - Threshold enforcement, quality metrics, HTML reports
• Code Quality - Quality enforcement rules and thresholds
• Command Line - All flags and exit codes
• For AI Assistants - Machine-readable output schemas