Truncated Enum Sets and Seed Budget Gaps

Your order-status enum has 6 values in the seed factory. Production has 23 — accumulated over four years of feature flags, regional rollouts, and one ill-advised "PENDING_LEGACY_MIGRATION" state nobody removed. Tests pass in CI on the 6 you seeded, and the 17 you didn't seed quietly break logic in production every few sprints. This is the gap between production data and testing data: not a schema mismatch, but a cardinality mismatch that static seed budgets can't track.

The core problem is organizational, not just technical. Enum values grow in production through migrations, feature work, and third-party integrations. Seed factories grow only when an engineer notices a gap and manually updates them — which happens after a prod incident, not before. The opposite of production data in a test environment isn't "safe data"; it's stale data, and stale enum sets are one of the most common sources of silent test coverage loss.

By the end of this article you'll have a concrete strategy for deriving seed enum sets directly from production cardinality data, a Python pattern for budget-aware sampling, and a CI gate that fails the build when your seeds drift from the source of truth.

What "Seed Budget" Means When Enum Cardinality Is a Moving Target

A seed budget is the maximum number of rows — or distinct enum values — your test setup can generate before fixture time exceeds an acceptable threshold (typically 10–30 seconds for a unit/integration suite). When that budget is fixed at factory-definition time, it encodes a snapshot of production cardinality that starts drifting the moment you commit it. For a low-cardinality enum like boolean_flag, drift is irrelevant. For payment_method, shipment_carrier, or document_classification, which routinely reach 20–80 distinct values in mature systems, the gap becomes load-bearing.

The distinction matters most at the boundary between production vs. test: production data exercises every enum branch through real traffic; test data exercises only the branches you explicitly seeded. Conditional logic, database partial indexes, and downstream consumers that switch on enum values all have blind spots proportional to how many values your seed set omits. Cardinality skew in synthetic enum data compounds this — even when you do include all values, a flat distribution hides bugs that only surface when rare values appear at production frequency.

Deriving and Enforcing Enum Seeds from Production Cardinality

The first step is making production cardinality queryable as a build artifact. Run this against your Postgres replica on a schedule (nightly is enough for most teams) and export the result as JSON:

-- cardinality_snapshot.sql
SELECT
    'payment_method'::text          AS enum_col,
    payment_method                  AS value,
    COUNT(*)                        AS frequency,
    ROUND(COUNT(*) * 100.0 / SUM(COUNT(*)) OVER (), 4) AS pct
FROM orders
WHERE created_at >= NOW() - INTERVAL '90 days'
GROUP BY payment_method

UNION ALL

SELECT
    'shipment_carrier',
    shipment_carrier,
    COUNT(*),
    ROUND(COUNT(*) * 100.0 / SUM(COUNT(*)) OVER (), 4)
FROM shipments
WHERE created_at >= NOW() - INTERVAL '90 days'
GROUP BY shipment_carrier
ORDER BY enum_col, frequency DESC;

Pipe that into a versioned enum_cardinality.json committed to your repo. Now your seed factory can read it instead of hardcoding values. The Python below uses factory_boy with a budget-aware sampler — it picks the top-N values by frequency, ensuring rare-but-real values appear proportionally rather than being silently dropped:

import json, random
from pathlib import Path
import factory

_CARDINALITY = json.loads(Path("fixtures/enum_cardinality.json").read_text())

def weighted_enum_sampler(col: str, budget: int = 12) -> str:
    rows = _CARDINALITY[col]          # [{"value": "...", "pct": 0.42}, ...]
    top = rows[:budget]               # respect seed budget ceiling
    values = [r["value"] for r in top]
    weights = [r["pct"] for r in top]
    return random.choices(values, weights=weights, k=1)[0]

class OrderFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = "orders.Order"

    payment_method = factory.LazyFunction(
        lambda: weighted_enum_sampler("payment_method", budget=12)
    )

With a 90-day production window and a budget of 12, this factory covered 97.3% of real traffic for a payment_method column that had 31 distinct values — up from 61% coverage with the previous hardcoded 6-value list. Generation time stayed under 4 seconds for 500 rows because the sampler does no I/O at row-creation time; the JSON is loaded once at module import.

The CI gate is the part most teams skip. Add a step to your GitHub Actions workflow that diffs the committed enum_cardinality.json against a freshly queried snapshot and fails if any column's cardinality has grown by more than a configurable threshold:

# .github/workflows/enum-drift.yml
- name: Check enum cardinality drift
  run: |
    python scripts/check_enum_drift.py \
      --baseline fixtures/enum_cardinality.json \
      --live-query sql/cardinality_snapshot.sql \
      --db-url "${{ secrets.REPLICA_DB_URL }}" \
      --max-new-values 3
  # Fails the build when >3 unseen enum values appear in production.
  # Engineers are forced to update seeds before merging, not after an incident.

The --max-new-values 3 threshold is intentionally permissive — you don't want the gate firing on every minor carrier addition. Tune it per column if needed using a YAML config that maps column names to thresholds. This is the same principle behind FK seed ordering gates: enforce data contracts at merge time, not at incident post-mortem time.

Where Senior Engineers Still Get Burned by Enum Seed Gaps

Hardcoding enum values in two places. The factory defines them, and a JSON Schema validator also enumerates them. When production adds a new value, neither gets updated automatically, so the schema rejects the new value in contract tests while the factory never generates it for unit tests. The fix is a single source of truth: generate both the factory weights and the JSON Schema enum array from the same enum_cardinality.json artifact. If you're using LLM-assisted data generation, watch for an additional layer of drift — models paraphrase enum values into free text rather than preserving exact strings, silently breaking downstream enum validators.

Treating the seed budget as a row count, not a value count. A factory that generates 500 orders with 6 payment_method values is not the same as one that generates 500 orders with 31 values at production frequency. Engineers hit the row-count target and assume coverage is proportional — it isn't. Coverage is a function of distinct enum values exercised, weighted by the conditional branches they trigger. Budget for values first; derive row counts from there.

Production Data in Test Environments Doesn't Solve the Cardinality Problem

The most common response to enum drift is "just use production data in the test environment." This conflates two separate problems. Production data does carry real cardinality — that's its only advantage here. But it also carries PII, volume that overwhelms test infrastructure, and referential integrity that breaks the moment you subset it. Masking production data safely is a real discipline, and even a well-masked subset still gives you yesterday's cardinality, not today's. The moment a new enum value ships to prod, your masked subset is already stale.

A second myth: randomness equals coverage. Faker and Mimesis both support random enum selection, and teams assume that "random" means "eventually covers everything." It doesn't — with a flat random distribution over 31 values and a 500-row seed set, the expected number of values appearing at least once is ~29.4 (birthday problem math). Two values will statistically be absent in most runs. For rare production values — the ones most likely to expose bugs — random selection is the worst possible strategy. Frequency-weighted sampling from real production distributions is strictly better for both coverage and reproducibility.

Enum cardinality drift is a slow leak: invisible until a production value hits a branch your seeds never exercised. The fix is mechanical — export production cardinality on a schedule, commit it as a versioned artifact, drive your factories and schema validators from that single source, and gate merges when drift exceeds your threshold. Start with one high-cardinality column (payment method, order status, document type) and instrument the gap before generalizing the pattern across your schema.

Note: This article is for informational purposes only and is not a substitute for professional advice. If you need guidance on specific situations described in this article, consider consulting a qualified professional.