Stop Prod Deploys From Wiping Test Data

Production deployments delete test data. Not because someone wrote a malicious migration — because a cleanup job that runs fine in staging quietly matches test records in prod when environment boundaries blur. The data disappears, the next test cycle fails with foreign-key violations instead of useful assertions, and the post-mortem lands on "process" rather than the missing guard rails that would have caught it at the pipeline layer.

The specific failure mode: candidate applications — records seeded for hiring-flow or onboarding tests — share status codes, tenant IDs, or email domains with real records. A deploy-time script that prunes status = 'DRAFT' rows doesn't know the difference. Neither does the ORM. The result is silent data loss that only surfaces when a test suite runs hours later.

By the end of this article you'll have a concrete tagging strategy, a deploy-gate pattern you can drop into GitHub Actions, and a clear mental model for which cleanup belongs before a test cycle versus which belongs at deploy time.

Why Candidate Applications Are a High-Risk Cleanup Target

Candidate application records are structurally identical to real ones. They carry valid foreign keys into users, jobs, and documents. They progress through the same state machine. A cleanup script that targets status IN ('DRAFT','PENDING') will happily delete seeded test candidates alongside abandoned real drafts — there's no intrinsic marker that distinguishes them unless you put one there intentionally.

The problem compounds in shared environments where the same database hosts both pre-production validation traffic and real user data — a pattern more common than it should be. Understanding the distinction between test and production data is the first control; the second is enforcing it structurally, not just by convention. Relying on a naming convention like test_ email prefixes is a social contract, not an engineering control, and social contracts break under deploy pressure.

Tagging, Gating, and Scoping Cleanup to Survive a Deploy

The most durable pattern is a provenance column on every table that receives synthetic data. A single SMALLINT or VARCHAR(16) column named data_origin (values: REAL, TEST, SEED) costs almost nothing and makes every cleanup query unambiguous. Add a partial index and the performance impact is negligible.

-- Migration: add provenance to candidate_applications
ALTER TABLE candidate_applications
  ADD COLUMN data_origin VARCHAR(16) NOT NULL DEFAULT 'REAL'
  CHECK (data_origin IN ('REAL','TEST','SEED'));

CREATE INDEX CONCURRENTLY idx_ca_data_origin
  ON candidate_applications (data_origin)
  WHERE data_origin != 'REAL';

Every seed script, factory, or AI-generated fixture must set this column explicitly. With factory_boy:

import factory
from myapp.models import CandidateApplication

class CandidateApplicationFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = CandidateApplication

    status = "DRAFT"
    data_origin = "TEST"          # never omit this
    applicant_email = factory.Faker("email")
    job_id = factory.SubFactory(JobFactory)

Now your deploy-time cleanup script can be scoped safely. Wrap it in a pre-deploy GitHub Actions step that refuses to run without an explicit data_origin filter:

# .github/workflows/deploy.yml (relevant step)
- name: Cleanup stale test candidates
  env:
    DATABASE_URL: ${{ secrets.PROD_DATABASE_URL }}
  run: |
    python manage.py cleanup_candidates \
      --data-origin TEST \
      --older-than-days 7 \
      --dry-run false \
      --require-origin-flag   # script exits 1 if flag is absent

The --require-origin-flag guard is the key: the management command checks that data_origin was explicitly passed and aborts if it wasn't. This turns an accidental omission into a deploy failure rather than silent data loss. A team that added this gate reduced unintended record deletion incidents from four per quarter to zero — not because the deploys changed, but because the abort path became the default. For a fuller view of where this step sits in the overall flow, see the test data lifecycle from generation to cleanup.

Deploy-Time Mistakes Even Senior Engineers Repeat

Scoping cleanup by status instead of origin. Status-based cleanup (WHERE status = 'DRAFT') is the single most common cause of accidental test data loss in production. It happens because the original cleanup script was written before the provenance column existed, nobody updated it when synthetic data was introduced, and it ran silently for months. The fix is a one-time migration plus a linting rule that rejects any cleanup query missing an AND data_origin = 'TEST' clause. A sqlfluff custom rule or a simple grep in CI catches this in seconds.

Treating cleanup as a post-deploy step rather than a pre-deploy gate. When cleanup runs after traffic shifts to the new version, real users may already be writing records that match the cleanup predicate. Running it pre-deploy — before the new code is live — keeps the window deterministic. The related mistake is skipping structured cleanup before each test cycle entirely, which lets stale synthetic records accumulate until they outnumber real ones and start distorting analytics or triggering rate limits.

Myths About Production Safety and Test Data Boundaries

"We use a separate schema, so there's no risk." Schema separation reduces blast radius but doesn't eliminate it. Cross-schema foreign keys, shared sequences, and application-layer ORMs that don't respect schema boundaries all create paths for a cleanup job to reach the wrong rows. Schema separation is a layer, not a guarantee. Treat it as defense-in-depth alongside provenance tagging, not a replacement for it.

"Prod data clones are safe test environments." A cloned production database still contains real PII unless you've run a masking pass — and most teams haven't done it rigorously. Beyond privacy risk, unmasked clones mean test cleanup scripts are now running against real email addresses and real financial records. If your team is using prod clones, the PII masking step is not optional. A third myth worth naming: randomness equals coverage. Seeding with Faker random values doesn't cover the edge cases that break candidate-application pipelines — null middle names, non-ASCII characters in legal names, or duplicate SSNs from a poorly seeded PRNG. Structured boundary-value generation with Hypothesis or a constrained Pydantic model catches those; pure randomness doesn't.

The engineering lift here is low: one provenance column, one factory default, one deploy-gate check. What's high is the cost of skipping it — corrupted test cycles, phantom failures, and the debugging time that follows. Start by auditing your existing cleanup scripts for status-only predicates. Add the data_origin column to your next migration. Wire the gate into your deploy pipeline before the next release. That's the whole playbook.

Note: This article is for informational purposes only and is not a substitute for professional advice. If you need guidance on specific situations described in this article, consider consulting a qualified professional.