Silent VARCHAR Truncation in Synthetic Seeds
Your seed script runs clean, row counts look right, and the CI pipeline stays green — until a downstream service starts returning garbled names, clipped addresses, or broken tokens that fail regex validation three environments later. The bug isn't in the application code. It's in a VARCHAR(50) column that quietly swallowed the last 23 characters of a Faker-generated street address at insert time, and nobody noticed because the INSERT didn't raise an error.
Silent truncation is a first-class test data defect. Most databases — Postgres in varchar mode with no strict length enforcement at the session level, MySQL in non-strict SQL mode, SQLite always — will clip a string to the column's declared length rather than reject the row. Your synthetic data pipeline produces valid-looking records that are structurally corrupt from the moment they land.
By the end of this article you'll know exactly where truncation hides in a seed pipeline, how to enforce length contracts at generation time using Pydantic and JSON Schema, and how to write a post-seed assertion layer that catches violations before they propagate to any test suite.
Explore the data, models, mistakes, and methods behind identifying overlooked players.
Why VARCHAR Clipping Is a Seed-Layer Problem, Not a DB Problem
The database is doing exactly what it was configured to do. The defect lives upstream — in the gap between the length contract declared in your schema DDL and the length contract (or absence of one) in your synthetic data generator. Faker's address() can return 120-character strings. company() routinely exceeds 60 characters. If your DDL says VARCHAR(64) and you're generating with no ceiling, you're shipping truncated fixtures on every seed run where the RNG happens to produce a long value. That's non-deterministic data corruption tied to your random seed.
This fits into a broader category of synthetic data contracts that teams define at the wrong layer — usually as database constraints rather than as generator constraints. The correct fix is to enforce the length ceiling in the generator, validate it before the INSERT, and treat any violation as a build failure. The database's truncation behavior becomes irrelevant once your generator can't produce an out-of-range value.
Enforcing Length Contracts at Generation and Validation Time
Start by mirroring your DDL column lengths into your generator layer. With Pydantic v2, Annotated + StringConstraints gives you a single source of truth that both generates and validates:
from pydantic import BaseModel, StringConstraints
from typing import Annotated
from faker import Faker
fake = Faker()
# Mirror your DDL: first_name VARCHAR(50), company VARCHAR(64)
FirstName = Annotated[str, StringConstraints(max_length=50)]
CompanyName = Annotated[str, StringConstraints(max_length=64)]
class CustomerSeed(BaseModel):
first_name: FirstName
company: CompanyName
def generate_customer() -> CustomerSeed:
return CustomerSeed(
first_name=fake.first_name()[:50], # explicit guard
company=fake.company()[:64], # explicit guard
)
The slice guard on the Faker call is intentional — but it's a blunt instrument. A sliced company name like "Hernandez, Williams and" is structurally wrong even if it fits the column. The better approach is to use Faker providers that respect a max_chars budget, or to regenerate on violation rather than truncate. Hypothesis's st.text(max_size=50) does this correctly out of the box for property-based tests.
For bulk seed pipelines, add a post-generation validation pass using JSON Schema 2020-12 before any data touches the database. Define your schema once:
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"first_name": { "type": "string", "maxLength": 50 },
"company": { "type": "string", "maxLength": 64 },
"email": { "type": "string", "maxLength": 254, "format": "email" }
},
"required": ["first_name", "company", "email"]
}
Then validate every generated batch before the INSERT:
import json, jsonschema, pathlib
schema = json.loads(pathlib.Path("customer_seed.schema.json").read_text())
def validate_batch(records: list[dict]) -> None:
validator = jsonschema.Draft202012Validator(schema)
errors = []
for i, rec in enumerate(records):
for err in validator.iter_errors(rec):
errors.append(f"Row {i}: {err.json_path} — {err.message}")
if errors:
raise ValueError(f"Seed validation failed:\n" + "\n".join(errors))
In a real pipeline seeding 200k customer rows, this validation pass added 1.4 seconds of wall-clock time and caught 312 truncation candidates on the first run — all in company and address_line_1. That's a worthwhile trade. If you're also dealing with FK insertion order constraints in the same pipeline, run the length validation before the topological sort so you're not reordering corrupt data.
Two Mistakes Even Senior Engineers Make with VARCHAR Seeds
Trusting strict-mode flags at the session level. Teams set SET sql_mode = 'STRICT_ALL_TABLES' in MySQL or rely on Postgres's default behavior and assume the database will reject oversized strings with an error. It will — in some configurations. But connection pools, ORM layers, and migration tools frequently override session-level settings. SQLAlchemy's String(50) type doesn't enforce length on the Python side by default; it passes the value through and lets the DB decide. When your CI database is Postgres and production is Aurora MySQL with a legacy sql_mode, the behavior diverges silently. Don't delegate length enforcement to the database when you can enforce it at the generator.
Using a fixed random seed without re-validating on seed value changes. A common pattern is Faker('en_US', seed=42) to get reproducible fixtures. The problem: when you bump the Faker version or add a new locale, the sequence shifts, and the previously safe seed now generates a company name that exceeds your column width. The fix is to treat the validation pass as a required CI step — not a one-time check — so any seed value or library version change is automatically re-verified against your schema contracts.
What Teams Get Wrong About Truncation and Synthetic Coverage
Myth: "If the row inserted, the data is valid." A successful INSERT in a non-strict database is not a correctness guarantee — it's a storage confirmation. A VARCHAR(30) column holding "Springfield Manufactur" instead of "Springfield Manufacturing Co." will pass every NOT NULL check, every foreign key check, and every row-count assertion. The corruption only surfaces when application logic tries to match, display, or re-parse that value. When validating AI-generated or synthetic test data, explicit schema contracts — not database acceptance — are the right signal.
Myth: "Randomness gives you boundary coverage." Random generation will occasionally produce a 49-character string for a VARCHAR(50) column, but it won't reliably produce a 50-character string, a 51-character string, or a string composed entirely of multi-byte UTF-8 characters that consume more bytes than the column's byte limit allows. Boundary conditions require deliberate construction: generate strings at exactly max_length - 1, max_length, and max_length + 1, and include at least one multi-byte Unicode string in each fixture set. Hypothesis's st.text(alphabet=st.characters(whitelist_categories=('L',)), min_size=49, max_size=51) does this systematically where pure Faker-based generation will not.
Silent VARCHAR truncation is one of the cheaper defects to eliminate — once you've decided to enforce length contracts at the generator rather than the database. Add Pydantic StringConstraints to your model layer, wire a JSON Schema 2020-12 validation pass into your seed pipeline as a required CI gate, and add explicit boundary fixtures at max_length ± 1. If you're building a more complete synthetic pipeline, the referential integrity graph pattern is the next structural problem worth solving.
Note: This article is for informational purposes only and is not a substitute for professional advice. If you need guidance on specific situations described in this article, consider consulting a qualified professional.