JSON Schema validation gives you a green checkmark, and your pipeline still ships garbage downstream. The schema said order_total is a number — it is. It didn't say it can't be negative, can't exceed a known business ceiling, or must equal the sum of its line items. Structural correctness and semantic correctness are different problems, and most teams only test the first one.

This gap is where data contracts actually break. A contract between a producer and a consumer isn't just about shape — it's about meaning, range, referential integrity, and temporal consistency. JSON Schema 2020-12 covers shape. Everything else is your responsibility, and most teams haven't built the tooling to enforce it.

By the end of this article you'll understand exactly where JSON Schema's validation boundary ends, how to layer deep data assertions on top of it, and which tools (Pydantic, Great Expectations, Pact, custom JSONPath checks) close the gap without drowning your test suite in maintenance.

Where JSON Schema's Contract Ends (And Where the Real Breaks Begin)

JSON Schema 2020-12 is a structural validator. It can enforce types, required fields, string patterns, array lengths, and enum membership. What it cannot enforce natively: cross-field invariants, foreign-key relationships, business-rule ranges derived from other fields, or temporal ordering between events. These are the constraints that actually define whether a payload is meaningful to a downstream consumer — not just parseable.

A data contract, in the Pact or AsyncAPI sense, is a bilateral agreement about behavior, not just syntax. When you treat JSON Schema as the full contract, you're essentially saying "I promise to send you a valid JSON object" — which is the weakest possible guarantee. Contract testing with realistic payloads requires asserting on values, relationships, and business invariants, not just field presence. The schema is a floor, not a ceiling.

Building Deep Data Assertions on Top of Schema Validation

Start by separating your validation layers explicitly. Schema validation (structural) runs first and fast — use jsonschema or Pydantic v2 for this. Deep data assertions (semantic) run second and are where you encode actual business rules. Mixing them produces brittle schemas full of minimum, maximum, and if/then clauses that nobody maintains.

# Layer 1: structural — Pydantic v2
from pydantic import BaseModel, model_validator
from typing import List

class LineItem(BaseModel):
    sku: str
    quantity: int
    unit_price: float

class Order(BaseModel):
    order_id: str
    order_total: float
    line_items: List[LineItem]

    @model_validator(mode="after")
    def total_must_match_line_items(self):
        computed = sum(i.quantity * i.unit_price for i in self.line_items)
        if abs(self.order_total - computed) > 0.01:
            raise ValueError(
                f"order_total {self.order_total} != computed {computed:.2f}"
            )
        return self

This Pydantic model does what JSON Schema can't: it asserts a cross-field invariant — that order_total is the actual arithmetic sum of the line items. The model_validator runs after field-level validation, so you get structural failures first and semantic failures second. This separation makes failures diagnostic rather than cryptic.

For event-stream or pipeline data, push these assertions into Great Expectations or a custom assertion layer. GE's expect_column_pair_values_A_to_be_greater_than_B and custom ExpectationConfiguration objects let you encode rules like "shipped_at must be after created_at" as versioned, reusable checks. At one payments team, adding 14 semantic expectations to an existing GE suite caught a date-timezone bug in under 3 minutes that had been silently corrupting 6-month-old event replays — a bug that JSON Schema validation had been passing for weeks.

# Layer 2: semantic — Great Expectations custom expectation (abbreviated)
from great_expectations.expectations.expectation import ColumnPairMapExpectation

class ExpectShippedAtAfterCreatedAt(ColumnPairMapExpectation):
    map_metric = "column_pair_values.shipped_after_created"
    # register and configure via expectation suite JSON

For API-level validation, complement Pydantic with JSONPath or JMESPath assertions in your test layer. JMESPath is particularly clean for asserting on nested structures without deserializing the whole document into a model:

import jmespath, pytest

def test_first_item_sku_is_not_empty(order_payload: dict):
    result = jmespath.search("line_items[0].sku", order_payload)
    assert result and result.strip(), "First line item SKU must be non-empty"

def test_no_negative_quantities(order_payload: dict):
    quantities = jmespath.search("line_items[].quantity", order_payload)
    assert all(q > 0 for q in quantities), "All quantities must be positive"

These two tests take microseconds and encode rules that would require awkward JSON Schema contains + if/then gymnastics. Keep them in a dedicated test_assertions/ directory so they're clearly separated from schema tests and easy to audit when the contract evolves.

Where Senior Engineers Still Get Burned on Data Assertions

The first mistake is encoding business rules inside JSON Schema using if/then/else and $defs to approximate logic that belongs in application or test code. The result is a schema that's 400 lines long, impossible to diff in PRs, and breaks every time a product rule changes. JSON Schema is a description language, not a rules engine. When you find yourself writing if: {properties: {status: {const: "shipped"}}} followed by a then block asserting shipped_at is present, move that logic to Pydantic or a dedicated validator function instead.

The second mistake is asserting only at the API boundary and ignoring what happens to the data as it flows through Kafka topics, dbt transformations, or Postgres materialized views. A payload that's valid at ingestion can be semantically broken by the time it reaches an analytics consumer — truncated strings, coerced nulls, timezone stripping. Property-based testing with Hypothesis is one of the few techniques that systematically surfaces these mid-pipeline corruptions, because it generates edge-case inputs that manual fixture authors never think to write.

What Most Teams Get Wrong About Data Contracts and Deep Systems

Myth 1: Pact contract tests replace semantic assertions. Pact verifies that a provider's response matches the shape a consumer recorded. It does not verify that the values inside that response are internally consistent or business-valid. You need both layers. Myth 2: If the schema is shared and versioned, the contract is covered. Schema versioning handles evolution (backward/forward compatibility), not correctness. A v2 schema can be perfectly backward-compatible while still allowing discount_pct: 150 or event_time set to 1970. Schema registries (Confluent, AWS Glue) solve the evolution problem, not the data quality problem.

Myth 3: Production data clones are the safest test data because they're "real." Prod clones carry PII, stale referential integrity, and edge cases that are artifacts of historical bugs — not intended behavior. They also make your test suite non-deterministic, because the data changes. For tests that actually stay green, you need generated data that satisfies your semantic assertions by construction, not by luck. Factories built with factory_boy or Pydantic model defaults, seeded with known values, are reproducible and auditable in a way that prod snapshots never are.

JSON Schema is a starting point, not a finish line. Layer Pydantic validators for cross-field invariants, Great Expectations for pipeline-level data quality assertions, and JMESPath checks for targeted value assertions in your API tests. Version all three alongside your schema. If you're building this infrastructure from scratch, the JSON Schema and test data guide covers the structural foundation — come back here once that layer is solid and you're ready to go deeper.

Note: This article is for informational purposes only and is not a substitute for professional advice. If you need guidance on specific situations described in this article, consider consulting a qualified professional.