JMESPath Predicate Scope Collapse on Nulls

Your JMESPath assertion returns an empty list and your test passes. The payload had ten records; eight had a null metadata.region; the projection silently dropped them. You validated two records and called it coverage. This is predicate scope collapse — not a bug in JMESPath, but a deliberate design choice that most assertion layers never account for.

The problem is structural. JMESPath projections (*, [], pipe chains) skip nodes that evaluate to null before the predicate even runs. That means filter expressions like items[?status == 'active'].id only see the subset of items where status is not null — and the assertion never knows what it missed. When your test data is sparse by design, this gap is wide.

By the end of this article you'll be able to identify collapse-prone expressions, instrument your assertion layer to surface the missing nodes, and write JMESPath queries that fail loudly when scope shrinks unexpectedly.

How JMESPath Projection Scope Is Defined — and Where It Breaks

JMESPath (RFC-compliant, used natively in AWS CLI, Boto3, Jmespath.py 1.x, and Postman test scripts) defines a projection as a operation that creates a new list by applying an expression to each element of an existing list. The critical rule, specified in the JMESPath spec, is that any element producing null during a sub-expression is excluded from the result — not replaced with null, excluded. A wildcard object projection like records[*].user.id over a list where half the user keys are absent returns a shorter list, not a list with nulls at the missing positions. Your assertion sees a smaller-than-expected array with no indication of what was dropped.

This is meaningfully different from how jq handles the same traversal. In jq (the command-line JSON processor, not a query language per se), [.records[].user.id] also omits nulls by default, but jq gives you error propagation, try-catch, and // empty alternation that make the omission explicit and catchable. JMESPath's design is optimized for terse AWS CLI filter expressions, not for exhaustive assertion coverage — a trade-off worth understanding before you choose it as your primary validation tool. When you need full-fidelity null-aware traversal in a pipeline, jq's --arg and paths(scalars) builtins give you structural introspection JMESPath simply doesn't have. Use JMESPath when you need portable, embeddable filter expressions inside Python or Postman; use jq when you need null-aware structural diffing in Bash pipelines or CI scripts.

Detecting and Containing Scope Collapse in Your Assertion Layer

The first step is making the collapse visible. Before asserting on a projected result, assert on the count of the input scope. If those two numbers diverge, you have silent drops.

import jmespath

payload = {"items": [
    {"id": 1, "status": "active", "region": "us-east"},
    {"id": 2, "status": None,     "region": "eu-west"},
    {"id": 3, "status": "active", "region": None},
    {"id": 4, "status": None,     "region": None},
]}

# Naive assertion — passes even though scope collapsed from 4 to 2
active_ids = jmespath.search("items[?status == 'active'].id", payload)
assert len(active_ids) == 2  # green, but what about the null-status records?

# Scope-aware assertion
total_items   = jmespath.search("length(items)", payload)
active_items  = jmespath.search("items[?status == 'active']", payload)
null_status   = jmespath.search("items[?status == `null`]", payload)  # always []!
# JMESPath cannot match literal null in a filter — this returns empty
print(null_status)  # []

# Correct approach: count the residual via Python
null_count = sum(1 for i in payload["items"] if i.get("status") is None)
assert null_count == 0, f"{null_count} items had null status — assertion scope was partial"

The items[?status == \`null\`] line is the trap. JMESPath's filter projection evaluates each element's status; a null value makes the comparison itself return null, which is falsy, so the element is excluded. You cannot use a JMESPath filter to find nulls — you need to step outside the query language. This is why null coalescing inside JMESPath expressions is a separate and equally dangerous failure mode: the language gives you no native way to distinguish "field was null" from "field was absent."

A practical mitigation is a scope-guard wrapper that runs before every JMESPath assertion in your suite:

def assert_jmespath(expression: str, payload: dict, expected, *, min_scope: int = None):
    """
    Evaluate a JMESPath expression and optionally enforce a minimum input scope.
    min_scope: if provided, assert the top-level list has at least this many elements
               before the projection runs.
    """
    result = jmespath.search(expression, payload)
    if min_scope is not None:
        # Derive the collection path from the expression (everything before '[')
        collection_path = expression.split("[")[0]
        scope = jmespath.search(f"length({collection_path})", payload) or 0
        assert scope >= min_scope, (
            f"Scope collapse: '{collection_path}' had {scope} elements, "
            f"expected at least {min_scope}. Expression: '{expression}'"
        )
    assert result == expected, f"Expected {expected}, got {result}"

Wrapping assertions this way caught a regression in a payment-service contract suite where a schema change made line_items[].tax_code nullable — 40% of projected records vanished silently and the suite stayed green for three days. Adding min_scope=len(payload["line_items"]) to each call made the failure surface immediately. For deeper treatment of how JMESPath flattens nested arrays and compounds this problem, the interaction between multi-level projections and null nodes is worth reading before you write assertions against deeply nested event payloads.

Where Senior Engineers Still Get Burned by Projection Scope

The most common mistake is trusting result length as a proxy for correctness. Engineers write assert len(result) > 0 and move on. That passes whenever at least one non-null node survives the projection — even if 90% of the scope was dropped. The mental model is "I got results, so the query worked." The correct model is "I got results from the subset of records that weren't null; I need to know what fraction that is." This happens because JMESPath's silent-drop behavior is buried in the spec and never surfaced by the library itself — jmespath.search does not warn you that it discarded nodes.

The second mistake is generating test data that's too clean. Factories built with factory_boy or Faker default to non-null values for every field unless you explicitly model nullability. Your local suite runs against pristine data; staging runs against data migrated from production where optional fields are null at realistic rates (often 20–60% for fields like middle_name, secondary_email, or metadata.*). The result is a suite that's structurally correct on clean data and silently wrong on real data. Sparse seed graphs that leave optional columns permanently null reproduce exactly this production distribution and should be part of your fixture strategy for any assertion that uses projections.

Myths About JMESPath Coverage That Need Correcting

Myth 1: A passing JMESPath filter assertion means the field exists on all records. It means the field existed on the records that survived the projection. The ones where the field was null or the parent key was absent were never evaluated. Myth 2: jq and JMESPath are interchangeable for validation. They overlap in use case but differ sharply in null semantics, error handling, and composability. JMESPath is embeddable in Python, Postman, and AWS tooling with zero dependencies; jq requires a subprocess or a Go/Rust binding. JMESPath has no try-catch, no path introspection, no streaming. Use JMESPath for portable filter expressions inside application code and test scripts; use jq for structural diffing, null-aware traversal, and pipeline transformation in CI Bash scripts where a subprocess call is acceptable.

Myth 3: Randomized test data gives you null coverage automatically. Faker and Mimesis generate random values, not random presence. Unless you explicitly set a field to None with some probability — fake.optional(fake.word(), p_empty=0.3) in Faker's optional provider — every generated record has the field populated. Real null coverage requires intentional sparsity modeling, not randomness. This is the same root cause behind assertion gaps when optional JSON fields collapse to null in contract testing: the generator and the validator both assume presence, so the null path is never exercised.

Predicate scope collapse is quiet, reproducible, and fixable. Add a scope-guard wrapper to your JMESPath assertions, model nullability explicitly in your seed factories, and verify result length against input length — not just against an expected value. If you're building assertion infrastructure from scratch, read the JMESPath spec section on projection semantics before writing your first filter expression; the five minutes will save you a week of debugging silent green tests.

Note: This article is for informational purposes only and is not a substitute for professional advice. If you need guidance on specific situations described in this article, consider consulting a qualified professional.