The Model Narrates, Code Decides: Four Rules That Keep Facts on a Deterministic Path

The boundary, drawn early and defended everywhere

But going back to the customer's question from Part 1, "can we automate this?", the honest answer was "yes, if we agree on one rule first". The architecture rests on one boundary, drawn early and defended everywhere:

Same pattern as service principals years ago: one identity, one place to grant access, instead of credentials scattered everywhere. Here it's one place where facts are allowed to come from, instead of "the model said so" scattered through a report. Four rules make that boundary real. Let's go through them one at a time.

Rule 1: every value carries provenance

Each attribute in the site record carries its source, its capture timestamp, and a locator pointing back to the authoritative record: the API endpoint, the document chunk, the feature service query. Not "FEMA", but the FEMA flood layer query for that geometry. Not "the zoning code", but the ordinance section itself. An engineer can click from any number in the report to where it came from.

In practice the attribute model looks roughly like this (simplified from the real Pydantic models, but the shape is the point):

class Provenance(BaseModel):
    source: Source              # REGRID, FEMA_NFHL, ORDINANCE_CORPUS, WEB, ...
    captured_at: datetime
    locator: str                # endpoint + params, or corpus chunk id
    auto_selected: bool = False # set when the Resolver picked without a human

class Attribute(BaseModel):
    value: Any | None
    provenance: Provenance | None
    reason_null: str | None = None

    @model_validator(mode="after")
    def cited_or_explained(self):
        if self.value is not None and self.provenance is None:
            raise ValueError("value without provenance")
        if self.value is None and not self.reason_null:
            raise ValueError("gap without reason_null")
        return self

That validator is the whole philosophy in six lines: a value without a citation is a bug, and so is a blank without an explanation. It runs at every hop, so an agent physically cannot attach an uncited number to the site.

Rule 2: every gap carries a reason

Services fail. Coverage has holes. A soils query times out, a wetlands layer has nothing for that county. When that happens the field renders in the report as an em dash with a reason_null behind it: "FEMA NFHL query returned no features for parcel geometry", "USDA Soil Data Access timed out after 2 retries". The gap is visible, never papered over.

The rule that follows from it is just as important: a partial failure never silently drops a parcel. If the geospatial agent gets three of four parcels enriched, the fourth is still in the site, with its gaps explained, not quietly missing from the report.

Rule 3: multiplicity is never collapsed

A "site" is rarely one parcel. It is often several parcels spanning several jurisdictions, an assemblage where one lot sits in the city and the next sits in the county. The naive design flattens that into one record early, and from then on nobody notices the second jurisdiction is gone.

So agents attach data per parcel, and a shared assert_multiplicity guard runs at every pipeline boundary. It hard-fails if a parcel or a jurisdiction ever goes missing between hops:

def assert_multiplicity(before: Site, after: Site) -> None:
    lost_parcels = set(before.parcel_ids) - set(after.parcel_ids)
    lost_juris   = set(before.jurisdiction_paths) - set(after.jurisdiction_paths)
    if lost_parcels or lost_juris:
        raise MultiplicityViolation(
            parcels=sorted(lost_parcels), jurisdictions=sorted(lost_juris))

It is unit-tested against the hard cases: a real four-parcel assemblage and a synthetic two-jurisdiction divergence. Part 5 covers the test suite, but this guard is the piece I'd write first on any similar project.

Rule 4: conflicts become divergences, not averages

When parcels disagree on a material attribute, two zoning designations, different flood zones, the consolidation stage does not pick a winner and does not average. It emits an explicit divergence entry flagged requires_engineer_review, with a deterministic source-authority recommendation attached as an annotation. The system never resolves the conflict itself.

The recommendation exists so the engineer isn't starting from zero, but it is an annotation, not a decision. The divergence stays in the report until a human closes it.

Where the model is actually allowed

Only two stages involve a model at all:

1. Constrained extraction from cited ordinance text, turning a retrieved section into a structured value like a setback or a parking ratio.
2. The executive summary, written from the frozen, consolidated site.

Both are post-checked by plain code. The summary agent reads the frozen Site object, it has no tools and no data access, and a deterministic grounding check verifies that every sentence maps to an existing cited attribute and that every divergence is surfaced in the narrative.

// for each sentence: resolve to ≥1 cited attribute on the frozen Site → else FAIL
// for each divergence on the Site: must appear in the narrative → else FAIL
// on FAIL: reject the draft, emit the trace, do not render

The pattern generalizes: every instruction to the model exists twice, once in the prompt and once as a deterministic check on the output. The prompt gets compliance most of the time. The check makes "most of the time" irrelevant.

The business translation

The parts of the report a firm could be sued over behave like software, not like a chatbot. The model's creativity is confined to prose that is mechanically checked against the data it summarizes. That is the sentence I used with the customer's partners, and it is the sentence that got the project approved.

What's next in Part 3

Rules are only as good as the runtime that enforces them. In Part 3 we look at how the pipeline actually runs: hosted agents on Azure AI Foundry, orchestrated with Microsoft Agent Framework, each with its own Entra identity, all passing one shared Site object forward under an attach-only contract, with two deliberate human pauses and one OpenTelemetry trace per run.