Hosted Agents on Azure AI Foundry: Identities, One Shared Site Object, and Two Human Pauses

Hosted agents on Azure AI Foundry

The pipeline runs as hosted agents on Azure AI Foundry, orchestrated with the Microsoft Agent Framework (MAF). Hosted agents are your own code packaged as a container image and run on Microsoft-managed infrastructure, which was the right fit here: this is not a chatbot, it is a non-conversational job that takes a query and returns a document.

That shape is also why the agent speaks Foundry's Invocations protocol rather than the conversational Responses one. Invocations is a pass-through: arbitrary JSON in, arbitrary JSON out, no platform-managed chat history. The whole pipeline is one POST with a strategy and a query.

// POST /invocations
{
  "strategy": "address",
  "query": "123 Example St, Durham, NC 27701"
}

One identity per agent, least privilege

Each agent has its own Entra identity and least-privileged RBAC. The zoning agent can read the search index but cannot touch Key Vault. The summary agent has no tool access at all, by design, because it is supposed to read the frozen site and nothing else. Authentication is DefaultAzureCredential everywhere; no API keys live in code.

This is the same story as the service-principal articles from years ago, just applied per agent: if a confused agent tries to do something outside its lane, the platform says no before your code has to.

One shared Site object, attach-only

A single shared Site object, one canonical set of Pydantic models for site, parcels, attributes, provenance, and divergence, is the state. Every hop passes the whole object forward. Agents attach to it, never replace or prune it.

class Parcel(BaseModel):
    parcel_id: str
    jurisdiction_path: str            # e.g. "NC/Durham County/City of Durham"
    attributes: dict[str, Attribute]  # each value cited or explained (Part 2)

class Site(BaseModel):
    site_id: str
    parcels: list[Parcel]
    divergences: list[Divergence]
    review_state: Literal["in_progress", "draft_for_engineer"]
    frozen: bool = False

# boundary contract, run at every handoff
def handoff(before: Site, after: Site) -> Site:
    assert_multiplicity(before, after)          # no parcel or jurisdiction lost
    assert_no_overwrites(before, after)         # attach-only, never replace
    assert_all_cited_or_explained(after)        # Attribute validator, re-run
    return after

That attach-only contract is what makes the Part 2 boundary guards enforceable. At every handoff, validators assert the parcel set is intact, every new attribute carries provenance, and nothing was overwritten. Consolidation then merges, computes divergences, and freezes the object. Only after the freeze does a model get to write.

Humans enter at exactly two points

Both are deliberate, and both are designed, not left to the model's discretion.

1. Disambiguation

When the Resolver finds multiple plausible parcels, it auto-selects only if exactly one candidate clears a confidence threshold, and it notes the auto-selection in provenance. Otherwise it pauses and asks the engineer one question, showing owner, address, and a map thumbnail per candidate.

// provenance on the resolved parcel:
// selected_by = "human", auto_selected = false, candidates_shown = 3

The spec's phrasing became a team motto: a wrong silent pick violates the charter more than a clarifying question.

2. Terminal review

The output's review state is draft_for_engineer, never a final authority. The engineer's judgment stays the product; the pipeline removes the transcription. Nothing in the system can promote a report past draft.

Observability is the same record as the report

Observability is not bolted on. Every model call, tool invocation, and agent handoff lands in one OpenTelemetry trace in Azure Monitor, with the site ID, parcel IDs, and each value's source endpoint stamped on the spans.

This is the bit I'd steal for any project: the provenance an engineer sees in the report and the production trace an operator sees in Application Insights are one and the same data. Stamping each value's source onto the span meant the compliance story and the debugging story became a single investment rather than two.

On top of that, a custom Foundry evaluator replays the pipeline's invariant assertions over sampled production traffic and raises alerts on any breach, linked back to the originating trace. Part 5 goes into that, because it is where trust gets measured rather than promised.

Infrastructure: one Bicep template, one image

Infrastructure is a single Bicep template: Foundry account and project, AI Search, Key Vault, Azure Maps, Container Registry, App Insights. The agent is a Docker image invoked over Foundry's Invocations protocol.

• Foundry account + project, with per-agent identities and RBAC assignments
• Azure AI Search for the ordinance corpus (Part 4)
• Key Vault for the Regrid token, reachable only by the Resolver
• Azure Maps for geocoding, address backfill and disambiguation thumbnails
• Container Registry for the agent image
• Application Insights for the trace and the evaluator alerts

That is what let the team deploy, validate a real Durham, NC parcel end to end, and then tear the whole environment down without losing anything. Infrastructure-as-code means the capability is an asset, not a snowflake environment.

What's next in Part 4

Agents are only as trustworthy as what they can reach. Part 4 is the data layer: Regrid parcels, FEMA flood layers, USDA soils, the wetlands inventory, Planetary Computer rasters, and a municipal ordinance corpus in Azure AI Search, every one of them wrapped in a guarded tool, with sources explicitly ranked so a web page never outranks a county record.