The closing brace

Once facts arrive one at a time, latency becomes a schedule.

Consider a support agent that classifies intent, checks urgency, and writes a reply. In a framework, those three fields are extracted from the complete object after generation finishes. All downstream actions (routing, escalation, sending the reply) begin at that point.

         Framework
(t=800ms)   |-- close }
            |-- route 
            |-- page oncall
(t=850ms)   |-- send reply


         Fact stream:
(t=50ms)    |-- /intent ---> route
(t=120ms)   |-- /urgency --> page oncall
(t=800ms)   |-- /reply ----> send reply

The routing decision fires at 50ms instead of 800ms. The escalation fires at 120ms. The reply still takes 800ms to generate, but the work that does not depend on it is already done.

Removing the closing brace lets the system do useful work during time it previously spent waiting. In multi-step workflows the savings compound: a researcher can start on step one while steps two through five are still being generated.

In a framework, every field arrives at the same time, after the closing brace. There is no way to say "I need this fact before that one." The object is atomic. With a fact stream, the field order in your schema decides when each fact becomes available. Put intent first because routing should fire early. Put urgency second because escalation should not wait for the reply. Put reply last because it is the longest field and should not block anything else. Schema design becomes priority specification.

The schema is a schedule.

Once facts stream individually, each one becomes a point where your code can act: route, log, validate, spawn work, or decide not to proceed. The object was hiding all of those decision points behind a single synchronization barrier. What used to need a supervisor agent, a context object, and a session manager now fits in one Python function:

async def chat(messages, agent, handlers):
    task = None
    async for event in agent(messages):
        if not event.is_leaf:
            continue
        match event.field:
            case "department":                                  # routing
                task = asyncio.create_task(handlers[event.value](messages))
            case "confidence" if event.value < 0.5:             # early cancellation
                if task: task.cancel()
            case "reply":
                messages.append({"role": "assistant", "content": event.value})
                await send(event.value)

Routing collapses to match. Today: supervisor LLMs that pick which sub-agent runs, or conditional edges wired between graph nodes. With a fact stream, the classification field arrives early and the consumer dispatches by value.

Early cancellation becomes possible. With a closed object you cannot act on early information because you do not have it yet. Here /department at 50ms spawns a research task; /confidence at 200ms decides whether to keep it. The schedule lets you race the model and bail when the bet was wrong.

Conversation memory is a list. Today: thread IDs paired with checkpointers, message_history= params, session-state objects. With a fact stream, you append to a list. The list is the memory.

Dependency injection is just function arguments. Today: RunContext[DepsT] generics, signature inspection, ctx.deps.x. With a fact stream, dependencies are closure variables or function parameters.

The agent stops being an object. Today: a class with a dozen kwargs (output_type, tools, deps_type, retries, message_history, model, …). With a fact stream, an agent is a function consuming a stream — agent above. The class collapses.

Plenty stays (partial-truth semantics, durability, retries, observability), at the runtime layer instead of the parsing layer.

The stream users see is really just carrying assertions:

("intent", "refund")
("account_id", "ACC-8821")
("confidence", 0.94)
("reply", "Hi Jane...")

These assertions become available one by one. The object is what you get if you collect them and freeze them. The stream is the record of the model's incremental commitments, closer to a transaction log over evolving structured state than to a serialized object.

The wire format is incidental. JSON Patch over SSE is one encoding (the .txt API also exposes the same stream over NDJSON); XML works too; any protocol that lets a model emit typed values at known paths works. And different models are good at different formats: some are reliable JSON generators, others do better with pseudo-Python, custom DSLs, or YAML. Pick whatever the model in front of you generates most accurately; the consumer never sees the wire anyway, they iterate over events with .field and .value.

For example, the same content could be emitted as XML:

<intent>refund</intent>
<account_id>ACC-8821</account_id>
<reply>Hi Jane, I've processed...</reply>

Or as YAML:

intent: refund
account_id: ACC-8821
reply: Hi Jane, I've processed...

The grammar engine constrains different tokens. The consumer still receives the same events, with event.field = "intent"= and event.value = "refund"=, and so on. The schema, the routing, the cancellation logic: none of it changes.

The schema starts doing most of the work. It defines what facts the agent can emit; the field order defines when each one becomes available; the types constrain what values are valid; and JSON Schema's composition primitives ($ref, allOf, oneOf, if/then/else) turn into orchestration primitives. Change the schema and you change the agent. The schema is the program; the model runs it.

This pattern shows up over and over: a synchronization point that nobody explicitly chose gets treated as natural, infrastructure builds around it, and when someone finally removes it, large parts of the infrastructure become optional.

At the bottom of every framework is (when they use structured outputs at all):

result = llm.invoke(prompt)

That one line is the source of more architecture than we usually admit. Frameworks, graph DSLs, parser layers, checkpointing schemes, state reconstruction, prompt replay: trace them back and you keep finding the same blocking call waiting for a complete object. Replace it with a fact stream of typed state, and most of what surrounds it turns out to be optional, or simpler.

All this architecture grew around a synchronization point we invented and then stopped seeing.