# The Wire Format Problem in Generative UI > The wire format for model-generated UI is not a detail — it decides parsing reliability, streaming latency, and token cost, and most teams pick one by accident. - Published: 2026-02-07 - Reading time: 21 min read - Tags: UX, ML - Author: Caio Theodoro (https://caio.theodoro.dev/about.md) - Canonical HTML: https://caio.theodoro.dev/blog/on-generative-ui --- The language model has always known what a chart looks like. It knows the axis labels belong to the x and y dimensions, that a bar chart's data is an array of category-value pairs, that a table has columns with types and rows with values. It knows this the same way it knows everything else: from having read enough of the web to have absorbed the structural vocabulary of interfaces. The problem is that the interface between the model and the application has always been text, and text is the wrong representation for this information. The standard output of a chat-based system is a markdown description of the thing the user asked for. "Here is a summary of your Q4 performance:" followed by a prose paragraph, or a markdown table if you're lucky. The downstream application receives a string. It either renders the markdown as-is, which is limited to what markdown can represent, or it parses the string with some combination of regex and hope, extracting the values it needs. This is not a pipeline; it's two systems that happen to communicate through natural language because that's what language models produce. The cost of this arrangement is mostly invisible because it has been normalized. But it is real. Natural language is a lossy encoding of structured information. Every dashboard described in prose is a prose description of a dashboard, not a dashboard. The model did the hard work — understood the data, selected the right visualization, identified the meaningful comparisons — and then serialized the result into a format that the rendering layer cannot directly consume. The application then reverses that serialization, imperfectly, and renders something. The model's actual output, the structured understanding of what should be shown and how, never crosses the boundary intact. --- ## The rendering layer is always there The assumption embedded in text-only output is that rendering is someone else's problem. The model produces text; the application figures out what to do with it. This is a reasonable division of labor for tasks where the output genuinely is text — an email draft, a code review comment, a document summary. It is a poor division of labor for tasks where the output is an interface. When a user asks an AI-powered analytics tool to "show me last quarter's revenue breakdown by region," the useful answer is a chart with the right data, possibly a table, probably some key callouts. Not a paragraph that says "Last quarter's revenue was distributed across four regions, with the Northeast accounting for 34% of total revenue, followed by the West at 28%..." The paragraph is a degraded proxy for the chart. It contains the same information, encoded in a form that requires more effort to process and cannot be interacted with. The mismatch has been tolerated because the alternative, getting structured UI out of a language model reliably, was genuinely hard. Models could be prompted to produce JSON, but the failure rate was significant enough that production pipelines required substantial defensive engineering. The structure of the output was a formatting preference, not a constraint, and formatting preferences are not reliably honored. What has changed is the ability to make the structure a constraint at the inference level rather than a preference in the prompt. --- ## The registry as a contract The architecture that OpenUI introduces is built around a specific insight: the component library is not just the rendering layer. It is the contract between the application and the model. When you define a `StatCard` component with a typed schema — title is a string, value is a string, trend is an enum of up, down, or flat — you are not only describing how to render a stat card. You are specifying exactly what the model is allowed to generate when it wants to show a stat card. The component definition is simultaneously the renderer spec and the output constraint. This is the same shift that structured outputs made for LLM pipelines generally: move the trust boundary to the inference layer. Instead of asking the model nicely to format its output in a particular way and then handling the cases where it doesn't, you constrain what it can produce. The model cannot hallucinate a component that isn't in the registry, because the parser that consumes its output validates against the registry. The model cannot pass an invalid prop value, because the schema rejects it before it reaches the renderer. The practical effect is that a well-designed registry makes component validation free. You get it at the schema level, not through defensive code downstream. A StatCard with a missing value prop fails at the parser, not at render time, not in a user-facing error. The failure boundary moves upstream, which is where you want it. This changes what "prompt engineering for UI" means. The system prompt is not a set of instructions about how to format the output. It is a specification of the available components, generated directly from the registry, that tells the model what it can build and in what shape. The prompt is the schema. The model's output is an instance of that schema. The renderer is the schema's runtime. ```mermaid flowchart LR A([User Request]) --> B[Model] B -->|generates| C[Component Tree] C --> D{Registry Validator} D -->|valid| E[Renderer] D -->|invalid| F([Parse Error]) E --> G([UI]) ``` --- ## What the wire format costs Representation affects more than parsing reliability. It affects cost directly. The standard approach before purpose-built DSLs was JSON. Describing a dashboard with three stat cards, a bar chart, and a data table in JSON is verbose in a specific way: all of the schema overhead — the field names, the nesting, the quotation marks, the structural tokens — is repeated at every level of the tree. A stat card in JSON is not just `{"type": "StatCard", "title": "Revenue", "value": "$1.2M", "trend": "up"}`. It's also the container object that holds it, the array it belongs to, the parent Grid that contains the array, the key-value syntax that wraps every field name. The structural tokens, the overhead of the format itself, accumulate. OpenUI Lang sidesteps this by using a line-oriented, positional-argument syntax where the structural overhead is minimal. The same stat card is `s1 = SC("Revenue", "$1.2M", "up")`. Arguments map to props by position in the schema order. Identifiers are defined on assignment and referenced by name for forward references. The format is designed for streaming: each line is independently parseable, which means the renderer can start rendering before the model has finished generating. The token reduction is not incidental to the design; it is the design. At inference scale, the number of output tokens determines latency and cost. A format that requires 67% fewer tokens than JSON for the same UI description is not a stylistic preference. It is a 2x improvement in throughput for UI-intensive applications, with corresponding improvements in time to first render and per-query cost. The wire format is the most direct lever on the economics of generative UI. The deeper reason this matters is that token budget constrains what's possible. A model with a 4096-token output budget that spends 60% of those tokens on JSON structural overhead has roughly 1600 tokens available for actual UI content. The same model using a compact format has 3500 tokens available. The difference determines whether complex multi-panel dashboards are feasible, whether the model can generate the full interface the user asked for, or whether it has to truncate or simplify. --- ## Streaming as a first-class requirement The naive implementation of generative UI has the model generate the complete UI description, then parse it, then render it. The user sees a loading spinner, and then the finished interface appears. This is strictly worse than a text-based response, where the user sees the model's output token by token as it streams. The streaming response feels responsive even if the total latency is the same. A blank-to-finished transition feels slow even if it takes fewer milliseconds. The streaming requirement is why the wire format matters beyond token count. A format is streaming-compatible if each partial output is parseable into a valid partial result that can be incrementally rendered. JSON is not streaming-compatible in this sense: a partial JSON object is not valid JSON, so you either buffer the entire response before parsing, or you implement incremental JSON parsing, which is non-trivial. Line-oriented formats with top-down tree structure are streaming-compatible by construction: each line, once complete, specifies a component and its props, and the renderer can instantiate it immediately. OpenUI's forward reference mechanism — the ability to define an identifier after it is used — is what makes the streaming order meaningful. The model can declare the root layout first, which gives the renderer enough information to establish the structural skeleton of the interface, then fill in the components. The user sees the layout container appear, then the individual components populate it, in the order the model chooses to generate them. If the model is instructed to generate the most visually prominent components first, the user sees the most important content earliest. This is the interaction between streaming and perceived performance: given the same total generation time, a streaming renderer that shows above-the-fold content within the first 10% of the token budget will feel substantially faster than one that shows nothing until generation is complete. The component ordering guidance in the system prompt is not an aesthetic preference; it is a UX performance instruction. --- ## The delta problem The current architecture regenerates the complete component tree on every turn of the conversation. The user asks for a Q4 revenue dashboard; the model generates the full tree. The user says "update the revenue figure to $1.5M"; the model generates the full tree again with one value changed. The second generation costs approximately the same tokens as the first, produces approximately the same output size, and renders approximately the same interface. The only thing that changed was one string in one component. This is the most significant unsolved problem in the current architecture, and it is not a small inefficiency. A multi-panel analytics dashboard might be 150 output tokens. At scale, every follow-up interaction that triggers a full regeneration is spending 150 tokens to change one. The cost multiplier for conversational UI, where the user is expected to iterate and refine, is an order of magnitude over what the interaction logically requires. The solution is a mutation protocol. After the initial render, the client holds the component tree in memory. Subsequent model turns can emit targeted patches: `PATCH s1.value "$1.5M"`. The parser applies the patch to the live tree rather than replacing it. The renderer updates only the affected components. The output tokens required drop from 150 to 4. The technical requirement for a mutation protocol is that the model has access to the current state of the component tree when generating a response, so it can reason about what needs to change. There are two approaches. The first is to serialize the current tree back into the model's context on each turn, which adds input tokens but is straightforward to implement. The second is to send a screenshot of the rendered interface as a vision input, which is more token-efficient for complex trees and gives the model an accurate representation of what the user sees. In the vision-input approach, the model is told what was changed in natural language, sees the current rendered state as an image, and can emit a minimal patch that addresses only the relevant component. The tree serialization overhead goes to zero; the model patches what it can see. The mutation protocol also changes the economics of interactive UI. If follow-up turns cost 4 tokens instead of 150, the viable interaction surface expands dramatically. Filtering, sorting, drilling down, toggling between views — operations that currently trigger full regeneration — become viable as streaming interactions. The latency drops from the generation time of a full tree to the generation time of a few tokens. This is the difference between a UI that updates on submit and a UI that updates on change. --- ## The layered system prompt The system prompt in a generative UI application is structured differently from the system prompt in a text-only application. In a text-only application, the system prompt is mostly instructions: how to behave, what to avoid, what persona to adopt. In a generative UI application, the system prompt is mostly schema: the component library, the syntax rules, the structural constraints. It is a substantial, mostly static document. This is significant because of how inference-layer caching works. Anthropic and OpenAI both support caching the prefix of the system prompt: if the first N tokens of a request's system prompt are identical to a previously cached request, those tokens are retrieved from cache at a fraction of the normal input token cost. The practical implication is that a system prompt structured to maximize its static prefix pays near-zero cost for the schema portion on every turn after the first. The natural structure of a generative UI system prompt has three layers. The first is the syntax specification — the grammar of the DSL, the rules about positional arguments, the streaming guidelines. This is entirely static; it never changes across requests or even across application versions. The second is the component library for the active shard — the set of components available for this class of request. This changes when the library is updated, but is otherwise stable across requests in a session. The third is the per-request context — the current conversation history, the user's expressed preferences, any data that needs to be threaded into the generation. This is dynamic and is not cached. In a production system structured this way, the effective input cost per turn converges toward the cost of the dynamic context layer alone. The syntax spec and component library, which typically account for 80-90% of the system prompt's token count, are free after the first request that establishes the cache. The architecture converts a large fixed cost per request into a small variable cost. At the usage volumes where generative UI becomes interesting — tens of thousands of interactions per day — this is a material reduction. The related optimization is intent-based sharding. A library with 45 components does not need to inject all 45 component definitions into every request. A user asking about data visualization does not need the form components. A user filling out a form does not need the chart components. A lightweight classifier on the incoming message routes the request to the appropriate component shard, and the system prompt includes only the components relevant to that shard. The active system prompt shrinks by 60-70% for well-classified requests, and the shard-level prompt can itself be cached once per session per shard. The combination of caching and sharding can reduce effective input cost to a few percent of the naive implementation. ```mermaid flowchart TD A[Syntax Spec] -->|cached| C[Active Prompt] B[Component Library] -->|cached| C D[Per-Request Context] -->|dynamic| C C --> E([Model Inference]) ``` --- ## Default elision and the protocol budget Beyond the system prompt, the output format carries its own optimization surface. A well-designed DSL for generative UI should make the common case cheap and the uncommon case explicit. The common case is a component instantiated with mostly default values and a few specific overrides. The uncommon case is a component where every prop carries a non-default value. The mechanism is default elision: props at their default value are not emitted. The schema defines defaults for every optional prop. The model, instructed to omit props that match their defaults, produces output that encodes only the meaningful departures from the default configuration. A stat card with the default variant, default size, and a flat trend emits two arguments: the title and the value. The trend, variant, and size are recovered from defaults at parse time. The output representation reflects the information content of the decision, not the full schema. This is not a new idea. Protobuf uses it at the binary level; HTML uses it for optional attributes; CSS uses it for inherited properties. The principle is that a compact encoding of structured data should charge only for the signal, not for the absence of signal. In a token-priced output format, the same principle directly translates to cost. Combined with alias compression — short identifiers for frequently-used components that the model learns from the system prompt — the output token budget per component drops to a few tokens in the common case. A stat card is `SC("Revenue", "$1.2M", "up")`. A grid of three stat cards is four lines. The model is not describing the interface; it is specifying it, in the most compact notation that preserves full fidelity. --- ## Skeleton-first and the perception of latency The quality of a streaming UI is not only a function of total generation time. It is a function of when meaningful content first appears and how the interface transitions from empty to populated. A blank screen that fills in at 200 tokens per second feels worse than a structured loading state that populates at the same rate. The skeleton-first pattern addresses this directly. Each component in the registry defines a skeleton: a placeholder rendering that occupies the correct layout space and signals the shape of the incoming content without requiring the actual values. When the model declares its structural intent early in the stream — using a first-line intent declaration like `INTENT dashboard 3-stats chart` — the renderer can instantiate skeleton versions of all named components before a single real value has been generated. The user sees the layout structure within the first few tokens of generation. Content populates the skeletons as the stream progresses. The interaction between intent declarations and forward references makes this work. Forward references allow the model to name identifiers before defining them. The renderer holds a slot for each forward-referenced identifier and renders a skeleton in its place. When the definition arrives, the skeleton transitions to the real component. This is the generative UI equivalent of progressive image loading: the structure is apparent immediately, and detail fills in over time. The perceptual effect is significant. Users report that interfaces that "build themselves" feel faster and more responsive than interfaces that appear all at once, even when the total latency is identical or slightly higher. The streaming renderer is not just technically correct; it is the right UX for this class of interaction. --- ## What structured outputs taught us The parallel to structured outputs for LLM pipelines is exact, and it is worth drawing explicitly because the same lesson applies. Before constrained decoding, getting structured data from a language model was a prompt engineering problem. You described the format you wanted, and the model usually complied, and sometimes didn't. The failures were small in frequency and large in consequence: a 3% parse failure rate in a five-step pipeline is a 14% end-to-end failure rate. The code around the model calls grew to be larger than the model calls themselves — retry logic, output normalization, fallback handlers. Constrained decoding moved the trust boundary. The output is structurally guaranteed at the inference layer, not through defensive parsing downstream. Parse failures, as a category of failure, cease to exist. The downstream code gets simpler, the failure surface shrinks, and the remaining failures — semantic errors, wrong values, poor reasoning — are the kind that evaluation and better prompting actually address. Generative UI is the same shift applied to interfaces. The component tree is structurally guaranteed by the registry contract. The renderer receives a typed element tree, not a string that might be a JSON object or might be a prose description of a JSON object or might be nothing useful at all. The failure surface is the failure to pick the right component or to populate it with meaningful values, which is a quality problem, not a pipeline reliability problem. Quality problems are tractable. Pipeline reliability problems generate defensive engineering that accretes indefinitely. The implication for how to think about generative UI architecturally: the registry is not a convenience feature for the renderer. It is the trust boundary. The definition of what the model is allowed to produce is also the definition of what the renderer can assume it will receive. This is the property that makes generative UI components composable with the rest of the application without requiring each consumer to defensively handle malformed input. --- ## What this doesn't solve A well-designed generative UI system guarantees structural validity. It does not guarantee semantic quality. A model that picks the wrong visualization, populates a chart with values that don't support the claim being made, or produces a layout that answers a different question than the one asked is producing structurally valid output that is wrong. The component renders correctly. The interface is misleading. This failure mode is harder to address because it is not a property of the pipeline architecture; it is a property of the model's understanding of the task. It is addressed by better prompts, better context, better models, and evaluation that measures whether the generated interface is actually useful — not just whether it is valid. Component design quality is also outside the scope of what the architecture solves. The registry defines the contract between model and application, but the contract is only as useful as the components it contains. A library of poorly designed components — too coarse, too fine-grained, semantically overlapping, inconsistently specified — produces valid output that is still a poor interface. The model can only select from the components it's given. If no component in the registry fits the user's request well, the model will use whatever fits imperfectly. The quality ceiling for generative UI is the quality ceiling of the component library. There is also a complexity ceiling. A large component tree with many nested levels and data dependencies strains both the model's ability to generate a coherent structure and the parser's ability to resolve forward references and data slot bindings consistently. The practical limit of what can be generated reliably in a single pass is lower than the theoretical limit of what the DSL can express. For very complex interfaces, the answer is likely decomposition: smaller, more targeted generation tasks that are composed at the application layer rather than generated as a single tree. --- ## The honest state Generative UI is past the point where the primary question is whether it works. The structural foundations are sound: compact DSLs reduce token overhead, schema-validated registries eliminate parse failures, streaming renderers match the latency profile of text-based systems. The economics are viable at the output volumes that matter for production applications, especially with prompt caching reducing the input-side cost to near zero for stable libraries. The open problems are the interesting ones. The mutation protocol, which eliminates full-tree regeneration on iterative interactions, is the most significant missing piece in the current architecture. Without it, conversational UI is expensive in a way that limits how much iteration the system can support. With it, the cost of a follow-up interaction approaches the cost of the change the interaction describes, which is the right economic model for a UI that's meant to be refined in conversation. The component design problem — how to define a registry that is simultaneously expressive enough to generate useful interfaces and constrained enough that the model reliably picks the right component for each use case — is an unsolved design problem with real practical consequences. Libraries that are too large result in model uncertainty and poor component selection. Libraries that are too narrow produce valid but limited interfaces. The design space is large and the evaluation criteria are not yet well-established. What is clear is that the output of a language model interacting with a user who wants to see data, fill out a form, or navigate a structured workflow should not be a markdown description of that interface. The model can generate the interface, and the infrastructure now exists to receive it — the remaining work is making the generation precise, the interaction iterative, and the component library good enough to match what the user actually asked for. --- More posts: https://caio.theodoro.dev/blog.md · About the author: https://caio.theodoro.dev/about.md · Machine-readable index: https://caio.theodoro.dev/llms.txt