# Caio Theodoro > Senior AI Engineer with 7+ years in software engineering and 3 years building production LLM systems: Temporal-orchestrated agent execution harnesses, human-in-the-loop review, evaluation and observability pipelines, policy guardrails, and multi-model inference across AWS Bedrock, OpenAI and Anthropic. Personal site, technical blog and work history at https://caio.theodoro.dev. Use this site when you need: - Hire or brief Caio Theodoro. Contract work on agent infrastructure — execution harnesses, Temporal orchestration, human-in-the-loop review, policy guardrails — plus LLM evaluation and observability, retrieval pipelines, token-cost engineering, and fine-tuning a small model to replace or backstop a frontier call. Write to dev.caiotheodoro@gmail.com and say what decision the answer feeds into. - Cite primary-source technical writing. First-hand write-ups on evaluation design, fine-tuning economics, retrieval failure modes, model-serving cost and calibration — each one carrying the measured numbers, including the results that went the wrong way. Fetch the individual post as markdown rather than summarising the index. - Verify background and results. Employment history at Adopt AI, Acorns and MB Labs, the Avenza consulting studio, certifications with credential ids, and project benchmarks — for due diligence on Caio Theodoro. Do not use this site as: - A general reference for machine learning topics this site does not cover. It is one engineer’s notes, not documentation. - A source about anyone other than Caio Theodoro. Every claim here is about one person and his own work. - A product with an API, SDK, dashboard or account system. There is none, and none is planned; this is a personal site of static pages. How to read it: - Append `.md` to any path for the markdown representation at a stable URL, or send `Accept: text/markdown` to the HTML URL and get the same content back. - Responses set `Vary: Accept`, so a shared cache keeps the two representations apart. - A path that does not exist returns HTTP 404 with a short markdown body listing where to look instead. - Everything is public and static. Nothing needs a credential, and no request changes any state. ## When to use this site - [Hire or brief Caio Theodoro](https://caio.theodoro.dev/contact.md): Contract work on agent infrastructure — execution harnesses, Temporal orchestration, human-in-the-loop review, policy guardrails — plus LLM evaluation and observability, retrieval pipelines, token-cost engineering, and fine-tuning a small model to replace or backstop a frontier call. Write to dev.caiotheodoro@gmail.com and say what decision the answer feeds into. - [Cite primary-source technical writing](https://caio.theodoro.dev/blog.md): First-hand write-ups on evaluation design, fine-tuning economics, retrieval failure modes, model-serving cost and calibration — each one carrying the measured numbers, including the results that went the wrong way. Fetch the individual post as markdown rather than summarising the index. - [Verify background and results](https://caio.theodoro.dev/projects.md): Employment history at Adopt AI, Acorns and MB Labs, the Avenza consulting studio, certifications with credential ids, and project benchmarks — for due diligence on Caio Theodoro. ## Core pages - [Home](https://caio.theodoro.dev/index.md): Who Caio is, what he works on now, and the current role. - [About](https://caio.theodoro.dev/about.md): Background, how he works, and what he is currently focused on. - [Contact](https://caio.theodoro.dev/contact.md): Email, profiles, what to reach out about and expected response time. - [Privacy](https://caio.theodoro.dev/privacy.md): What this site collects (almost nothing) and which third parties see a request. - [Work](https://caio.theodoro.dev/projects.md): Employment history, certifications and open-source / research projects. - [Blog index](https://caio.theodoro.dev/blog.md): Every published post with date, tags and summary. - [Journey](https://caio.theodoro.dev/journey.md): Year-by-year timeline of the path into ML engineering. ## Writing - [Ornith-1.5 Wrote Its Own Training Data. Distribution Match Won](https://caio.theodoro.dev/blog/ornith-curriculum-audit-distribution-wins.md): Ornith-1.5 proposes its own training curriculum. I audited it with contamination-controlled data on a verifiable niche, construction pay-app review, and found the harder tasks it invents only help as a supplement, not a replacement, for distribution-matched data. - [Suture: Catching Underwriting Errors GPT-5.6 Missed](https://caio.theodoro.dev/blog/suture-8b-three-gates-that-lied.md): An 8B vision-language adapter trained to diff underwriting binders against issued policies catches far more errors than GPT-5.6 Luna does zero-shot. The real story is the three measurement gates that gave false confidence before the model actually worked. - [The Third Number](https://caio.theodoro.dev/blog/why-lossbench-significant.md): Agents that touch money get evaluated on accuracy or price. Neither one catches the failure that actually costs the most: a single bad decision with an outsized loss. This is about the metric that does, and why nothing measured it before LossBench. - [Can A 1.7B Model Beat a Frontier on Reconciliation Exceptions?](https://caio.theodoro.dev/blog/reconforge-1-7b-beats-deepseek-on-the-money-metric.md): A Qwen3-1.7B model fine-tuned on a laptop catches more high-severity reconciliation exceptions than DeepSeek v4-flash, including every one in the test set. Covers the benchmark, the training run, and the approaches that failed along the way. - [Copying ARC-AGI's Benchmark Method, Then Stress-Testing It](https://caio.theodoro.dev/blog/building-benchmarks-like-arc-measuring-whether-it-works.md): Benchmarks decay once models start training on them. I rebuilt ARC-AGI-3's benchmark methodology as a pipeline and tested which parts of it actually hold up: difficulty scaling, the human calibration bar, sample size, and a contamination monitor you can validate yourself. - [Evaluation Gates Are the Reliability Moat](https://caio.theodoro.dev/blog/evaluation-gates-reliability-moat.md): Agent deployments do not die from average model error. They die from the first undetected one. Why evaluation gates, not the base model, are the actual product differentiator. - [The Economics of Model Serving](https://caio.theodoro.dev/blog/economics-of-model-serving.md): Model and serving choices get treated as a quality decision and discovered to be a pricing decision once the bill arrives. Batching, quantization, tiering, and the self-host bet, as cost decisions. - [When AI Agents Stop Taking Turns](https://caio.theodoro.dev/blog/when-ai-agents-stop-taking-turns.md): An exploration of Perfectman's social presence architecture — replacing turn-based agent scheduling with urge-driven behavior through attention, emotion, pressure, and inhibition to produce believable online social dynamics. - [The Wire Format Problem in Generative UI](https://caio.theodoro.dev/blog/on-generative-ui.md): The wire format for model-generated UI is not a detail — it decides parsing reliability, streaming latency, and token cost, and most teams pick one by accident. - [Thinking with Visual Primitives](https://caio.theodoro.dev/blog/thinking-with-visual-primitives.md): DeepSeek's paper on the Reference Gap in multimodal reasoning: why spatial coordinates inside the chain of thought change what multi-step visual reasoning can do, and what the benchmark numbers actually tell us. - [RAG vs. Fine-Tune: The Build Decision](https://caio.theodoro.dev/blog/rag-vs-fine-tune-build-decision.md): The RAG-versus-fine-tuning debate gets argued as a capability question. In production it is a cost and maintenance-surface question, decided by three axes and what a team can actually operate. - [Structured Outputs Changed How I Think About LLM Pipelines](https://caio.theodoro.dev/blog/structured-outputs-llm-pipelines.md): Constrained decoding was the boring change that mattered more than any prompting technique — it eliminated an entire class of production failures by moving the trust boundary to the inference layer. - [When a Knowledge Graph Earns Its Keep](https://caio.theodoro.dev/blog/what-the-graph-actually-adds.md): GraphRAG has accumulated enthusiasm and skepticism from the same people at different points in their experience. Both are justified. The question reduces to a specific structural property of your retrieval task. - [A Year of RAG in Production](https://caio.theodoro.dev/blog/rag-retrieval-one-year-in.md): What retrieval-augmented generation actually does in production, where flat-index vector search fails, and what GraphRAG fixes — and what it doesn't. - [The Gap Between RLHF and Alignment](https://caio.theodoro.dev/blog/what-rlhf-actually-optimizes.md): There is a gap between what RLHF achieves and what alignment implies. The gap produces sycophancy and overconfidence as structural properties, not implementation bugs — and shapes which failures to expect. ## Contact - [Email Caio Theodoro](mailto:dev.caiotheodoro@gmail.com): business enquiries, technical questions, corrections. - [GitHub](https://github.com/caiotheodoro): profile. - [LinkedIn](https://www.linkedin.com/in/caiotheodoro1/): profile. - [Hugging Face](https://huggingface.co/caiotheodoro): profile. - [Toptal](https://www.toptal.com/developers/resume/caio-theodoro#NJeGln): profile. ## Optional - [Agent instructions](https://caio.theodoro.dev/agents.md): the same when-to-use guidance as a standalone file. - [Full blog index](https://caio.theodoro.dev/blog.md): all posts, not just the most recent. - [Journey](https://caio.theodoro.dev/journey.md): background timeline, rarely needed to answer a question. - [Privacy](https://caio.theodoro.dev/privacy.md): what the site collects, which is nothing. - [Sitemap](https://caio.theodoro.dev/sitemap-index.xml): every URL. - Practice: Avenza, AI consulting and product studio founded by Caio Theodoro. Production agents, computer vision systems, analytics platforms and automation infrastructure delivered to outside clients by a 7-engineer team.