[cd ~]

# a eulogy for el jefe papacito

What building a cloud agent for Creatorwood taught me about harness engineering & verifying code.

2026-10-20 · 6 min

*wrote this as an internal company blog on August 5th. Jefe has since been deleted and replaced by Cody; internal usage has skyrocketed to over 7,600 messages. More on this coming soon.

I spent the last 2 months of freshman year at UMD deep diving into the secret sauce behind companies building cloud agents. In a stroke of genius, I thought of a name and started cooking on: goodboy.

I quickly fell in love and used goodboy to keep building during painstaking lectures where my professors had banned laptops 🤦‍♂️. I eventually found myself sending off feature ideas between sets at the gym.

Soon after the success of goodboy, I joined Creatorwood full time. My first quest was to take the learnings from goodboy and build a cloud agent integrated into the company repo, slack, and infrastructure. Here, El Jefe Papacito was born.

Lets start with Jefe's two jobs: code review and engineering tasks. These are two separate beasts and should be treated as such while building.

The 4 stage pipeline of doom

Inspiration behind splitting up the pipeline: AI Reviewing AI Code: Does This Make Sense? by Daksh Gupta.

Easily the largest pitfall of Jefe was the predefined 4-stage pipeline I forced it to follow: planner → implementer → reviewer → PR creator. Each agent was restricted in its role, and its output was a markdown file that was passed to the next agent.

The issue with markdown files is that it's a lossy handoff process. The planner agent would read through 40 files, write the plan.md file, and then exit with all of the implicit decisions & context wasted. The implementer agent would come along next and re-read the same 40 files and prepare for implementation.

Four agents in a relay, each passing a thin plan.md to the next while its stack of context goes in the trash

When using an agent for the art of engineering, instead of forcing it through a rigid process, give it a goal and the ability to verify its work. Allow it to use any combination of tools/subagents to complete the task. Less is more and the result is beautiful.

Review is the one place where splitting into multiple agents actually improves the output. This is a great writeup by Cursor about how they built Bugbot: Building a better Bugbot by Jon Kaplan. Cursor achieved best results by running eight parallel agents with a randomized diff order. Each agent came back with its own findings, influenced by the ordering of files in the diff, and then an orchestrator agent took the most repeated bugs found across the 8 agents as highest prio. Always split for coverage, never for the process.

Five agents review shuffled copies of the diff in parallel; their findings converge on the most found bug

Custom Harness vs. Agents SDK

I built goodboy on the back of the popular minimalist harness Pi Coding Agent. The allure of Pi is total control. The harness comes with literally nothing. No subagents. No MCP. You decide what you value in a harness and either build it yourself or use an extension built by someone else.

This was a losing battle I fought on two fronts.

The first was building all of it. When developing a cloud agent on an external server with limited observability, one of the optimizations is to reduce the surface area for errors. Using Pi, I was writing execution, custom json SSE streaming, injecting a limited env, tools, tool context, cancellation of tasks, parsing of results for cost & tokens, and also balancing the system prompts to work for different classes of models. Each one of these boundaries was a place for an error waiting to happen.

The second battle was maintaining the moving target. New models came out, the extensions changed, structured outputs shifted. On top of iterating on the layer on top of the harness, the harness kept evolving.

Building Jefe, I switched over entirely from Pi to AgentsSDK which was purpose built for autonomous agents. It brought everything that made Pydantic magical in Python. Typed and validated outputs were supported first-class, which turned out to be the biggest reliability win.

A quality SDK built and maintained by the model creators is a night and day difference in the reliability of the agent once it's running in the cloud.

Pick and choose your battles. Don't fall for shiny object syndrome here.

Verification

The first step in making a successful cloud agent is giving it a way to verify its work. In a product like Creatorwood's which is heavily UI workflow driven, just unit and integration tests won't cut it. The only way for the agent to understand what it built is to go through the flow as a user would.

For this, I had the reviewer agent spawn a subagent to go through the frontend and verify the implementation of the feature. It used that to loop through making changes and verifying its work.

When I gave Jefe a task, it went off, and came back after an hour with a PR. When the PR matched the solution I had envisioned, it felt magical. The painful part was when something small was wrong. That's when I had to spend the energy, context switch, and pull the branch locally to tweak what I wanted. A cloud agent should be ready to take feedback, quickly make edits, and change course as the user steers.

Cost is in the turns

When running cloud agents, there are two types of cost. The obvious one is money from token spend, but worse is time burnt by wasted turns.

After setting up the pipeline, I reviewed the turns each agent took to reach the final output and found all the repeated tasks each agent performed. I turned those repeated steps into tool calls and scripts that the agents could call deterministically. For example, after putting together the PR description, Jefe called a create_pr tool that opened the PR against staging from whatever branch it was on.

Every step made deterministic shrinks the surface area for errors and saves turns. LLMs should only spend tokens on judgement and non linear decisions.

One agent walks in circles wasting turns; another pulls a create_pr lever and gets a PR in one step

Write reviewable code

Source: Nobody Is Going to Read the Code by CodeRabbit.

Founder of Energy & previous researcher at OpenAI, Midjourney said it best:

Every piece of code written by Jefe was prompted to be the simplest version possible. The code was easy to review, making sure there was no backlog of PRs. It followed the following principles:

  • Small and scoped PRs - one PR does one task, nothing over 1500 lines
  • PR descriptions had the assumptions, premises, and diagrams explaining issues and code changes
  • Demos for anything frontend related (video walkthroughs)

Semantic Diffs - was on the Jefe roadmap

Inspiration: Semantic Diff in CodeRabbit: See What Actually Changed.

I strongly believe that line by line diffs aren't the best way for agents to review code. Breaking down the raw diff into objects known as the semantic diff is a more efficient way to capture the true bugs in code: logic issues and edge cases that will be caught in production.

Handed a semantic diff, agents can review code based on its intent, not noise like the line numbers and function names. This is easily the largest upgrade to Cody's PR reviews.

More Inspiration

A blog on giving agents runtime context:

An amazing article on the responsibilities of a harness:

The hidden cost of agentic engineering:

Devin's automated testing:

Artwork inspiration: Ian Xiaohei Illustrations