Prompt Engineering for API Creators: Why You Need Two Agents for the Job

"It HAS to work"

API creation is all about consistency. If you've ever been knee-deep in the trenches of building or maintaining an API, you know the unspoken mantra: don't piss off your clients. They don't want surprises, they want reliability. They want errors served up on a silver platter, neatly formatted, and easy to handle. No one is in the mood to deal with your unexpected surprises or the endless debugging that comes with them.

The hubris of we engineers

So, imagine my frustration as I'm working on an API designed to analyze images, extract key information, and deliver it seamlessly across multiple clients with unpredictable image sets. I thought I had it all figured out. My prompts were smart, efficient, and, in my mind, downright brilliant. I was approaching it from a holistic standpoint—telling the AI to examine the images, identify what it could see, and return the data in the format I specified. I was building a machine, an efficient and well-oiled one, or so I thought.

When Primitive Prompts Outperform

Then enters my boss. Now, he's not your typical engineer, but he gets the bigger picture. His approach is different—he's obsessed with the end goal in a way that I wasn't considering. While I'm trying to wrap up the API, he's throwing in these caveman prompts that, to my utter disbelief, are outperforming my prototype. And it grates on me because his prompts look primitive. They’re messy, unrefined, and I would never serve them to a client. But here's the kicker—they work. Not just work—they excel. And that’s when it hit me like a ton of bricks: I needed two agents to do this job.

The Reality of Large Language Models

Why? Because large language models (LLMs) aren't thinkers—they’re next-word predictors. They’re not “thinking” in the human sense; they’re translating input through math to produce an output. That's it. So, when you tell the AI to think internally, to do some mental gymnastics before outputting, it doesn’t really compute the way you think it does. The realization was so stark that I’ve almost completely abandoned old lines of thinking when it comes to prompt engineering.

The Epiphany: pencil & paper(ish)

The epiphany? If you want the AI to do something well, to think about something, tell the AI to write it out. LLMs operate in computers where memory is a tool to operate the language model—not a place where the model stores thoughts or ideas. If you ask the AI to consider something internally, you’re losing out on quality. For instance, if you want the AI to explore ten categories before choosing one, have it write out all ten categories, then pick one, and proceed from there. Make it write everything down as if it’s working with paper and pencil. This approach mirrors how humans process complex tasks—we write things down to clarify our thoughts.

The Key to Better Results

Yes, this creates a nightmare of output, but thankfully you can make another AI agent to help you find the data (parse) you want to serve to your client. The key takeaway: never ask an AI to perform broad creative thinking AND narrow, formatted output in the same prompt or with the same agent in a single request.

When you need the AI to go wide, use language that encourages expansive thinking on paper. When you need precision, use a separate agent designed to parse, scour, and format that broad output into something predictable and end client-friendly.

Separate and Succeed

In short, if you want better results, stop trying to get a single AI agent to be a jack-of-all-trades. Separate your processes. Let one agent be the creative genius and another the meticulous editor. Do this, and I swear, you'll see the quality of your outputs skyrocket.