A practical guide to building AI agents that actually work outside of demos — covering tool selection, memory, error recovery, and deployment patterns we've validated in production.
Rahul Sharma
Head of AI Engineering
An AI agent is a system that perceives its environment, makes decisions, and takes actions to achieve a goal. Unlike a chatbot that responds to a single prompt, an agent can plan multi-step tasks, use external tools, and adapt its behavior based on intermediate results. The gap between a demo and a production agent is enormous — and most engineering blogs skip straight past it.
This article is about that gap. We've shipped five production agent systems over the past eighteen months, across legal tech, fintech, and healthcare, through our AI agent development practice. Here's what actually matters.
LangChain has emerged as the dominant framework for building production AI agents. Its composable architecture lets you mix and match LLMs, tools, memory stores, and orchestration patterns without rewriting your integration layer each time you swap providers.
That said, LangChain is a framework, not a silver bullet. Its abstractions leak. You will read the source code. But for building agents that need to survive provider API changes, swap models mid-flight, and integrate with your existing observability stack — it's the right foundation.
We tried building our own thin wrapper three times. Each time we ended up recreating half of LangChain's internals. On the fourth project we stopped fighting it.
Before writing a line of agent code, make three architectural decisions: what tools your agent can call, how it maintains memory across turns, and what happens when it fails. Every other decision flows from these three.
Tools are the agent's hands. Design them with these constraints: each tool should do exactly one thing, accept simple inputs, and return structured outputs. An agent calling a search tool that returns raw HTML will waste tokens and hallucinate more than one that returns a structured JSON response with title, url, and snippet.
const searchTool = tool(
async ({ query }: { query: string }) => {
const results = await searchApi.query(query, { limit: 5 })
return results.map(r => ({
title: r.title,
url: r.url,
snippet: r.snippet,
}))
},
{
name: 'web_search',
description: 'Search the web for current information. Use for facts, recent events, or data not in training.',
schema: z.object({ query: z.string().describe('The search query') }),
}
)Keep your tool count under 10. Agents with more than 10 tools consistently underperform in our benchmarks — the model spends reasoning tokens on tool selection rather than the actual task.
Long-running agents accumulate context. Claude's 200K token window is generous, but at scale — thousands of concurrent agents, each with long histories — you need a tiered memory strategy: in-context working memory for the current task, a vector store for semantic recall across sessions, and a structured store for facts that must never be hallucinated.
We use a simple rule: anything the agent should "remember" that can be verified against a source of truth goes in the structured store. Anything that's useful-but-lossy goes in the vector store. The in-context window is only for the current task.
Production agents fail. APIs timeout. Tools return unexpected formats. Models occasionally refuse to call the right tool. Design your error handling before you write your happy path — not after.
Every tool call should have a max-retry count (we use 3) with exponential backoff. Every agent invocation should have a hard timeout. And every agent that faces a terminal error should produce a structured failure response — never a raw exception that surfaces to the end user.
The most important pattern is graceful degradation: if the agent can't complete the task, it should tell the user what it completed, what it couldn't, and what they need to do manually. A partial result is almost always better than an error message.
Agents are stateful, long-running, and expensive. This makes them unlike most web services you've deployed before. Treat each agent invocation as a workflow, not a request — log every step, checkpoint state to durable storage, and design for resumability from any checkpoint.
We run our agents on AWS Lambda with a max timeout of 15 minutes for simple agents and on ECS with no timeout for complex multi-day agents. For the latter, agent state is checkpointed to DynamoDB every 30 seconds. This kind of autonomous, self-correcting orchestration is the core of what we build in our agentic AI development engagements.
At 1,000 agent invocations per day, token costs compound fast. Three controls that matter: prompt caching (Claude's system prompt cache can reduce costs 60–70% for agents with long system prompts), output length ceilings (set max_tokens on every call), and model routing (use Claude Haiku for simple tool-selection steps, Sonnet for reasoning, Opus only for the highest-stakes final synthesis).
After five production deployments, these are the lessons that cost us the most to learn: test your agent against adversarial inputs on day one, not day 90. Build your observability stack before your agent logic. Design your fallback UX before your success UX. And resist the urge to add tools — the best agents we've built have fewer tools than you'd expect.
AI agents are one of the most powerful software primitives available right now. They're also one of the easiest to ship badly. The difference is a few architectural decisions made at the start of the project — if you'd rather have that judgment on your team from day one, you can hire AI developers who've made these calls before.
Tags
Continue Reading
Two approaches, very different trade-offs. RAG is faster, cheaper, and keeps your data current. Fine-tuning gives you style and behaviour control. Here's when each wins — and when to use both.
Most prompt engineering guides are written for demos. Production prompts face hallucinations, adversarial inputs, context length limits, and latency budgets. Here's what actually holds up.
Moving computer vision models from Jupyter notebooks to production serving infrastructure requires solving latency, throughput, model versioning, and data drift challenges. Here's our battle-tested playbook.
From AI agents to mobile apps — iMOBDEV builds what your business needs.
