Something in AI engineering has become increasingly interesting to me.
People are jumping directly into AI agents without first understanding the engineering foundations that made agents possible. The path usually looks like this. A framework is released. A model gets connected to a tool, a vector database is added, a few prompts are written, and an orchestration layer ties it together.
The system works, at least on the inputs it has seen so far, and the builder starts calling themselves an AI engineer.
To be clear, I am in favor of agents. They are becoming an important abstraction for building software around increasingly capable models. An agent is still a system, though. It has inputs, outputs, dependencies, state, failure modes, latency constraints, security boundaries, evaluation requirements, infrastructure requirements, and operational costs.
Those concerns did not appear because agents became popular. They existed long before agents, and most of them are general engineering problems that AI systems inherit.
AI engineering did not begin with agents
Before today's agent frameworks, there was machine learning. Before modern machine learning systems, there was statistical learning. Before that came decades of research in artificial intelligence, optimization, probability, search, computer science, distributed systems, databases, software engineering, and mathematics.
The modern AI stack is the result of all of those layers accumulating over time. That history matters because every abstraction hides complexity.
What one framework call hides
One call, fourteen concerns. A high-level agent framework puts them behind a single interface. Every one of them is still running underneath.
A high-level agent framework can hide all of the following:
- Model access. Model invocation, context construction, and retrieval.
- Control flow. Tool routing, state management, retries, concurrency, and caching.
- Integration. Authentication, API failures, and observability.
- Lifecycle. Evaluation, deployment, and cost management.
Hiding complexity leaves it in place. The abstraction makes development faster, and the engineering underneath still has to be done.
The question production actually asks
Modern AI development is often presented as if the central problem were "How do I make the model more intelligent?" In production, that is only one part of the problem. A more important question is:
That question covers fifteen failure modes. I group them by where they originate, because the origin usually determines the fix. Open any row for the failures that start there.
Software engineering still runs through most of AI engineering
This is one of the most important things I have learned while working through the rapid evolution of AI systems: a surprisingly large portion of AI engineering is still conventional software engineering.
- Design and code. Architecture, abstraction, APIs, documentation, and code quality.
- Data and state. Databases, caching, and concurrency.
- Systems. Networking, distributed systems, infrastructure, and performance engineering.
- Delivery. Version control, testing, CI/CD, and containerization.
- Operations. Observability, logging, error handling, security, and incident response.
The AI component changes how one part of the system behaves. Everything around it still has to be engineered.
The probabilistic nature of modern AI often makes these disciplines more important. Traditional software generally tries to produce deterministic behavior from deterministic inputs. AI systems frequently operate under uncertainty, so the surrounding engineering has to compensate for it.
Validate model outputs against a schema before anything downstream consumes them. Put timeouts and bounded retries around external calls. Define fallbacks for when a component degrades. Gate releases on evaluation results.
Machine learning fundamentals still matter
There is another layer that is increasingly easy to skip. If you want to work seriously on AI systems, understanding how models learn is still valuable. You should be able to reason with concepts such as these:
- Learning and inference. Training and inference, supervised and unsupervised learning, optimization, loss functions, fine-tuning, and inference trade-offs.
- Generalization and measurement. Overfitting, generalization, bias and variance, evaluation, calibration, and model uncertainty.
- Data and representation. Data distribution, distribution shift, feature representation, and embeddings.
The goal is enough understanding to reason about what the model is doing, which is well short of deriving every optimization algorithm from first principles.
Without it, you are operating the system from the outside. You can call the API, configure the framework, and connect the tools, but when the system behaves unexpectedly, you may not know why. That is where the difference between framework usage and engineering becomes obvious.
Distribution shift
A concrete case. An agent that scored well in offline evaluation starts choosing worse tools in production.
One of the first hypotheses to test is distribution shift between the evaluation set and live traffic. Testing it requires knowing what that shift looks like in your data, and that is an ML question.
Compare the inputs your agent sees in production with the evaluation set it was measured on. If they have drifted apart, the offline score no longer describes the system you are running.
The agent is one layer of a larger stack
A production agent can be described as a stack. The exact architecture varies by system, but the principle holds.
The production AI stack
Agent orchestration is the eighth of ten layers. Most of what determines whether it works sits below it, and most of what determines whether it keeps working sits above it.
Read it from the bottom up. Computer science and mathematics sit at the base, followed by software engineering, machine learning, deep learning and foundation models, inference infrastructure, retrieval and knowledge systems, and tool integration. Agent orchestration comes eighth. Evaluation and observability, then production operations, sit above it.
The agent is one abstraction sitting on top of many others.
This is why learning only an agent framework can create a dangerous illusion of competence.
Vibe coding an agent has a ceiling
I have nothing against AI-assisted coding. It is one of the most powerful productivity shifts software engineering has seen. The problem begins when generated code becomes a substitute for understanding. Compare two statements:
The first shows that you can produce software. The second shows that you can engineer a system. AI coding tools can dramatically reduce the cost of implementation, but they do not automatically transfer architectural judgment to the developer. That gap matters more as systems become more autonomous.
A practical test for any agent you have shipped is whether you can answer these nine questions.
Why did it select that tool?
Tool routing is one of the concerns a framework hides. The decision still has to be explainable after the fact.Why did retrieval produce that context?
Irrelevant retrieved context is one of the fifteen failure modes, and it shapes every answer downstream.Why did that request fail?
Model, context, infrastructure, data distribution, security, or cost. The origin usually determines the fix.Where is state stored?
Two tool calls can modify the same state at once, and a service can fail halfway through a write.How do retries behave?
If a retried operation has side effects and is not idempotent, it runs again. Retries need a budget.How are failures recovered?
A downstream service can fail halfway through execution and leave state partially updated.How is the system evaluated?
Offline scores describe production only while the evaluation set still represents it.How are costs controlled?
Cost per task can become economically unreasonable long before anything visibly breaks.How is the system secured?
Prompt injection can arrive inside a retrieved document or a tool output as well as in the user's message.If you cannot, the generated code may be running, but you are not necessarily in control of the system.
The lifecycle is much larger than the demo
One idea I particularly appreciate in Thomas R. Caldwell's The AI Engineering Bible is its framing of AI development as a complete engineering lifecycle, in which building the model is one part of one stage.
The ordering matters, because the work does not end when the model responds correctly. Step through it.
Design
What are we actually trying to solve? What data do we need, what constraints exist, what architecture makes sense, and how will success be measured?
Data, constraints, architecture, and success metrics.
Build
How do we turn that design into a working system?
Data pipelines, training, prompting, orchestration, integration, and testing.
Deploy
How does this become a production service?
Infrastructure, containers, APIs, CI/CD, and a deployment strategy.
Scale
What happens when usage increases?
More users, more requests, more data, more throughput.
Optimize
How do we make the system economically and technically efficient?
Latency, cost, model compression, quantization, and resource utilization.
Maintain
How does this system remain reliable six months or two years from now? Many prototypes never reach this question.
Monitoring, drift detection, retraining, security, governance, versioning, and continuous improvement.
That lifecycle is a much better mental model for AI engineering than Prompt → Agent → Ship. The book's running Customer Support AI example makes this concrete by carrying one production system through the entire lifecycle, showing how decisions made early in the architecture affect deployment, scaling, and maintenance later.
The real complexity lives between the components
Individual components are often not the hardest part. You can learn an embedding model, RAG, tool calling, an agent framework, a vector database, and an LLM API one at a time. Production complexity emerges from the interactions between them.
Take a simple workflow: user, agent, retrieval, LLM, tool, database, response. Every arrow is an interface, and each one raises its own engineering questions. Switch between them below. The nodes stay the same, only the failing interface changes.
User → Agent: the request changes midway
The agent has already started work when the user edits, extends, or replaces the request. The system is now executing against an intent that no longer exists.
Work continues on the old request. Partial results from both versions mix in state and in the final response.
An explicit task state that a new request can supersede, and traces that show which version of the request each step served.
What happens when the user's request changes midway through execution?
Agent → Retrieval: retrieval fails
The retrieval call errors, times out, or returns nothing, and the agent has to decide what to do without the context it planned on.
The agent answers anyway, with no signal that it had no grounding.
Timeouts, bounded retries, and a fallback path that is explicit about answering without retrieved context.
What happens when retrieval fails?
Retrieval → LLM: the context is ignored
Retrieval succeeds and the right documents reach the prompt, but the model answers from its own parameters instead.
Answers sound confident and draw on nothing that was retrieved. The retrieval layer looks healthy the whole time.
Evaluation that checks answers against the retrieved documents, run before every release.
What happens when the model ignores the retrieved context?
LLM → Tool: the call goes wrong
The tool returns data the model did not expect, or the agent retries the same operation three times. If that operation has side effects and is not idempotent, it has now run three times.
Malformed data flows downstream as if it were valid. Side effects repeat.
Schema validation on tool outputs, idempotent operations for anything with side effects, and a retry budget.
What happens when the tool returns unexpected data, or the agent retries the same operation three times?
Tool → Database: shared state under pressure
The database is slow, or two tool calls modify the same state at once.
Timeouts cascade back up the chain. Concurrent writes leave state inconsistent.
Timeouts at every call site, concurrency control on shared state, and a way to recover from partial writes.
What happens when the database is slow, or two tool calls modify the same state?
The LLM itself: its behavior changes
Nothing in your code changed, but the model provider updated the model and its behavior shifted underneath every other component.
Prompts that were tuned for the old behavior degrade quietly.
Regression evaluation whenever the model changes, and observability that makes the shift visible.
What happens when the model provider changes its behavior?
Autonomy makes fundamentals more important
There is an interesting paradox here: the more autonomous AI systems become, the more traditional engineering matters.
What one wrong step can reach
A chatbot has one major output. An agent can call APIs, access databases, execute code, modify files, interact with external systems, trigger workflows, make decisions, delegate tasks, and maintain state.
Each of those capabilities is another path by which a single wrong step reaches a real system, and together they increase the blast radius of failure. That calls for stronger guardrails, permissions, observability, validation, sandboxing, error recovery, human-in-the-loop mechanisms, evaluation, and auditability.
The mapping from capability to control is fairly direct.
| Capability | Control it needs |
|---|---|
| Code execution | A sandbox |
| Writes to files and databases | Scoped permissions and an audit trail |
| API calls | Validated arguments and a recovery path |
| Irreversible actions | A human in the loop |
| All of the above | Guardrails, observability, and evaluation |
Research literacy keeps your knowledge current
The pace of research is another reason fundamentals matter. The field is moving too quickly to treat today's frameworks as permanent knowledge.
Framework APIs. Models. Inference methods. Context architectures. Evaluation methodologies. New research that changes what is possible.
ML fundamentals. Systems engineering. Research literacy. Software engineering.
If your understanding is tied exclusively to one framework, it becomes obsolete when that framework changes. If it is grounded in ML fundamentals, systems engineering, research literacy, and software engineering, you can adapt.
At that point the framework is a tool you use, and your foundation does not depend on it.
My own path through this shift
A quick note on where I am coming from: I'm Haroon, and I entered the AI field at the end of 2023, during the GPT era. I started from the fundamentals of machine learning and its core principles rather than from today's agent abstractions.
Since then, I have watched the industry move through a fast sequence of changes:
- LLMs became mainstream.
- RAG became mainstream.
- AI coding accelerated.
- Tool use became practical.
- Agentic workflows emerged.
- Multi-agent systems became a major research and engineering direction.
- Inference infrastructure matured.
- We are now moving toward increasingly autonomous AI systems.
The deeper I go into AI engineering and research, the more convinced I am that there is no point at which you finish the fundamentals and move on to being an AI engineer. They are the foundation you keep working from for your entire career.
You do not need to know everything
None of this means every AI engineer needs to be a mathematician, a distributed-systems researcher, a deep-learning scientist, a compiler engineer, a security researcher, and an infrastructure expert at the same time. That is unrealistic, and specialization is necessary.
Specialization and foundational ignorance are different things, though. A strong AI engineer can move across layers when necessary, which takes:
- Enough ML to reason about model behavior.
- Enough software engineering to build maintainable systems.
- Enough distributed systems to understand scale.
- Enough infrastructure to deploy reliably.
- Enough security to understand the attack surface.
- Enough evaluation to measure whether the system actually works.
- Enough research literacy to understand what is changing.
- Enough systems thinking to connect all of those pieces.
The future of AI engineering will reward systems thinkers
Knowing how to call the newest model will be a common skill. The next generation of AI engineers will be the people who can answer questions like these.
| Area | The questions |
|---|---|
| Problem framing | What should we build? Why should we build it this way? What assumptions are we making? |
| Evaluation | How do we evaluate it? How do we know it is improving? |
| Reliability | How does it fail? How do we recover from those failures? |
| Operations | What happens at 10x traffic? What happens when the model changes? What does each request cost? How do we secure the system? How do we maintain it for years? |
Build agents, but understand what you are building
I love the current agent ecosystem. I experiment and build with these systems, and I study new architectures as they appear. I believe agents will become an important interface for software.
I would still encourage anyone entering AI today to resist one temptation.
Learn the foundations. Understand ML, and learn software engineering deeply. Study systems, including distributed systems. Understand data and inference, and learn how models are evaluated and deployed. Learn security. Read research. Build things, then break them, measure them, debug them, deploy them, and operate them.
Then build agents.
Once you understand the layers underneath the abstraction, an agent framework stops looking like magic. It becomes what it actually is: another engineering tool for building intelligent systems. That shift in perspective is, in my opinion, one of the most important transitions an aspiring AI engineer can make.
The bar worth aiming for
Impressive AI demos are cheap to build now. The goal is to become someone who can build an AI system that survives reality.
That is a much higher bar, and it is the one worth aiming for.