Context Engineering Basics

What context engineering is, how it differs from prompt engineering, and the principles and memory patterns that help context scale.

​

What Is Context Engineering?

Context engineering is the practice of deciding exactly what information a large language model (LLM), or a group of LLM agents, sees during a task. It covers which data the model sees, how that data is organized, and how it is framed.

Context has four main parts:

• Information (I): Facts, documents, or intermediate results that go into the model.
• State (S): What the model must know about the current session, such as the conversation so far or the structure of a task.
• Tools (T): External systems that the model can access, such as APIs or data sources.
• Format (F): How everything is wrapped, such as prompt templates, instructions, or response formats.

If you treat these parts like code (you version, test, measure, and improve them), LLM outputs become more predictable and reliable across use cases.

​

Scaling Agents Means Scaling Context

Large models perform well on single-shot tasks. But real-world systems often use agents that delegate, call APIs, and persist over time. These long-running systems have common failure modes:

• Context drift: Agents develop conflicting views of the truth.
• Bandwidth overload: Full histories strain context limits and make responses slower.
• Tool blindness: Agents get raw data but no guidance on how to use it.

In practice, stale or inconsistent context is the leading cause of coordination failures. Even basic memory update strategies can change agent behavior significantly over time. For this reason, memory management must be deliberate.

​

Prompt Engineering ≠ Context Engineering

Prompt engineering and context engineering are related, but they are different disciplines. Both shape how language models behave, but they operate at different levels of abstraction.

Prompt engineering is about the how. It is the work of finding the right wording, tone, and examples to guide the model in a single interaction. It is like writing the best possible “function call” to the model.

Context engineering is about the what, when, and how of the information that the model observes. It spans full workflows, manages memory across turns, and gives the model access to relevant tools and schemas. If prompt engineering is a clean function call, context engineering is the design of the full service contract, including interfaces, dependencies, and state management.

Dimension	Prompt Engineering	Context Engineering
Optimises	Wording, tone, in‑context examples	Selection, compression, memory, tool schemas
Timescale	Single request/response	Full session or workflow
Metrics	BLEU, factuality, helpfulness (per turn)	Task success vs. token cost, long-horizon consistency

Context engineering becomes essential when systems move from isolated prompts to persistent agents and long-running applications. It makes coordination, memory, and interaction across tasks scale. With it, a language model stops being only a tool and becomes part of a system.

​

Six Principles for Optimizing Context

When systems grow from one-off prompts into long-running workflows, context becomes a key engineering surface. These principles describe how to design and manage context for LLMs and agent-based systems.

Principle	Why It Matters	Example Technique
Salience over size	More tokens do not mean more value. Signal matters more than volume.	Salience scoring + reservoir sampling to retain only statistically “interesting” chunks
Structure first	Models and tools handle structured inputs more reliably than unstructured text.	Use canonical world-state objects; track changes with diff logs
Hierarchies beat flat buffers	Effective recall happens at multiple levels of detail, not in a flat sequence.	Multi-resolution memory via Hierarchical Memory Transformers
Lazy Recall	Do not pay the context cost until the information is necessary.	Use pointer IDs and on-demand retrieval (RAG)
Deterministic provenance	You cannot debug what you cannot trace. Source tracking is critical.	Apply “git-for-thoughts” commit hashes to memory updates
Context–tool co-design	Shape information for use, not only for storage. Tools need actionable input.	Embed tool signatures alongside payloads so the model knows how to act

Together, these principles move context design toward systems that are leaner, easier to interpret, and better aligned with model behavior and downstream actions.

​

Applying Context at System Scale

Systems that use long or complex context need well-designed memory. The patterns below are different ways to manage context. The best choice depends on how much information your system handles and how long it must remember it.

Three-tier memory divides context into three layers: short-term (exact text), mid-term (summaries), and long-term (titles or embeddings). The system keeps recent details and still remembers important older information. This pattern is a good fit for chats or agents that run over many turns. Hierarchical Memory Transformers (HMT) use this design.

Recurrent compression buffers compress earlier parts of a stream, such as a transcript or log, into smaller representations. The system can bring these back later if necessary. This saves space, and the system can still recall details when they are relevant.

State-space backbones store memory outside the prompt in a hidden state that carries over between turns. With this pattern, the model handles much longer sequences efficiently. It is especially useful on devices with tight memory or speed limits, such as mobile or edge systems. Mamba is one example of this pattern.

Context cache and KV-sharing spread memory across different servers by saving reusable attention patterns. This prevents repeated work and keeps prompts small. It is a strong choice for systems that run many requests in parallel. MemServe uses this technique.

Hybrid retrieval has two steps. First, it filters data with keywords or metadata. Then, it uses vector search for meaning. This gives fewer irrelevant results, especially in datasets with a lot of similar content.

Graph-of-thought memory puts ideas into a graph. Entities are nodes and their relationships are edges. The model gets only the relevant part of the graph, not the full graph. This works well for complex tasks such as analysis or knowledge reasoning. Teams often build it with tools such as Neo4j or TigerGraph.

Each pattern scales memory and context in a different way, so the right pattern depends on the problem. All of them help systems stay efficient, accurate, and responsive as context grows.

​

Optimizing Context Like Code

To optimize context like code is to apply the same engineering cycle to context that teams apply to software: observe it, measure it, change it, and test for regressions. The cycle has these parts:

• Logging: The system logs every prompt and context segment, so you know exactly what the model sees at each step.
• Span labels: Each span gets a label that shows if it was used, ignored, hallucinated, or contributed to the final output.
• Return on input (ROI): For each span, ROI compares cost to value: ROI = token cost ÷ impact on accuracy.
• Trimming: Spans with low ROI are removed. References (pointers) to them stay, in case retrieval is necessary later.
• Salience models: A salience model learns from past usefulness and predicts which spans to include in context automatically.
• Adversarial context tests: Shuffled inputs or missing key details show how robust the model is and how much it depends on context structure.
• Regression evaluations: Repeated tests across agent roles and tasks catch drift or failures that come from context.
• Versioned context bundles: Context bundles get snapshots, diffs, and review before release, in the same way as code.

​

From Prompts to Protocols - Takeaways

Multi-agent systems are powerful because they divide knowledge and responsibility across roles. But the same structure becomes fragile when context is outdated, overloaded, or misaligned.

Context engineering changes prompting from trial-and-error into system design. It makes sure that each agent sees the right information, in the right form, at the right time.

Reliable systems treat context as a core artifact, not only an input. They observe it, version it, and optimize it. On that foundation, agents stop behaving like chat interfaces and start to act like collaborators.