A Shorter Prompt Won’t Fix Your Agent: 10 Problems I Hit

I started my recent Strat Thread Agent work with two visible problems. The system prompt—the standing instructions the Agent receives—was too large. The tool catalogue was larger still, and loading tools step by step kept breaking prompt-cache reuse.

The first fixes seemed obvious: shorten the copy, load capabilities only when needed, and add a cache key. It looked like tidying a school bag: remove a few heavy books and call the job done.

That explanation did not survive contact with the traces, which are travel logs for requests. A production Agent also has to choose which abilities to load, call tools with the right shape, recover work after failure, show text as it arrives, and leave enough records to explain a problem later. A weak rule in any of those places can eventually look like “the model was not smart enough.”

This article records ten concrete problems I found. The central lesson is:

Agent optimization is not adding one more sticky note to the prompt after every failure. Context, capabilities, tools, state, and evidence need one stable set of rules.

Here, context means the information the Agent can see on this turn. A tool is an interface it can use to read data, calculate something, or save a result. Think “material on the desk” and “tools in the drawer.”

Three numbers that changed the diagnosis

In one early audit, the Strat Thread market page sent roughly 35.5 KB before user messages or conversation history were added:

Component	Size
System prompt	about 5.5 KB
20 always-on tool definitions	about 12.8 KB
Page Markdown, directory, actions, and strategy contracts	about 17.2 KB

In a separate seven-day snapshot, 108 of 807 tool calls failed—about 13.4%. The largest categories were not business rejections. They were mixed parallel calls, invalid arguments, and mismatched schemas.

Strategy creation was worse. In one 30-day database sample, create_strategy was called 30 times and succeeded on the first call only five times, or roughly 16.7%. The model had to submit an entire TypeScript strategy, parameter definitions, parameter values, and a version in one transaction. Repairing one type error meant regenerating the whole payload.

Those numbers represent three different kinds of waste:

context waste: the model rereads material irrelevant to the current task
structural waste: the model generates values a schema or server could determine
execution waste: failures, retries, and repair turns multiply physical requests

Prompt compression addresses only part of the first.

1. Do not put every lesson into the system prompt

The original prompt carried identity, language, safety, planning, page knowledge, strategy rules, compaction policy, and tool instructions. Every production failure suggested one more sentence for the common prompt. It was like covering a door with sticky notes: eventually the door is still there, but nobody can find the handle.

I now split context into five layers:

Foundation
  identity, authority, success evidence, and task loop

Capability index
  what executable capabilities the Agent can load

Skill references
  what domain knowledge exists and when to read it

Dynamic context
  current page, object, time, and user input

Conversation events
  messages, tool calls, results, approvals, and plan state

The foundation must be short, deterministic, and valid across tasks. Domain knowledge should not live there merely because it might become useful. Chinese and English should not both be sent; each locale needs its own natural, compact prompt.

This is not only about token cost. As policy grows, the model has more difficulty identifying which constraints matter for the current objective. OpenAI’s current model guidance similarly recommends stating each instruction once, exposing only relevant tools, and keeping tool descriptions concise—while validating each reduction on representative tasks rather than assuming shorter is always better.

2. Tool descriptions are prompt content too

Prompt audits often count system and developer prose while ignoring tool names, descriptions, and JSON Schema. The model still reads all of them. A schema is the rule sheet for a form: which boxes exist, what each box may contain, and which ones cannot be empty.

In one Strat Thread cleanup, descriptions for 31 static tools fell from 8,780 B to 3,688 B, a 58% reduction. Descriptions for 100 business actions fell from 20,627 B to 8,228 B, or about 60%. The removed text was mostly duplication: repeating the tool name, restating schema fields, and explaining that a feature existed.

A tool description should help the model decide:

when to use it;
which inputs cannot be safely inferred;
whether it has side effects or requires approval;
whether it is safe to retry; and
what success and failure mean.

Large catalogues should not be sent in full on the first turn. The foundation can retain capability loading, skill discovery, and essential control tools; business tools can be disclosed through a capability directory or tool search. The current OpenAI prompt-caching guide recommends deferred tool loading to reduce early input tokens and appending discovered tools at the end of context.

There is an important counterweight: deferred disclosure must not make capabilities invisible. The first turn still needs a compact index of what can be discovered. A model will not search for a capability it has no reason to believe exists.

3. When the cache misses, check how context is arranged

Progressive loading is often implemented like this:

turn 1: regenerate foundation + tool A
turn 2: regenerate foundation + tool A + tool B
turn 3: reorder foundation + tool B + tool A + tool C

The product sees increasing capability. The cache sees a rewritten prefix. Insert tool B before tool A and the conversation, calls, and results after that point no longer occupy the same token sequence.

The safer structure resembles an append-only log:

stable foundation [cache breakpoint]
  + capability A
  + user message / tool call / result
  + capability B
  + next message / result

OpenAI’s current guidance for multi-turn Agents says to preserve stable content at the front and append new messages and tool history. Tools can be introduced with additional_tools; GPT‑5.6 supports explicit cache breakpoints after stable content. Measurement must include both cached_tokens and cache_write_tokens, along with ordinary input, output, latency, and realized cost.

My cache contract now requires:

deterministic foundation text and tool order;
timestamps, page state, and volatile identifiers after the stable prefix;
retained capability history rather than reordering or clearing it on every message;
a cache key that identifies the prompt version, language, mode, and stable shard—not a random request;
deterministic prefix and request-envelope review for every Agent change; and
paid baseline/candidate sampling only when cache performance is an explicit acceptance target.

The correct token-weighted rate is:

sum(cached_input_tokens) / sum(input_tokens)

Do not average request percentages. I covered the mechanics, OpenLIT traces, and cache cost model in the previous prompt-cache article.

4. Capability, skill, and tool need one relationship

The architecture once had two disclosure systems. Some capabilities came directly from load_agent_capabilities; some strategy tools appeared only after two skills were loaded. Workflow was optional, even though the model needed planning ability to decide whether Workflow should be loaded.

In one strategy run, the model activated strategy_creation but missed skills. The Context 2.0 resource existed and loaded successfully later, yet the Agent concluded that the runtime specification had not been returned. This was not missing model knowledge. The orchestrator had allowed an incomplete state.

The three concepts need one definition:

Concept	Responsibility
Capability	A discloseable group of executable abilities and behavioural boundaries
Skill	Versioned domain knowledge needed to perform a class of work correctly
Tool	The interface that actually reads, computes, or writes something

Workflow, user questions, plan state, and skill discovery belong to the foundation control plane. Strategy, market data, backtesting, and system actions can remain progressively disclosed. Compact skill references must make important knowledge discoverable from the first turn; full skill bodies load only when needed.

A single manifest should generate the capability index, prompt fragment, tool group, dependencies, lifecycle, and fingerprint matrix. If one capability still requires manual edits across six or seven registries, prompt/runtime drift will return.

5. A global Agent must not degrade into a page macro

Early implementations bound strategy editing and market-chart tools to specific routes. Outside a strategy page, editing disappeared. Outside the market page, chart operations disappeared. The model learned the wrong causal rule: navigation unlocks capability.

Users expect a global Agent to operate the system, not merely automate the visible page.

The boundary I settled on is:

page state provides optional object and presentation context; it does not register, authorize, or unlock tools;
tool inputs carry explicit strategy IDs, versions, or symbols instead of relying on the current route;
background-capable work executes first, then the Agent asks whether the user wants to open the corresponding page;
UI-bound operations return “not executed” plus a recovery route, and navigate only after confirmation;
if a page changes while a read is in flight, the stale result becomes a recoverable tool error instead of failing the whole conversation with a 409.

Once tools and routes are separated, navigation becomes a presentation choice rather than an authorization mechanism.

6. Fix the form rules before adding another instruction

When tool arguments fail, the usual response is another sentence telling the model to follow the format carefully. That cannot repair a provider schema that disagrees with runtime validation.

In one implementation, operate_market_chart used a strict discriminated union at runtime while the model-facing drawings field accepted arbitrary objects. ask_user did not express conditional field relationships. A proxy could also remove strict or force parallel_tool_calls back to true.

The correct sequence is:

generate runtime validation and provider JSON Schema from one type source;
enable strict: true for compatible schemas;
set additionalProperties: false on every object;
model optional values explicitly rather than leaving open objects;
set parallel_tool_calls: false when a turn must produce at most one call;
after argument failure, expose only the failed tool and allow one focused repair; and
maintain a capability matrix for models and proxies instead of assuming “OpenAI-compatible” means behaviourally identical.

The OpenAI function-calling guide recommends strict mode for reliable schema adherence and documents how disabling parallel tool calls limits a turn to zero or one function call.

Correct shape still does not mean correct business behavior. Strict mode can guarantee that positionPercent is a number. It cannot guarantee that generated strategy code uses the right runtime API. A form can contain a valid phone number and still name the wrong person. These failures need separate handling.

7. Do not make the final write carry the whole world

The low first-call success rate for strategy creation exposed a tool-granularity problem. The old create_strategy required:

a display name;
an initialVersion that was almost always v1;
complete TypeScript source;
a parameter schema that already contained defaults; and
another full parameters object repeating those defaults.

That is too large a transaction. Repairing one type error requires regenerating all source and arguments, creating new opportunities for drift.

A more reliable design is:

prepare_strategy_draft
  input: name, source, essential parameter definitions
  server: fill defaults, validate, fingerprint
  output: TTL-bound draftId scoped to user and session

create_strategy
  input: draftId
  server: verify approval, fingerprint, and idempotency, then persist

The pattern generalizes to high-risk writes: prepare and validate the large object, persist it behind a short handle, and let the final commit reference that handle. Approval binds to actual content and its fingerprint; the model does not copy the payload again after approval.

8. A conversation grows, and its data gets old

The first long-session problem is size. Strat Thread added /new, /clear, and /compact, plus automatic compaction near a context threshold. Compaction is not merely deleting messages: the successor run must take over the stream, late frames must not overwrite new state, and failure must leave the original context recoverable.

The second problem is less visible. Historical content can be out of date. Market candles, prices, financial metrics, and event streams can produce confident but wrong analysis when an Agent reuses old tool results.

Time-sensitive tasks therefore need a freshness contract:

read current time
  -> inspect historical as-of metadata
  -> fetch the latest data again
  -> compare timestamps and close state
  -> update the session snapshot
  -> compute and analyse

If the fresh read fails, the Agent should report the last verified as-of boundary. It must not convert dependency failure into a claim that data is current. Memory preserves the past; freshness validation decides whether the past still represents now.

9. One user click can become ten model requests

A chat completion may take one call. An Agent can plan, select a tool, analyse its result, call again, and summarize. One user request can expand into five to fifteen physical model requests. Several Agents running through Promise.all create a sharp burst against OpenRouter or an upstream provider.

Reliability cannot be reduced to “retry five times.” A retry policy must understand safety:

retry automatically only when no text, tool call, terminal event, or side effect has been produced;
respect Retry-After for 429 and selected 5xx responses, otherwise use exponential backoff with jitter;
do not blindly retry 400, 401, 402, 403, schema errors, or context overflow;
place global concurrency control in the shared LLM gateway, not inside each Agent instance;
observe connection, time-to-first-token, and total-response timeouts separately; and
treat provider fallback, model fallback, and application retry as three distinct layers.

Release compatibility matters too. An application default such as luna can fail when the production gateway exposes only gpt-5.6-luna; a skill may document a new strategy API while the worker image still runs an older runtime. Model catalogue, tool capabilities, runtime API, and image version should be validated before users discover the mismatch.

10. Show the stream, and leave enough clues to debug it

When users see a long pause followed by a full answer appearing at once, the provider stream may still be working. One historical Strat Thread path merged deltas into cumulative worker snapshots, polled Redis from the web client every 750 ms, showed no text while tool arguments were generated, and replaced the live state with the full terminal message immediately.

That creates pseudo-streaming: the transport has deltas, while the product shows heartbeats and one final block. Parallel tools may also complete at different times but appear together after the slowest call crosses a barrier.

A clearer protocol distinguishes:

output_text.delta
tool_call.in_progress
tool_call.done / failed
output_text.done
response.completed

Events need increasing sequence numbers. completed means the business response is final; it should not authorize the client to skip buffered text deltas. The OpenAI Responses API similarly separates text delta, text completion, and response completion events.

Without observability, every one of these failures becomes “the model occasionally hangs.” Observability simply means leaving enough safe records to rebuild what happened. I now expect each conversation to expose:

a root Agent run;
every physical LLM request;
a child span for every tool execution;
session ID, requested and actual model/provider, latency, and tokens;
tool name, call ID, status, and safe error code;
redacted and length-limited arguments and results;
retry counts, 429/5xx/timeout rates, and request amplification; and
cache reads, writes, ordinary input, and output.

Message and tool content should be redacted, bounded, and retained only as long as needed. The goal is not permanent surveillance. It is the ability to reconstruct one failure along the path from user request to model, tool, and subsequent model call.

The review checklist I use now

I no longer ask only whether the prompt is clear. I review an Agent change in this order:

Layer	Required question
Context	Is the new material stable foundation, on-demand knowledge, or a dynamic suffix?
Cache	Did it rewrite an old prefix, tool order, cache key, or request envelope?
Capability	Does the model know the capability exists, and is activation lifecycle unique?
Tool	Are runtime and provider schemas derived from one minimal, strict source?
State	Can page changes, compaction, continuation, or retries lose or duplicate effects?
Freshness	Is the answer based on current facts or historical memory?
Provider	Are model, proxy, timeout, concurrency, and fallback behaviours compatible?
UX	Is tool progress visible, and can terminal state skip pending deltas?
Evidence	Can one session reveal physical requests, tools, errors, and token cost?

Routine Agent changes do not need a paid cache benchmark. Deterministic prefix and request-envelope fingerprints, schema audits, type checks, and regression tests cover most structural risks. When cache hit rate is explicitly an acceptance target, use identical baseline and candidate workloads, model and provider routes, and a controlled request count.

Turning “model problems” back into system problems

All ten problems point to the same conclusion: a production Agent cannot depend on the model to repair architectural gaps on every turn.

Prompts need layers. Capabilities need discovery. Tools need one schema. State must survive continuation safely. Writes must be idempotent. Time-sensitive data needs an as-of boundary. Streaming needs event semantics. Failures need a trace.

The model should make judgments inside those boundaries. The system should prevent it from entering states where reliable judgment is impossible.

I now evaluate Agent optimization with more than answer quality: first-turn input, cache reads and writes, tool success rate, physical requests per task, retries, data freshness, end-to-end latency, and whether a failure is diagnosable.

Once those metrics are visible together, the prompt returns to its proper role. It is one interface in an Agent system—not a patch for defects in the other nine layers.