When an API Gets Slow, Do Not Add Machines Yet

When an endpoint goes from 80 ms to 800 ms, the instinct is often: “We need more machines.” Sometimes that is true, but it is usually too early. The first step in performance work is to split the time. Is the browser waiting, is the application computing, is the database scanning, or is an external service stalling?

Once those parts are separated, a slow request is much less mysterious. This is a lightweight investigation path for small projects. It favors evidence over an elaborate toolchain.

First, is the slowness typical or a long tail?

“The API is slow” can mean two different things:

• It is slow every time; for example, a list page consistently takes 700 ms.
• It is slow occasionally; most requests take 100 ms, but a few jump above three seconds.

The first points toward fixed work in code, queries, or the network path. The second is more likely to involve lock waits, exhausted connection pools, cache misses, or an unstable downstream dependency.

Do not only watch an average. At minimum, track these three numbers:

Metric	The question it answers
P50	What is the typical user experience?
P95	Is the slowest small slice already painful?
P99	Are there spikes that may signal an incident?

Imagine an orders endpoint with a P50 of 90 ms and a P95 of 1.8 seconds. Improving its average from 150 ms to 120 ms will barely change what users notice. The useful question is why the slowest five percent are so different.

Give every request a time bill

Instead of scattering ad hoc logs throughout the code, assign a request ID and measure a few important stages:

const startedAt = performance.now();
const orders = await orderService.list(userId);
const afterQuery = performance.now();
const payload = serializeOrders(orders);

logger.info({
  requestId,
  dbMs: Math.round(afterQuery - startedAt),
  serializeMs: Math.round(performance.now() - afterQuery),
  totalMs: Math.round(performance.now() - startedAt),
  resultCount: orders.length,
}, "list orders completed");

Three or four stages are enough to start: authentication, database work, external calls, and response assembly. Do not log phone numbers, tokens, or complete request bodies. Debugging needs context, not a copy of private data.

That time bill quickly separates common cases:

• The database takes 20 ms but the request takes 900 ms: look at external APIs or unnecessary serial work.
• The database consistently takes 600 ms: inspect the SQL, indexes, returned rows, and connection waiting.
• Only large lists are slow: check serialization, N+1 queries, and endpoints that lack pagination.

Before tuning the database, ask what you are fetching

Many slow queries do not need a faster database; they need a more restrained request.

An admin list that fetches every historical order and then queries the user once per row will hurt as data grows. A sensible repair order is:

1. Add pagination with a reasonable default page size.
2. Select only fields the page displays; avoid habitual SELECT *.
3. Replace per-row lookups with one join or a batched query.
4. Add indexes that match real filtering and sorting conditions.

Indexes are not simply “the more, the better.” If a query filters by tenant_id and then sorts by created_at to fetch the most recent 20 records, its index should usually reflect those conditions. After adding one, inspect the query plan: was it used, how many rows were scanned, and did the database need a temporary sort?

Also check data distribution. A test tenant with dozens of rows does not reveal what happens when one production tenant has millions.

Two external-call traps

The first is serial waiting. This is readable, but its duration is the sum of all three calls:

const profile = await getProfile(userId);
const coupons = await getCoupons(userId);
const notices = await getNotices(userId);

If they are independent, run them together:

const [profile, coupons, notices] = await Promise.all([
  getProfile(userId),
  getCoupons(userId),
  getNotices(userId),
]);

The second is a missing timeout. A stuck downstream service can hold connections and workers until normal traffic slows down too. Every external call needs an explicit timeout, bounded retries, and a fallback result. “Bounded” matters: retries amplify pressure during peak traffic and can turn a small failure into a cascade.

Do not finish with “it feels faster”

Leave three things behind after an optimization: P50/P95 before and after, a load-test or production-traffic sample, and a rollback path. For example:

Added a (tenant_id, created_at) index to the orders list and replaced individual user lookups with a batch query. Across 100 sample requests, P95 fell from 1.6 s to 210 ms. If write pressure becomes abnormal, roll back the index migration first, then restore the previous query path.

That small note tells the next person why the change exists, where the gain came from, and what risk remains.

Performance work is never truly “finished.” A more realistic goal is to notice anomalies early and tie each change to a clear bottleneck. Scaling machines is still an option—once you know what is actually consuming the time.