9.7 Performance, reliability & ops at a glance

A backend that works in the demo still has to stay fast and stay up. You don’t need to be an SRE, but you do need to respect a handful of operational realities — caching, retries, rate limits, logging, and “it built but does it actually run in prod” — and to ask the AI to handle them. This chapter is that respect, at a glance. (Depth lives across 2.7, 2.8, 6.8, 6.9.)

Key vocabulary

Caching: Keeping a copy of an expensive result (a query, an API response, a rendered page) so the next request is served fast. The famous hard part is invalidation — knowing when the cached copy is stale and must be refreshed.
Retry with backoff: When a call fails (a timeout, a 503), try again after a growing delay (1s, 2s, 4s…) rather than hammering. Safe only when the operation is idempotent, or you risk repeating a side effect.
Rate limiting: Capping how many requests a single caller can make in a window. It protects public endpoints from abuse and runaway cost, and is signalled with a 429 Too Many Requests.
Observability: Being able to tell what your running system is doing — via logs (events), metrics (numbers over time), and traces (one request's path). Without it, a production problem is invisible.
Error handling: Catching failures, returning a useful status code and message, and logging the detail — without leaking internals (stack traces, secrets) to the client. Swallowed errors are silent bugs.

Caching: the easy win and the hard part

Caching is the highest-leverage performance move: store a result the first time and serve the copy fast afterward — at the CDN, in the server, or in a fast store like Redis. The famous catch is invalidation: knowing when the cached copy is stale. Cache too long and users see old data; cache too little and you’ve gained nothing. At the builder level you mostly decide what is safe to cache (rarely-changing, read-heavy data) and for how long — and you ask the AI to add it where a query is hot.

FIG 1 · a cache read path A hit serves from cache (fast); a miss falls through to the DB, fills the cache, and returns — so the next read is a hit.

Failure is normal: retries, backoff, rate limits

Networks fail and dependencies hiccup, so a robust backend expects failure. Set timeouts so a slow dependency can’t hang a request forever, and retry transient failures with backoff (growing delays) rather than hammering — but only when the operation is idempotent (the lesson from 9.2/9.6), or a retry repeats a side effect. Going the other way, protect your own public endpoints with rate limiting: a cap per caller that returns 429 when exceeded, defending against abuse and runaway cost. A public write endpoint with no rate limit is an open invitation.

Data	Cache it?	Note
Rarely-changing, read-heavy (config, catalog)	yes — longer TTL	biggest win, lowest risk
Per-user, frequently-read	sometimes — short TTL	mind staleness + invalidation
Changes every request / sensitive	no	caching adds bugs, not speed

Cache read-heavy, rarely-changing data first. When in doubt, don't cache — stale data is its own bug.

Observability and honest errors

When something breaks in production you can only fix what you can see, so the backend must emit logs (what happened), metrics (how often / how slow), and ideally traces (one request’s path). “If it’s not logged, it didn’t happen.” Equally, handle errors honestly: never swallow an exception (a silent failure is the worst kind), return a useful status code, and never leak internals — a raw stack trace or a database error sent to the client is both confusing and a security hole. And remember the gap that surprises everyone: works locally ≠ works deployed. The deploy can fail on a missing env var, an unreachable DB, or a build-only error — so when it breaks in prod, check config and logs first (6.4, 6.9).

01 Learning objectives

0 / 6 done

02 Curated reading

Cloudflare — What is caching?
essential 10m — What caching buys you and why invalidation is hard.
AWS — Exponential Backoff And Jitter
optional 12m — How to retry safely without a thundering herd.
Google SRE Book — Monitoring Distributed Systems
optional eng blog 18m — Logs, metrics, and the signals that matter.

03 Knowledge check

knowledge check3 questions · pass ≥ 70%

01easy
A public write endpoint with no rate limit is mainly a risk because…
02easy
Sending the raw stack trace / database error back to the client is good error handling.
03medium
The famously hard part of caching is…

04 Interview questions

browse all ↗

What gets asked on this topic — tap a card for how to approach it, the follow-ups, and the trap. Company tags are best-effort & sourced.

Commonly asked mid concept occasional Why do public endpoints need rate limiting, and why must retries use backoff?
Rate limiting caps how many requests a single caller can make in a window, returning 429 when exceeded. Public endpoints need it to defend against abuse, scraping, brute-force, and runaway cost — without it, one client (or attacker) can overwhelm the service or run up your bill. Retries handle transient failures, but naive immediate retries create a *thundering herd* — everyone retrying at once hammers an already-struggling dependency. Backoff (growing delays, ideally with jitter) spreads them out so the system can recover; and retries are only safe on idempotent operations, or you repeat side effects.
What a strong answer covers
- Rate limiting (429) protects against abuse and runaway cost.
- Naive retries cause a thundering herd on a struggling dependency.
- Backoff (growing delays + jitter) lets the system recover.
- Only retry idempotent operations, or you repeat the side effect.
Follow-ups they push on
- What status code signals rate limiting? (429)
- Why is idempotency a prerequisite for safe retries?
Red flag Shipping a public write endpoint with no rate limit, or retrying a non-idempotent POST in a tight loop.
source: AWS — Exponential Backoff And Jitter ↗