Part 2 · Core Building Blocks Chapter 4

The system's front door.

Three jobs share one entrance: spread, shape, and govern.

Learning objective

Understand the three main jobs at the system edge so you can explain, in plain English, how traffic gets routed, protected, and shaped before it reaches your services — and pick the right edge component for the job.

Before you read

Make a prediction first.

Predict

Answer before the explanation.

At the front door of a system, which job is about spreading traffic, and which job is about enforcing policy?

Commit

Write a rough answer.

Before reading, decide whether a load balancer, reverse proxy, and API gateway are the same thing or different jobs.

Connect

Notice where it returns.

This boundary shows up later in upload systems, APIs, chat gateways, and rate limiting.

Concrete first

Picture an interview moment.

A candidate draws a system. The interviewer points at the line between the user and the app servers and asks: "What's that box?" The candidate says "load balancer." The interviewer asks "why?" — and the candidate stalls.

That stall is what this chapter exists to prevent. The vocabulary at the edge — load balancer, reverse proxy, API gateway — is a trap if you treat it as three unrelated boxes. They're three jobs, often performed by overlapping tools, and they can be explained by what each one decides.

Mental model

One front door. Three jobs.

Spread traffic. Shape requests. Govern APIs.

The vocabulary maps to the lanes: load balancer = spread, reverse proxy = shape, gateway = govern. One product can wear more than one hat.

First principles

Why three jobs and not one?

The three jobs exist because they answer three different questions, and each gets harder as the system grows.

Spread

Stateless work, parallelized.

If a request can run on any healthy server, distributing them gets you both throughput and failover for free. Health checks are non-negotiable — a failed server should disappear from rotation in seconds.

Shape

Decisions made before the app sees the request.

TLS termination, path routing, header rewriting, response compression, edge caching. Each is something you'd rather do once at the edge than re-implement in every service.

Govern

Policy, centralized.

Auth, rate limits, quotas, tenant rules, API versioning. These are the things you'd hate to ship in 12 different services with subtle differences.

Why it matters in interviews

"Add a load balancer" is not an answer.

Weak

We'll put a load balancer in front. Maybe a gateway too.

Strong

I want a load balancer here so traffic is spread across stateless app servers and unhealthy instances drop out automatically. I'd add a reverse proxy if path routing or TLS termination is worth doing once at the edge — and a gateway only if the API needs centralized auth, quotas, or versioning.

The interviewer isn't grading your vocabulary. They're listening for whether you can name the job before the component.

Key ideas

Five anchors to carry into the room.

Load balancers, reverse proxies, and gateways all live near the edge of the system.
They're defined more by function than by product names.
A load balancer answers: which healthy server should get this request?
A reverse proxy answers: how should this request be received and routed?
A gateway answers: what API rules should be enforced before this gets through?

Path routing in practice

One edge. Three different paths. Three different services.

The "shape" job earns its keep when the URL itself tells the edge where the request belongs. This is the most common way candidates encounter reverse proxies in real systems.

The edge is a router with opinions. Path-aware routing means the internal service layout can stay invisible to the client.

Speaking script

Lines for the front-door conversation.

Opening

I think of this layer as the system's front door. Its job is to spread traffic, shape requests, and sometimes govern API access.

Sketching

At minimum, I want traffic distributed across multiple healthy application instances so one server isn't a bottleneck.

Routing

If different paths need different backend services, a reverse proxy can direct traffic based on host or path.

Restraint

I'd add an API gateway only if centralized concerns like auth, quotas, or versioning justify it.

Defending

I don't want to add edge complexity unless the entry path actually needs it.

Common mistakes

How candidates trip at the front door.

Saying "add a load balancer" as a reflex without explaining the need.
Treating load balancer, reverse proxy, and gateway as completely unrelated boxes.
Adding an API gateway to every design even when the system is small.
Ignoring health checks and failover at the edge.
Forgetting that the edge layer can become a single point of failure if not replicated.
Using sticky sessions casually without explaining the trade-off.

Misconception check

Correct the wrong model before it sticks.

Wrong intuition

What feels tempting

Load balancer, reverse proxy, and gateway are interchangeable labels.

Better model

What to replace it with

They can sit near the same place, but they answer different questions: where should traffic go, how should it be shaped, and what rules must be enforced?

Interview move

What to do in the room

Name the job first, then choose the front-door component that performs that job.

Trade-offs

What each edge choice costs you.

Edge choice	Good when	Weak when	Interview line
Direct to one app server	System is tiny and early-stage.	Creates a clear single-server bottleneck and weak availability.	Acceptable for the smallest version, but it doesn't scale or fail over well.
Load balancer Default	You have multiple app instances and need traffic spread plus health-based routing.	It doesn't by itself solve API policy or complex request routing.	I want a load balancer here so requests are spread across healthy stateless servers.
Reverse proxy	You need path-based routing, TLS termination, caching, compression, or shielding internal services.	Adds edge complexity if the system doesn't need those controls yet.	A reverse proxy helps me control how requests enter and which internal service they reach.
API gateway	You need centralized auth, quotas, tenant policy, versioning, or API-specific controls.	Overkill for a simple internal system or a single basic endpoint.	I'd use a gateway when API policy is important enough to centralize instead of reimplementing in each service.

Mini case study

The same prompt you've seen — now zoomed to the edge.

Prompt: "Design a URL shortener." (We'll keep coming back to this. Spaced repetition is doing the work.)

Why a load balancer fits

Redirect traffic is read-heavy and bursty.
Stateless redirect servers scale horizontally trivially.
Health-based routing protects against silent server failure.

Where a reverse proxy helps

Terminate TLS once at the edge.
Route admin APIs differently from redirects.
Hide internal server layout from the public.

When a gateway earns its place

Authenticated link-management APIs.
Per-tenant quotas or rate limits.
Versioning across multiple services.

The pattern, made portable

Public read path may only need spreading.
Authenticated write path may justify governing.
Different paths can use different edge behavior.

Worked example to solo answer

Fade the support before the real practice.

Do not jump straight from reading to a full answer. First see the shape, then complete part of it, then answer alone.

I do

Study the model move.

I would say: "For profile photo uploads, the gateway can authenticate and limit requests while the load balancer spreads accepted traffic."

We do

Complete the missing piece.

Separate the upload path from the view path and decide which front-door job each path needs.

You do

Answer without notes.

For the practice prompt, draw only the client, front door, app service, and storage. Label the front-door job.

Practice

Try it before you read the model answer.

Prompt

"Design an API for uploading and viewing profile photos."

Would you use a load balancer, a reverse proxy, an API gateway — or some combination?
State the job each one is doing.

Show a strong model answer

I'd start with a load balancer so requests are spread across multiple application instances and unhealthy servers can be removed automatically. I'd likely also use a reverse proxy because upload and image-view paths may route differently, and TLS termination is a good fit at the edge. I'd only add a fuller API gateway if the public API needs centralized auth, quotas, or versioning across multiple services. The main point is to keep the front door doing only the jobs the system actually needs.

Training loop

Make this chapter stick.

Before moving on, turn recognition into production. Close the model answer, answer from memory, then retry one small slice.

Recall

Say the chapter's core idea without looking. Then name one related idea from an earlier chapter.

Vary

Change one constraint in the practice prompt and answer again in half the time.

Score

Use the rubric to pick one dimension below 3, then retry only that dimension.

Recap

Three things to take into the room.

Name the job, then name the component.

"Spread / Shape / Govern" is the brain's hook before the vocabulary.

One product can wear several hats.

Don't argue brands. Argue responsibilities.

The edge isn't free.

Each capability adds a hop, a failure mode, and an operations bill.

Reusable interview line

"The front door is doing one of three jobs here — spread, shape, or govern — and I'd add the lightest component that does the job we actually need."