Latency for Websites and APIs: What Matters and Why

Quick answer: for a website or an API, latency is the time lost to network round trips between the user and your server. Opening a fresh HTTPS connection and making one request needs several round trips in a row, so the network cost is roughly three to four times the round-trip time (RTT) before the first byte of the answer arrives, on top of whatever time the server needs to do its work. The levers that matter most are putting servers or caches close to users, reusing connections, cutting the number of sequential requests, and keeping server processing time low.

Why does latency matter more than bandwidth for most websites?

Latency is the delay for data to travel across a network, measured in milliseconds (ms). Bandwidth is how much data the connection can carry per second. If you want the full distinction, read latency vs bandwidth vs throughput. For web pages the practical point is this: a page is not one large download but dozens or hundreds of small requests, and many of them cannot start until an earlier one finishes. The HTML must arrive before the browser knows which stylesheets, scripts and images to request. A script may then request more files. Each of those steps waits at least one round trip.

When the files are small, the time to move the bytes is tiny compared with the time spent waiting for the first reply. A 50 KB stylesheet on a 100 Mbps connection transfers in about 4 ms, but the request that fetches it may wait 40 ms or more for the reply to start. That is why users on a fast connection to a distant server can still see a sluggish site, and why users with a modest plan close to the server see a quick one.

What happens before the first byte arrives?

A browser opening a page over HTTPS for the first time performs these steps in order. Each one needs a reply before the next can begin.

1. DNS lookup: turning the hostname into an IP address. A cached answer costs almost nothing; an uncached one costs at least one round trip to a resolver, and sometimes more if the resolver must ask other servers. See DNS explained.
2. TCP handshake: one round trip to open the connection (SYN, SYN-ACK, ACK). See TCP and UDP explained.
3. TLS handshake: one extra round trip with TLS 1.3, two with TLS 1.2, to agree on keys and verify the certificate. See the TLS and HTTPS handshake.
4. HTTP request and response: one round trip, plus the time the server needs to produce the answer, until the first byte comes back.

With TLS 1.3 over TCP that is three round trips (TCP, TLS, HTTP) after DNS. With TLS 1.2 it is four. The browser's measurement of "how long until the first byte" is called time to first byte; TTFB and web performance covers it in detail.

RTT to server	Network share of TTFB (TLS 1.3, new connection, DNS cached)	Network share with TLS 1.2
10 ms	About 30 ms	About 40 ms
50 ms	About 150 ms	About 200 ms
100 ms	About 300 ms	About 400 ms
250 ms	About 750 ms	About 1,000 ms

Add the server's own processing time to each of these. The table shows why a distant user with a 250 ms RTT waits almost a second for the first byte even when the server answers instantly.

How does connection reuse change the picture?

Once a connection is open, later requests to the same host can reuse it, skipping the TCP and TLS handshakes. A request on a warm connection costs one RTT plus server time. Browsers keep connections alive, and HTTP/2 and HTTP/3 let many requests share one connection at the same time. This is why the first request to a host is the slowest and later ones are faster.

Connection reuse matters for APIs too. A client that opens a new connection for every call pays the handshake cost every time. A client that keeps a connection pool pays it once. For a server-to-server integration with 80 ms RTT, a new connection per call adds roughly 160 ms (TCP plus TLS 1.3) to each request, while a pooled connection adds nothing.

What do HTTP/2 and HTTP/3 change?

HTTP/1.1 handles roughly one request at a time per connection, so browsers open several connections in parallel, each paying handshake costs. HTTP/2 multiplexes many requests over a single TCP connection, removing most of that waste, but because it runs on one TCP stream, a single lost packet can briefly stall every request on it. HTTP/3 runs over QUIC on UDP, combines the transport and encryption handshakes so a new connection usually needs one round trip, and keeps streams independent so loss on one does not block the rest. Connections that have been established before can often resume with zero extra round trips for the handshake. The full story is in HTTP/1.1, HTTP/2, HTTP/3 and QUIC.

The gains are largest when RTT is high, packet loss is present, or users move between networks (mobile). On a short, clean path the difference is usually small, so do not expect HTTP/3 alone to fix a slow origin.

Where should servers be located?

Distance sets a floor you cannot beat. Light in optical fibre travels at about 200,000 km/s, which is roughly 5 µs per km one way, or about 1 ms of RTT per 100 km of fibre path. Real routes are longer than straight lines. A user 8,000 km from your only server will see at least 80 ms RTT in theory and in practice often 120 to 200 ms. If your users are spread over several continents, a single origin region cannot serve all of them well.

The options, from simplest to most involved:

• Pick the region closest to most users. Cloud providers offer many regions; see data centres, cloud regions and edge.
• Put a CDN in front. Static files and cacheable pages are served from an edge location near the user. See CDN and anycast.
• Replicate the application and data across regions. This gives dynamic requests a nearby server too, but brings complexity in keeping data consistent.

What makes API latency different from page latency?

APIs tend to be called many times, often one after another. A mobile app screen that makes five dependent calls, each with a 120 ms round trip and 30 ms of server time, spends about 750 ms waiting before it can draw. Several things are specific to APIs:

• Sequential dependencies: call B needs the result of call A. Combining calls into one request, or letting the server return related data together, removes whole round trips.
• Server-to-server calls: inside one data centre, RTT is typically well under 1 ms, so many small internal calls are cheap. Across regions the same pattern becomes expensive.
• Dynamic responses: they are often uncacheable, so the origin location matters more than for static assets.
• Tail latency: users and downstream services feel the slow requests, not the average. If one in twenty calls is slow and a screen makes ten calls, most screens include at least one slow call. Track percentiles such as p95 and p99 rather than only the mean.
• Cold connections and retries: a timeout followed by a retry doubles the wait the user sees.

How do you measure website and API latency?

Separate the network from the server

The most useful step is to split the total wait into parts. With curl you can print the timing of each phase using its -w option (variables such as time_namelookup, time_connect, time_appconnect, time_starttransfer and time_total). Browser developer tools show the same breakdown per request in the Network panel, labelled as DNS, connecting, SSL, waiting (TTFB) and content download.

Measure from where users are

A test from your office tells you about your office. To see what visitors in other regions experience, measure from those regions. You can get a first look from your own device with GlobeLatency: add your hostname as a custom destination in a test, pick a small static file such as /favicon.ico to measure network latency, or a full page URL to include server processing time. The tool times real HTTPS requests from your browser until the response headers arrive, so the numbers include DNS, TCP and TLS when a new connection is needed, and sit a little above raw ICMP ping. Custom destinations stay in your browser. For continuous watching, the monitor keeps recent readings and shows average, best and worst. Remember that this measures from your location only; see CDN performance testing for how to judge edge performance from several places.

Use percentiles over time

Take many samples, at different times of day, and read the median and a high percentile. A single reading hides spikes. See mean, median, min and max.

Worked examples

Example 1: the same site, two visitors

A site's only server is in one city. Visitor A is 200 km away with a 6 ms RTT; Visitor B is 9,000 km away with a 170 ms RTT. The server needs 60 ms to generate the HTML. With TLS 1.3 and a fresh connection:

• Visitor A: 6 (TCP) + 6 (TLS) + 6 (HTTP) + 60 = about 78 ms to first byte.
• Visitor B: 170 + 170 + 170 + 60 = about 570 ms to first byte.

The server is identical, yet Visitor B waits more than seven times longer. Adding a CDN that caches the HTML at an edge 15 ms from Visitor B could cut their first byte to around 45 + 5 = 50 ms for a cache hit, because only the short hop to the edge is paid.

Example 2: a chain of API calls

A single-page app loads a profile, then the profile's team, then the team's projects, one after another. RTT is 90 ms and each call takes 25 ms on the server, over a reused connection: 3 × (90 + 25) = 345 ms. If the backend offers one combined endpoint that returns all three in 40 ms of server time, the total is 90 + 40 = 130 ms. Fewer round trips beat a faster server here.

Example 3: the cost of a new connection per call

A backend job calls a partner API 20 times in a row from a machine 120 ms RTT away. If it opens a new TLS 1.3 connection each time, each call costs about 120 + 120 + 120 = 360 ms plus server time, so 7.2 s for the batch before server work. With one pooled connection the first call costs 360 ms and the other 19 cost 120 ms each, 2.64 s in total. Nothing about the network changed; only reuse.

Common mistakes and myths

• Testing only from the server's own region. Everything looks fast nearby. Check from where your users actually are.
• Blaming bandwidth. Buying a bigger pipe for the server rarely fixes a delay made of round trips.
• Looking only at averages. A good mean can hide a bad p99 that your most affected users experience.
• Confusing TTFB with total load time. A fast first byte can still be followed by slow scripts and large images.
• Assuming a CDN speeds up everything. It helps cacheable content most. Uncacheable calls still travel to the origin.
• Ignoring third-party requests. Tags, fonts and widgets from other hosts each need their own DNS lookup and connection.
• Comparing ICMP ping to HTTP timing. They measure different layers; see ping vs HTTPS latency.

What to do next

Start by finding out where the time goes: split a request into DNS, connect, TLS, waiting and download, and measure from more than one place with the free latency test. If the waiting portion is large, read TTFB and web performance. If the network portion is large for distant users, read CDN and anycast and CDN performance testing, then work through how to reduce latency for the full list of options.