Internet Routing, BGP and Peering: Why Paths Are Indirect

Quick answer: The internet is not one network but a federation of tens of thousands of independently operated networks called autonomous systems (ASes). They exchange routing information using the Border Gateway Protocol (BGP). BGP selects paths mainly by commercial relationships and policy, not by geography or measured speed, so real paths are often longer than the straight-line route, and the way your ISP connects to the rest of the world strongly shapes your latency. This page explains the concepts you need to interpret why one destination is fast and another is oddly slow.

What is an autonomous system?

An autonomous system is a set of IP address ranges (prefixes) and routers managed by one organisation under a common routing policy. Each is identified by an AS number, such as AS64496 (a number reserved for documentation). A typical home user sits inside an ISP's AS. Content companies, cloud providers, universities, and enterprises run their own. A large global ISP may operate several.

Inside an AS, the operator uses an internal routing protocol (OSPF or IS-IS are common) tuned for efficiency. Between ASes, BGP is used. The distinction matters because inside a network the goal is usually shortest or lowest-cost path, while between networks the goal is whatever the contracts and policies say.

How does BGP work?

Each AS announces the prefixes it owns to its neighbours. Neighbours re-announce those prefixes to their own neighbours, adding their AS number to an AS path. A router that hears several announcements for the same prefix picks one. The decision considers, in order of typical importance:

1. Local preference: the operator's own policy, such as "prefer routes learned from customers, then peers, then paid transit" because customer routes earn money and transit costs money.
2. Shortest AS path: fewer ASes between source and destination, a rough proxy for simplicity, not distance or delay.
3. Other tie-breakers, including origin type and, within a network, the closest exit point.

Nothing in this process measures latency. A route through 3 ASes that circles half a continent can beat a 4-AS route that is geographically direct. That single fact explains most "why is this path so indirect?" questions. For the broader picture of how traffic gets from device to server, see how the internet works.

What are tier-1 networks, transit providers, and peering?

Networks interconnect in two main commercial ways:

Arrangement	Who pays	What is reachable	Typical effect on latency
Transit (customer to provider)	Customer pays provider	The whole internet, via the provider	Reliable reach; path may be longer if provider is far or congested
Settlement-free peering	Usually neither side pays the other	Only each side's own customers and prefixes	Often the shortest, lowest-latency route between the two
Paid peering	One side pays for a direct connection	Same as peering	Similar to peering, with dedicated capacity

A tier-1 network is conventionally defined as one that can reach every other network on the internet through settlement-free peering alone, without buying transit. Only a small number of networks are widely described that way, and the exact list is debated and changes over time, so treat "tier" labels as informal. Smaller ISPs buy transit from larger ones, and also try to peer directly with the content networks their users want most.

What is an internet exchange point (IXP)?

An internet exchange point is a physical location, usually one or more data-centre sites with shared switches, where many networks connect to exchange traffic directly. Instead of building a separate cable to every partner, a network connects once and can peer with many others across the same fabric. Benefits include lower cost, shorter paths that stay inside a region, and improved resilience. When a local ISP and a local content network both attend the same exchange, traffic that would otherwise detour through another country can stay in town, often cutting round-trip time by tens of milliseconds.

Exchanges exist in most major cities, with large ones in Europe, North America, and Asia. A country with few exchanges or weak local peering will often send domestic traffic out through a foreign hub and back, a pattern called tromboning.

What is hot-potato routing and why are paths asymmetric?

When traffic must cross from network A to network B at one of several possible interconnection points, the sending network usually chooses the exit closest to the source, minimising its own backbone use. This is hot-potato routing. The alternative, cold-potato, keeps traffic in your own network until it is close to the destination, which gives better performance at higher cost; some large content networks do this for their own customers' benefit.

Because each network makes its own choice, the outbound path and the return path can differ. This is asymmetric routing. It is normal. The measured round trip is the sum of both directions, so a long return path inflates the time even when the forward path looks short, and a traceroute shows only the forward half.

Why are internet paths longer than the straight line?

Several independent factors stack up:

• Cable routes follow rights of way, coastlines and existing duct corridors, not geodesics (undersea cables and the speed of light).
• Networks meet where they have equipment, typically in a limited set of cities.
• Policy beats distance: BGP prefers cheaper or customer-learned routes.
• Hot-potato exits hand traffic over early.
• Queueing and processing at each router add small, variable delay.

As a rule of thumb, measured RTT between two cities is commonly 1.3 to 2 times the theoretical minimum for the great-circle distance, and occasionally much more. Worked example: two cities 800 km apart have a physical minimum of roughly 8 ms RTT (about 1 ms per 100 km of fibre path). A direct, well-peered route might measure 10 to 14 ms. If the ISP has no direct path and traffic goes via a hub 600 km away in a different direction, the fibre path may be about 1,500 km, giving about 15 ms of propagation alone, and about 18 to 25 ms measured.

How do route changes cause latency spikes?

BGP paths change for ordinary reasons: a link fails, a provider does maintenance, an operator adjusts traffic engineering, or a peering link becomes congested and traffic is shifted. Effects you can observe:

• A step change: latency jumps from 35 ms to 60 ms and stays there until the route changes back. This is the signature of a new path, as opposed to congestion, which tends to fluctuate with time of day.
• Brief loss or spikes during convergence: after a change, routers across the internet take seconds to minutes to settle, during which some packets are dropped or looped.
• Daily patterns: a peering link that fills up in the evening produces higher latency and jitter at that time (jitter explained), while the same destination is fine in the morning.

A continuous monitor makes these patterns visible: a flat line that jumps and stays is a path change; a wavy line is load. GlobeLatency's monitor shows each destination's recent readings as a sparkline, so you can spot a step change without reading raw numbers.

What are BGP leaks, hijacks, and outages?

BGP historically trusts what neighbours announce. Mistakes or malice can therefore have wide effects, described here generally rather than as specific events:

• Route leak: a network announces routes to the wrong audience, such as passing provider-learned routes to another provider. Traffic is attracted to a network that cannot handle it, causing congestion, high latency, or loss.
• Prefix hijack: a network announces address space it does not own, attracting traffic meant for someone else. A more-specific announcement (a smaller prefix) wins over a broader one.
• Withdrawal outage: a network's announcements disappear, for instance after a misconfiguration, and other networks no longer know a path to it. The destination looks completely unreachable although its servers are running.

Defences include prefix filtering, maximum-prefix limits, and RPKI route origin validation, which lets owners cryptographically declare which AS may originate their prefixes. Adoption is substantial but incomplete. For an ordinary user the symptom is simply that some destinations become slow or unreachable for a while, while others work normally, which is a hint the problem is not your Wi-Fi.

How does ISP peering quality show up in tests?

If your ISP connects well to the networks hosting popular services, you will see low latency to those destinations even when they are far away. Poor peering shows as:

• One destination or provider being much slower than others at a similar geographic distance.
• Latency to a nearby destination higher than to a farther one.
• Evening-only degradation toward specific providers while everything else is unchanged.
• Large differences between you and other visitors, which the global comparison expresses as a percentage against the world average for that destination.

Switching DNS resolver, changing browsers, or rebooting will not fix peering. A VPN sometimes does, by sending traffic over a different path, and sometimes makes things worse (VPN and latency). If the pattern is persistent, collect evidence and use talking to your ISP.

How do you read a traceroute?

Traceroute (tracert on Windows) sends probes with TTL values of 1, 2, 3 and so on. Each router that decrements the TTL to zero returns a "time exceeded" message, revealing itself. Example output, simplified and invented purely for illustration:

Hop	Host (example)	RTT	Interpretation
1	192.168.1.1	2 ms	Your home router
2	isp-gateway.example.net	9 ms	First ISP router; access network (cable, DSL, fibre)
3	core1.city-a.example.net	11 ms	ISP core
4	transit.example.org	13 ms	Hand-off to another network
5	core2.city-b.example.org	58 ms	Large jump: long-distance link, probably a different city or a sea crossing
6	edge.destination.example	60 ms	Destination network

How to read it:

• Look for a jump that persists in all later hops. A big increase at hop 5 that carries through to the end means delay was added there (distance or congestion). A single hop with a high time and normal times afterwards is usually that router deprioritising replies to probes, not a real problem.
• Stars (* * *) mean no reply to probes, often because a router is configured not to answer. They are not necessarily loss; check whether later hops answer.
• Hostnames often include airport or city codes that hint at geography, which helps spot detours.
• Only the forward path is shown. Tools such as mtr repeat probes and report loss and variation per hop, which is more reliable than a single run.

Traceroute uses ICMP, UDP, or TCP probes, depending on the tool and options, and does not measure what a browser measures. Compare with ping vs HTTPS latency.

Common mistakes and myths

• "The shortest path is always used." BGP follows policy; the shortest AS path is not the shortest distance.
• "A high hop count means high latency." Ten hops inside one city can total 5 ms; two hops across an ocean can total 120 ms.
• Treating one slow traceroute hop as the culprit. Look at where the increase begins and persists.
• Assuming the return path equals the outbound path.
• Blaming a website when only your ISP's route to it is poor, or the reverse. Test several destinations before deciding.
• Confusing congestion with a route change. One is wavy, the other is a step.

What to do next

Run a broad test so you can see whether slowness is specific to one network or general: run a free latency test across several regions and providers and see the live map for geography. If one provider is consistently slow for you, read troubleshoot high latency and then talking to your ISP. To understand why anycast services often avoid these problems, continue with CDN and anycast.