Anycast Is Not Load Balancing: How BGP Turns Geography Into a Failover Mechanism

Anycast Is Not Load Balancing: It Is Geography Exploited as a Failover Mechanism

Anycast is a network addressing method in which the same IP prefix is announced from many geographically separate locations at once, and the internet's own routing system delivers every request to whichever site is currently closest by path length. First defined in RFC 1546 in 1993 and formalized for IPv6 in RFC 2526, it delegates your traffic-distribution problem to BGP, the protocol that glues autonomous systems together. That delegation is exactly what makes it fast, fragile, and worth understanding before you touch it.

Most explanations fail engineers because they frame anycast as "a load balancer but bigger." That framing is wrong, and it leads to broken designs. A load balancer is a box you own, with health checks you configure, distributing requests across backends you can enumerate. Anycast has no box, no session table, and no way to enumerate which site received a given user five minutes ago. The internet's routers — hundreds of thousands of them, owned by thousands of autonomous systems — make the per-packet delivery decision using a routing protocol you influence but never control.

This article rebuilds the concept from the protocol up: what BGP actually does when ten sites announce one prefix, why that buys you millisecond failover and free DDoS dilution, and the three specific ways anycast deployments fail (state loss, route leaks, withdrawal storms). It closes with a build-vs-buy decision table, because the honest answer for most teams is not "build an anycast network" — it is "rent one from someone who already announced the prefix."

What Actually Happens When Ten Cities Announce the Same Prefix

The mechanics reduce to one sentence: each site runs BGP, announces an identical prefix, and every router on the path picks the announcement with the shortest AS-path, which in practice tracks network geography. An autonomous system (AS) is a routing domain under one policy — a telco, a cloud provider, your company if you register an ASN. BGP's default tie-breaking rule prefers the shortest AS-path, so a user in Frankfurt is handed to the Frankfurt site and a user in São Paulo to the São Paulo site, with zero application code involved.

Three properties fall out of that single rule, and they are the entire value proposition:

1. Proximity routing for free. The shortest path in AS-hop count correlates strongly with fiber distance. As of November 2026, Cloudflare's network — the largest anycast deployment most people touch daily — announces one prefix from data centers in more than 335 cities under a single ASN (AS13335), which is why 1.1.1.1, its public DNS resolver launched on April 1, 2018, answers queries in roughly 11 milliseconds on average worldwide.
2. Failover without DNS. When a site dies, its border routers stop announcing — either deliberately, via health-check-driven BGP session teardown, or involuntarily, when the BGP session itself drops. The withdrawal propagates, and traffic re-homes to the next-nearest site in seconds, with no DNS record change and no TTL wait. A comparative analysis published in November 2025 puts the contrast plainly: unicast failover governed by DNS updates takes seconds to minutes because resolver caches keep handing out the dead IP, while anycast failover is governed by BGP convergence and completes in milliseconds to seconds.
3. Attack dilution by construction. A volumetric DDoS attack aimed at one anycast address is split across every announcing site, because the attacking traffic originates from everywhere and therefore follows every path. The load per site shrinks roughly by the number of well-placed sites. Cloudflare's Q3 2026 DDoS report, published in late October 2026, quantifies what this absorbs: 18.2 million network-layer attacks mitigated in one quarter — an average of 8,226 per hour, one every 0.44 seconds — alongside 14.4 trillion HTTP requests, with short-burst attacks up 208% year over year. No single-site scrubbing fleet survives that cadence.

The counter-intuitive part: none of this is configured per request or per user. An operator sets policy (local preference, AS-path prepending, communities) at the edges of the routing domain, and the distributed decision-making happens in thousands of routers the operator will never log into.

Why the Internet's Oldest Critical Service Bet on Anycast First

The DNS root server system is the canonical existence proof that anycast works at planetary scale: thirteen IP addresses (a.root-servers.net through m.root-servers.net) serve the entire hierarchical lookup chain, yet behind those addresses sit well over a hundred physical instances spread across continents, each pair sharing one IP only because anycast makes it coherent. Before anycast, the root infrastructure was genuinely thirteen machines in thirteen places — a single point of failure per letter, and a fat target. The switch to anycast, which the roots adopted through the 2000s, multiplied the effective footprint without multiplying the address space, so an attack or an outage degrades one instance rather than the letter of the alphabet.

The root operators picked anycast for reasons that generalize to any service you run:

1. Queries are tiny, idempotent, and stateless. A DNS lookup is one UDP packet in, one UDP packet out, and any instance can answer any query identically. No session stickiness required.
2. Latency compounds down the tree. Every uncached lookup passes through a root; shaving 150 ms off that hop improves the whole resolver chain.
3. Availability is existential. If the roots go dark, the internet does not immediately stop — cached records persist — but new delegations fail. The F-root alone runs dozens of instances worldwide; ISC, its operator, contracts Cloudflare to host additional F-root instances, an arrangement Cloudflare documents publicly on its DNS root learning page.

The lesson to steal for your own architecture: anycast is a match made for request/response workloads where every node can answer every request. DNS, public APIs, token validation endpoints, time servers (NTP), and object cache front-ends fit. Anything with a session, a queue, or a server-side conversation in progress needs the caveats in the state section below.

Anycast vs. DNS-Based Failover: The Convergence Gap

The decision between anycast and DNS-based global load balancing (GSLB) is a decision about who moves traffic during an incident: the routing system in seconds, or the resolver cache in minutes. Both are legitimate; they operate on different clocks and different failure surfaces. The November 2025 comparison and a June 2026 global load balancing analysis agree on the numbers below, and they should anchor your choice.

The conclusion the table forces: anycast wins when you need sub-10-second regional failover for stateless traffic; GSLB wins when you need per-user affinity, per-region content, or fine-grained health signals without owning routing assets. Google's own SRE book — which documents the GSLB approach the company uses internally — is candid that DNS-based distribution is the workhorse precisely because path selection is too important to leave fully to BGP in complex deployments. Mature systems layer both: anycast for the front-door edge (DNS, TLS termination, static cache), GSLB behind it for the stateful tiers.

The State Problem: Why Anycast Breaks TCP Conversations

Anycast has one hard technical limit inherited from IP itself: a return packet from an anycast address is not guaranteed to reach the same instance that sent it, which is why an anycast address can serve requests but cannot safely originate long-lived stateful conversations. The mechanism is not subtle. A TCP connection is pinned by four values (source IP, source port, destination IP, destination port), and the destination IP is the anycast address — shared by every site. If the route to your site is withdrawn mid-conversation, the next packet from the client is delivered to a different site, an instance that has no socket for that four-tuple. The client gets a connection reset, or worse, a silent stall.

This produces a hard classification of workloads:

1. Safe by design (stateless request/response): DNS, NTP, public API GETs, token introspection, cache lookups, redirect servers. Re-home the route mid-packet and the worst case is a retry.
2. Safe with retransmission machinery: HTTP/2 or HTTP/3 sessions, gRPC streams, database drivers with reconnect logic. Usable if the application tolerates connection-level retries and you accept occasional resets during failover windows.
3. Unsafe without external state: login sessions, shopping carts, websockets with server-side rooms, FTP, database connections holding transactions. The standard fix, which the June 2026 Hong Kong deployment guide describes, is to terminate anycast at the stateless edge and back every edge node with shared session storage (a replicated Redis cluster, a shared database) so any site can resume any session.

Related and equally binding: anycast addresses are not valid source addresses for client-initiated traffic, because replies route to "the nearest instance," not to the originating host. An anycast endpoint listens; it does not dial out.

The Failure Modes Nobody Puts in the Sales Deck

Anycast's weaknesses are the mirror image of its strengths: the same global, fast-propagating mechanism that heals failures spreads them too. Four failure modes account for nearly every public anycast incident, and each has a documented, recent example.

1. Withdrawal storms (the blast-radius problem). On November 18, 2025 at about 11:48 UTC, Cloudflare deployed a BGP policy intended to withdraw a small set of routes on one router in Ashburn, Virginia. The match condition was too broad and withdrew everything; the withdrawal raced through the global mesh and took 19 data centers off the internet at once, taking Discord, Shopify, and half the API economy down with them. The post-mortem, dissected in a detailed November 2025 Chinese-language engineering review, called the lesson by name: the change had no bounded blast radius. Anycast turns local config errors into global outages in seconds.
2. Route leaks and hijacks (the trust problem). BGP trusts announcements. In June 2026, a BGP leak intended to block Telegram in India escaped its intended scope and was documented by Kentik's routing team as the latest in an accidental-hijack lineage going back to Pakistan Telecom's famous YouTube incident of 2008. If someone announces your prefix — or you announce someone else's, or prepend wrongly — the routing system cannot tell the difference. RPKI signing and route filtering are the mitigations, and per routing-security tracking by MANRS and vendors like Kentik as of 2026, adoption is widespread among large networks but far from universal.
3. Asymmetric load (the geography problem). Shortest-AS-path does not mean "capacity-aware." A region with one fat site and one thin site will happily overflow the thin site when the fat one's route flaps, because BGP has no concept of load. Operators compensate with local preference and prepending — a June 2026 academic optimization study (AnyPro) notes that client-to-site mappings routinely misalign with operator expectations because inter-domain routing is opaque. Expect to tune, not to configure once.
4. Split-brain state. Failover moves traffic to a site that may hold no warm data: empty caches, cold session stores, stale replicas. The site was healthy at the routing layer and is starving at the application layer. Health checks that gate BGP announcements must test the application, not the TCP handshake.

A 2026 anycast-specific routing analysis published in late October 2026 adds a structural observation worth internalizing: anycast's "self-healing" property also amplifies configuration errors — one bad policy line propagates to every node announcing the prefix simultaneously, so error and recovery travel the same rails.

What It Takes to Run Your Own Anycast Network

The admission price for self-operated anycast is an ASN, a globally routable prefix, and BGP sessions at two or more points of presence — assets that cost real money and carry permanent obligations, not a checkbox in a cloud console. Both a June 2026 Hong Kong infrastructure guide and a November 2025 multi-region deployment walkthrough state the requirements identically, and RIR documentation confirms the process.

1. Register an autonomous system number (ASN) with a regional internet registry — RIPE NCC, ARIN, or APNIC. RIPE is the common recommendation for flexibility; as of 2026 the fee for a small independent ASN starts around €50 per year through sponsoring LIRs, and 4-byte ASNs are the standard issuance.
2. Acquire a routable prefix. The practical floor for IPv4 is a /24 — a 256-address block — because most ISPs filter longer (smaller) prefixes. IPv6 needs a /48 or similar. Fresh IPv4 space must come via transfer markets; the free pools are exhausted.
3. Establish BGP sessions at each site with transit providers and, ideally, internet exchange (IX) route servers. Each site announces the identical prefix; peering policy must be registered in an IRR database, per the route-server requirements documented by IX operators such as MSK-IX and TOP-IX.
4. Tune distribution: set local preference for preferred exit paths, use AS-path prepending to de-prioritize backup sites, and apply BGP communities so transit providers can apply policy on your behalf — the toolkit every deployment guide from 2025–2026 describes with the same route-map snippets.
5. Automate withdrawal. Health-check daemons (ExaBGP-style announcers tied to load balancer or application probes) must remove a site's announcement when the site degrades. Without this, anycast distributes traffic to corpses.

The honest cost picture: ASN fees, prefix acquisition (the dominant cost for IPv4), monthly transit or colocation at each PoP, engineering time for routing policy and monitoring, and the ongoing risk obligations of keeping announcements clean. For a two-site setup the hard costs typically land in the low thousands of US dollars per year before transit — an amount that buys a lot of managed service instead.

Build vs. Buy vs. Hybrid: Matching Architecture to Team Size

For most engineering teams the right answer is not to own routing assets but to buy anycast behavior at the layer where it pays: managed DNS with anycast authority servers, a CDN or WAF with an anycast edge, or a load-balancer-as-a-service that announces on your behalf. The 2026-era consensus across the deployment guides cited above is blunt: a single VPS or even a small fleet cannot announce anything, because the provider owns the prefix — and providers' "anycast" add-ons are precisely the productized version of the machinery above.

When you evaluate a vendor's anycast claim in November 2026, verify three things: published PoP count and whether all PoPs announce one prefix (ask for the ASN), whether health-driven withdrawal is automatic and how fast, and whether RPKI-signed announcements are on record in public routing data. A provider that cannot answer those is selling the word, not the mechanism.

Five Diagnostic Commands That Show Anycast Working

You can empirically confirm anycast behavior from any Linux box with standard tooling, in under ten minutes. These commands distinguish "the nearest site answered" from "a load balancer answered" and are the same checks the 2026 deployment guides prescribe for post-launch verification.

1. Confirm a single name resolves to one IP everywhere: dig +short www.example.com from three geographically distinct machines (a laptop, two cloud instances in different regions). An anycast service returns the same address everywhere; a per-region DNS scheme returns different addresses.
2. Trace the path: traceroute -n 1.1.1.1 from each location. The hops terminate in different cities (Frankfurt, Singapore, Los Angeles) while the destination IP is identical — the visual signature of anycast.
3. Measure proximity: ping -c 10 1.1.1.1 from each site. Medians within ~30 ms of the metro latency indicate the nearest instance; uniform ~150 ms+ medians from everywhere mean you are crossing an ocean — no local instance for you.
4. Identify the serving instance: public echo services on anycast endpoints (for example, DNS resolver research pages and dig CHAOS TXT id.server @1.1.1.1 where supported) often name the specific PoP answering.
5. Verify failover from outside: from a monitoring box, continuously probe your own anycast address while draining one site's announcement in a maintenance window; successful probe streams show a seconds-scale blip, not a minutes-scale outage — the convergence gap made visible. If you run the probe fleet on cheap regional nodes, checking each probe host's health first — a quick read of load and memory, the same habit our CPU usage checking walkthrough formalizes — keeps the experiment honest, because one stalled collector creates phantom "failovers" that were never routing events.

Deploying a Stateless Service Correctly Behind Anycast

The deployment sequence that survives production has seven ordered steps; skipping the middle ones is how teams learn the state lesson live. The sequence synthesizes the staged rollout checklists from the two 2026 deployment guides and the Cloudflare post-mortem's blast-radius directive.

1. Partition state before partitioning geography. Move sessions and caches to shared storage reachable from every prospective site. A shared Redis cluster and a replicated primary datastore are prerequisites, not optimizations.
2. Make every instance answer identically. Same application build, same static assets, same TLS certificates on every node. An anycast mesh where nodes return divergent content produces regionally inconsistent sites that users describe as "random bugs."
3. Script the announcement. Tie BGP announcement (ExaBGP or the provider's API) to application-level health checks — HTTP 200 on a synthetic endpoint, not ICMP reachability. Withdrawal on failure, announcement on recovery, with hysteresis to prevent flapping.
4. Stage sites in one region first. Prove the health-withdrawal loop with a single site live and a shadow site announced to a test prefix. The 2025 Cloudflare incident is the counterexample: the first global exposure of an untested policy was the production failure.
5. Add sites one at a time, observing traffic shares. After each new announcement, compare request rates per site against your expectation; correct imbalances with local preference and prepending before adding the next site.
6. Test failover explicitly. Kill a site's announcer during low traffic and watch: client error rates should show a short spike (in-flight resets), then recovery — no DNS change, no manual action. Record the convergence time; that number is your real availability story.
7. Monitor globally, alert on withdrawal. BGP monitoring (public route collectors, or vendor BGP-stream services that track announcements and hijacks in real time) should page you when your prefix's announcement set changes unexpectedly — that is both your failover system working and your worst failure mode announcing itself.

Teams running this stack on cloud instances should also treat each site's capacity as a routing input: a small region whose instances are CPU-saturated during a neighboring region's failover will turn a routing success into an application outage. Sizing each PoP to absorb its neighbors' baseline is the rule the 2026 guides converge on.

An Architecture That Uses Anycast for What It Is Good At

A reference pattern that fits most teams with global users: put anycast (or an anycast service) at the two layers where traffic is stateless, and keep sessions pinned behind them. The pattern matches the "hybrid architecture" tier the 2026 Hong Kong guide recommends for large platforms and mirrors how the big public deployments are actually organized.

1. Edge (anycast): DNS resolution, TLS termination, static asset cache, and rate-limit enforcement at 3+ PoPs, each able to serve any request.
2. Routing (GSLB or internal): after TLS terminates, an L7 layer routes stateful operations to regional backends using health and load signals — the division Google's SRE literature describes, with BGP-anycast handling geography and application-level balancing handling affinity.
3. Session/state layer: shared session store (Redis or equivalent) with regional replicas plus a replicated source of truth; every edge can resume any session, so re-homing is invisible to users.
4. Origins (unicast): databases and write-heavy services live on unicast addresses reachable only from the edge tier, never announced publicly. They fail over with their own tools (replication, clustering), not with BGP.

The economical way to assemble the same shape, if you do not own an ASN: the edge tier can be a CDN or anycast DNS product, the origins a pair of affordable lightweight cloud instances — for example the 2 vCPU / 2 GB / 60 GB NVMe tier that SurferCloud's ULightHost line prices at $3.9 per month for new users as of November 2026, renewed at $9.5 per month — one in Hong Kong and one in Los Angeles. What matters is that each origin holds a full copy of the stateless service and the edge can reach either; you get geographic failover from the provider's routing and state safety from your architecture. If you later host that edge tier yourself, keep the operational basics tight — the same checklist that keeps any production box alive (monitoring CPU and memory the way our Linux CPU usage guide walks through, renewing TLS certificates on schedule) protects the sites your routing depends on.

FAQ

Is anycast the same as a CDN?

No. A CDN is an application-layer cache replicated across PoPs; anycast is a network-layer addressing and routing technique. Many CDNs use anycast internally — it is the reason a CDN's edge IP answers from a nearby city — but you can run anycast with no caching at all (DNS roots do exactly that), and you can run a CDN without owning the anycast machinery if you buy the edge from someone else.

Can I set up anycast on my own VPS?

Not really. A VPS gives you an IP inside the provider's prefix, and announcing your own prefix requires your own ASN, a routable /24 (IPv4) or /48 (IPv6), and BGP sessions with transits — assets a single rented instance cannot host. Both 2026 infrastructure guides state this flatly: individual servers cannot originate announcements. The realistic equivalents are provider anycast add-ons or moving the stateless tier onto a service that already runs anycast.

Does anycast protect against every DDoS attack?

It protects against volumetric attacks by diluting them, not against application-layer attacks. If a botnet sends 2 Tbps of junk at one anycast address, the traffic splits across sites and each site's pipe absorbs a fraction — which is how the large public networks survive quarterly attack volumes in the millions of events. But an HTTP-level attack that looks like legitimate traffic, or an attack concentrated near one PoP, still needs rate limiting, WAF rules, and scrubbing. Anycast widens the moat; it does not replace the gate.

Why did Cloudflare go down in November 2025 if anycast is supposed to be resilient?

A BGP policy change on one router matched too many routes and withdrew nearly all announcements from that router, and the withdrawal propagated to 19 data centers within seconds, removing them from the internet simultaneously. Anycast was not the vulnerability's cause or its cure — it was the amplifier. The resilience anycast gives you against site failure is the same speed that spreads operator error, which is why the post-incident industry lesson was bounding the blast radius of routing changes, not abandoning the topology.

Is anycast safe for websockets or long-lived gRPC streams?

Only with reconnection logic on both ends. A mid-stream route withdrawal delivers subsequent packets to a different site that has no socket for the conversation, producing resets. Production patterns: terminate anycast at the handshake/edge layer and hand sessions to unicast backends; or treat every stream as resumable with client retry and resumption tokens, so a re-home costs one reconnect instead of user-visible failure. A managed edge in front of your own boxes is the cheapest hybrid — for example anycast TLS termination at a CDN answering to a pair of regional origins, the shape our Hong Kong edge deployment guide walks through for Asia-facing traffic.

How is anycast different from round-robin DNS load balancing?

Round-robin DNS rotates the addresses a name resolves to, and each client stays pinned to one address until its cache expires — so failover is bounded by TTLs (minutes, typically). Anycast rotates nothing; one address is live everywhere, and the routing system moves traffic near-instantly when a site withdraws. The two also compose badly if mixed naively: pointing round-robin DNS at anycast sites hides which site failed, muddying both health checks and debugging.

Do I need anycast if my users are all in one country?

Usually not for latency — one well-placed unicast site with a CDN in front achieves the same user experience. Anycast still adds two things: a layer of volumetric DDoS dilution, and failover if you later expand. The pragmatic move for single-market products is a CDN/WAF (which is anycast you rent) in front of one or two origins, deferring the routing-asset question until geography actually hurts.