OnCallReady

Lesson 23.14 · Azure II: Networking & AKS · 12 min read

Hub and spoke: peering, transit, and why spokes cannot see each other

In plain words

Imagine a train network with one central station and lines going out to each town. Each town has a track to the central station, but no track to any other town. To travel from town A to town B you go into the central station, through the ticket barrier, and out on the other line. And coming back, you have to take the same way, or the barrier won't recognise your return ticket.

That's hub and spoke. The hub VNet holds the firewall and gateways; each spoke peers only with the hub. Peering isn't transitive, so spoke-to-spoke needs four things: a route in spoke A to the firewall, the firewall allowing it, forwarded traffic allowed on spoke B's peering, and a return route in spoke B. A peering is two objects, and until both exist it's only Initiated.

A bank has dozens of teams, each with its own VNet, and one firewall, one link to the office network and one DNS setup everyone must share. Connecting every VNet to every other one would be a tangle nobody can secure. So almost every enterprise Azure network - and certainly a bank's - is hub and spoke: one central VNet with the shared services, every team's VNet connected only to it.

What you need to know already: 23.1 (VNets, peering is not transitive, NVA, on-prem, VPN/ExpressRoute gateways), 23.7 (route tables, next hop VirtualAppliance, asymmetric routing), 23.3 (NSGs), 23.10 (private DNS zones), 12.1 (Terraform resources).

                  on-prem (VPN / ExpressRoute)
                           |
                 +---------+---------+
                 |  vnet-hub         |   10.30.0.0/16
                 |  Azure Firewall   |   10.30.254.4
                 |  VPN gateway      |
                 +----+--------+-----+
              peering |        | peering
        +-------------+        +-------------+
        | vnet-spoke-a|        | vnet-spoke-b|
        | 10.31.0.0/16|        | 10.32.0.0/16|
        +-------------+        +-------------+

The hub holds shared services (firewall, gateways, DNS resolvers - servers that answer DNS questions for the spokes - and Bastion); each workload or environment gets a spoke. Spokes are peered only with the hub.

Peering

az network vnet peering create: create one side of a peering; --vnet-name the VNet it belongs to; -n its name; --remote-vnet the VNet on the other side; --allow-vnet-access let the two address spaces talk; --allow-forwarded-traffic accept packets that did not start in the remote VNet but were forwarded through it (by a firewall, say).

az network vnet peering create -g rg-hub --vnet-name vnet-spoke-a -n spoke-a-to-hub \
  --remote-vnet vnet-hub --allow-vnet-access --allow-forwarded-traffic
az network vnet peering create -g rg-hub --vnet-name vnet-hub -n hub-to-spoke-a \
  --remote-vnet vnet-spoke-a --allow-vnet-access --allow-forwarded-traffic

A peering is two objects, one on each side. Until both exist the state is Initiated; with both it is Connected:

# rg-hub = a platform team's hub (the hub-spoke mission builds one)
az network vnet peering list -g rg-hub --vnet-name vnet-hub -o table
AllowForwardedTraffic    AllowGatewayTransit    AllowVirtualNetworkAccess    Name            PeeringState
-----------------------  ---------------------  ---------------------------  --------------  ------------
True                     False                  True                         hub-to-spoke-a  Connected
True                     False                  True                         hub-to-spoke-b  Initiated

Initiated means half a peering - no traffic flows. It is the first thing to look for when "the peering exists but nothing works".

Peered address spaces must not overlap:

ERROR: (VnetAddressSpacesOverlap) Cannot create or update peering ... Virtual networks ... cannot
be peered because their address spaces overlap. Overlapping address prefixes: 10.31.0.0/16.

This is why the VNet lesson told you to plan ranges globally.

Not transitive

spoke-a <-> hub and hub <-> spoke-b does not give spoke-a <-> spoke-b. A peering only carries traffic between the two VNets it joins. For spoke-to-spoke:

  1. a UDR in spoke-a: 10.32.0.0/16 -> VirtualAppliance 10.30.254.4 (the hub firewall)
  2. the firewall allows and forwards it
  3. spoke-b accepts forwarded traffic on its peering to the hub (allowForwardedTraffic) - traffic arriving from the firewall did not originate in the hub
  4. a UDR in spoke-b for the way back: 10.31.0.0/16 -> 10.30.254.4. Without it replies go... nowhere (no peering to spoke-a) - or, with other routes in place, around the firewall, which drops the asymmetric half-flow.

Every one of these four is required. The incident after this lesson is missing two of them.

Gateway transit

Transit here means passing through. --allow-gateway-transit on the hub side and --use-remote-gateways on the spoke side let spokes use the hub's VPN/ExpressRoute gateway (23.1), so on-prem reaches every spoke through one gateway.

Alternatives

Azure Virtual WAN is Microsoft's managed hub (routing between spokes is built in, with a secured hub firewall). AVNM (Virtual Network Manager) can create mesh connectivity and push security admin rules centrally. The underlying rule - one peering, two VNets, no transit - still explains why things do or do not work.

Terraform shape

resource "azurerm_virtual_network_peering" "spoke_to_hub" {
  name                         = "spoke-a-to-hub"
  resource_group_name          = "rg-hub"
  virtual_network_name         = azurerm_virtual_network.spoke_a.name
  remote_virtual_network_id    = azurerm_virtual_network.hub.id
  allow_forwarded_traffic      = true
  use_remote_gateways          = true
}

(and its twin in the hub). Spoke teams often do not have rights on the hub VNet - the hub side is created by the platform team's own Terraform, run separately, which is where "half-peered" states come from.

Diagnosing spoke-to-spoke, in order

1. az network vnet peering list  (both spokes and the hub)     all Connected?
2. allowForwardedTraffic on the DESTINATION spoke's peering     traffic from the firewall accepted?
3. route table on the SOURCE subnet: dest prefix -> firewall    goes to the hub at all?
4. route table on the DESTINATION subnet: source prefix -> fw   comes back the same way?
5. firewall network/application rule for the flow               allowed, and logged?
6. NSGs on both subnets                                         allowed at both ends?

Steps 1-4 are the ones this chapter's incident breaks. Step 5 lives in the firewall's logs. Step 6 is the NSG lesson.

Later (Ch 24): the firewall's logs land in Log Analytics (AZFWNetworkRule / AzureDiagnostics), where KQL shows each allowed or denied flow.

DNS in a hub

Private DNS zones are linked to every spoke (or to the hub, with spokes using a DNS resolver in the hub). If spoke B's service is behind a private endpoint, spoke A must be able to resolve its privatelink name too - "the network works, the name does not" is the other half of every hub-and-spoke ticket.

What you can now do

Why it helps

This is the network shape you'll work in, and "spoke A can't reach spoke B" is one of the most common platform tickets. The causes are always the same short list: a half-created peering stuck in Initiated because the spoke team had no rights on the hub, allowForwardedTraffic missing on the destination side, or a missing return route that makes the flow asymmetric. The six-step diagnosis in this lesson finds them in order.

It also explains organisational patterns: why the platform team owns the hub and its automation creates the hub side of every peering, why address spaces must be planned centrally, and why Virtual WAN or Virtual Network Manager are options at scale. These are standard architecture interview questions for Azure platform roles.

FAQ

Why doesn't spoke A reach spoke B through the hub automatically?

Because VNet peering only carries traffic between the two VNets it connects. Spoke A's peering to the hub doesn't give it any route to spoke B's addresses, and the hub doesn't forward anything unless there's a router in it. Spoke-to-spoke needs a firewall or NVA in the hub, route tables in both spokes pointing at it, forwarded traffic allowed on the peerings, and a firewall rule for the flow.

What does PeeringState Initiated mean?

Only one side of the peering exists. A peering is two separate objects, one on each VNet, and traffic flows only when both exist and the state is Connected on both sides. Initiated is common when a spoke team creates their side but lacks rights on the hub, and the hub side is created later by the platform pipeline, or never. It's the first thing to check when a peering exists but nothing works.

What does allowForwardedTraffic do?

It lets a VNet accept traffic over a peering that didn't originate in the peered VNet itself. Traffic from spoke A forwarded by the hub firewall arrives in spoke B with spoke A's source address, not a hub address, so spoke B's peering to the hub must allow forwarded traffic or it's dropped. It's needed on the destination side of any flow that passes through an NVA or firewall.

What is gateway transit?

A way for spokes to use the hub's VPN or ExpressRoute gateway instead of each having their own. allowGatewayTransit on the hub's side of the peering and useRemoteGateways on the spoke's side make on-premises routes available to the spoke and the spoke's range reachable from on-prem through the one gateway. It saves cost and centralises connectivity, and usually combines with UDRs so that traffic still passes the firewall.

When would I use Virtual WAN instead of a hub VNet?

When you have many spokes or regions and want Microsoft to manage the hub routing. Virtual WAN hubs route between connected VNets, branches and VPNs automatically, and a secured hub adds Azure Firewall with central policy. It trades flexibility for less routing to maintain yourself. Azure Virtual Network Manager is another option for managing connectivity and security admin rules across many VNets. The no-transit rule still explains traditional hubs.

In an interview Mid

Two spokes cannot communicate through the hub firewall. How do you troubleshoot it?

Peering is not transitive, so spoke-to-spoke needs four pieces, and I check them in order:

  1. az network vnet peering list on both spokes and the hub - every peering Connected? Initiated = only one side exists, nothing flows.
  2. allowForwardedTraffic on the destination spoke's peering - traffic arriving from the firewall did not originate in the hub.
  3. Route table on the source subnet: destination spoke's prefix -> VirtualAppliance the firewall's IP. Without it the packet never goes to the hub.
  4. Route table on the destination subnet for the way back. Without it replies go nowhere, or around the firewall, which drops the asymmetric half-flow.
  5. The firewall rules allow the flow (its logs show allowed/denied).
  6. The NSGs on both subnets.

If the network is fine but a name is not, it is DNS: the private DNS zone must be linked to (or resolvable from) the calling spoke.

Also asked: Is VNet peering transitive, and what does that mean in practice? · What does gateway transit do in a hub-and-spoke network? · Why does a private endpoint in one spoke often not resolve from another spoke?

Practise this lesson in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.