Azure DNS and the Standard load balancer
Two pieces of plumbing sit in front of almost every service: the name people type and the load balancer that spreads their connections over healthy servers. When either is wrong, the service is down even though every VM is fine. This lesson hosts a public domain in Azure DNS, then builds a Standard load balancer - public and internal - and breaks it the three ways it breaks in real life: a probe that asks the wrong question, an NSG that drops the probe, and a backend nobody added to the pool.
Need to know: an Azure DNS zone answers for a domain once the registrar delegates it to the zone's four name servers (NS records). Records live in record sets (A, AAAA, CNAME, MX, TXT, NS, SRV, CAA, PTR); a CNAME cannot sit at the apex - use an alias record for Azure resources there. A Standard load balancer (Basic is retired) has a frontend (public IP, or a private IP for an internal LB), a backend pool (NIC IP configurations), health probes (TCP, HTTP, HTTPS) sent from 168.63.129.16 (the AzureLoadBalancer tag) and load-balancing rules joining them. Only healthy backends get new flows; with none healthy, connections time out. DipAvailability is the probe health metric, VipAvailability the data path's.
A public DNS zone
$ az network dns zone create -g rg-az5-lb -n az5-lab.example --query "{zone:name, ns:nameServers}" -o json
{
"ns": [
"ns1-03.azure-dns.com.",
"ns2-03.azure-dns.net.",
"ns3-03.azure-dns.org.",
"ns4-03.azure-dns.info."
],
"zone": "az5-lab.example"
}
Creating the zone does not make the internet ask it. At the registrar (or the parent zone) you replace the domain's NS records with these four name servers - delegation - and from then on every resolver on the internet ends up at Azure DNS for az5-lab.example. Forget it and records you create here are answered by nobody (the lab delegates *.example zones automatically, so dig works at once). A child zone (dev.az5-lab.example) is delegated the same way, with NS records in the parent zone.
Records come in record sets: all the records of one name and type, with one TTL:
$ az network dns record-set a add-record -g rg-az5-lb -z az5-lab.example -n www -a 203.0.113.10 -o none
$ az network dns record-set a add-record -g rg-az5-lb -z az5-lab.example -n www -a 203.0.113.11 -o none
$ az network dns record-set cname set-record -g rg-az5-lb -z az5-lab.example -n shop -c www.az5-lab.example -o none
$ az network dns record-set mx add-record -g rg-az5-lb -z az5-lab.example -n @ -e mail.az5-lab.example -p 10 -o none
$ az network dns record-set txt add-record -g rg-az5-lb -z az5-lab.example -n @ -v "v=spf1 -all" -o none
$ az network dns record-set list -g rg-az5-lb -z az5-lab.example --query "[].{name:name, kind:type, ttl:TTL}" -o table
Name Kind Ttl
---- -------------------------------- ------
@ Microsoft.Network/dnszones/NS 172800
@ Microsoft.Network/dnszones/SOA 3600
www Microsoft.Network/dnszones/A 3600
shop Microsoft.Network/dnszones/CNAME 3600
@ Microsoft.Network/dnszones/MX 3600
@ Microsoft.Network/dnszones/TXT 3600
$ dig +short www.az5-lab.example
203.0.113.10
203.0.113.11
$ dig +short shop.az5-lab.example
www.az5-lab.example.
203.0.113.10
203.0.113.11
@ is the apex (the zone name itself). A record set's TTL (3600 s by default) is how long resolvers may cache it - lower it a day before a migration, not during. Rules the exam checks:
- a CNAME must be the only record at its name, so never at the apex (
@already has SOA and NS); set-recordfor CNAME (one value),add-recordfor the multi-value types;- an alias record set (A/AAAA/CNAME pointing at an Azure resource: a public IP, Traffic Manager, Front Door, a CDN endpoint) follows the resource's IP when it changes and can sit at the apex;
- private DNS zones (Azure II) answer only inside linked VNets and can auto-register VM names; public zones answer the internet.
A public Standard load balancer
Two web VMs in two zones (vm-web1, vm-web2, nginx on port 80) sit behind nothing yet. The pieces:
$ az network lb create -g rg-az5-lb -n lb-web --sku Standard --public-ip-address pip-lb-web --frontend-ip-name fe-web --backend-pool-name be-web -o none
$ az network lb probe create -g rg-az5-lb --lb-name lb-web -n hp-web --protocol Http --port 80 --path /healthz -o none
$ az network lb rule create -g rg-az5-lb --lb-name lb-web -n http --protocol Tcp --frontend-port 80 --backend-port 80 --frontend-ip-name fe-web --backend-pool-name be-web --probe-name hp-web --idle-timeout 15 --enable-tcp-reset true -o none
$ for v in vm-web1 vm-web2; do az network nic ip-config address-pool add -g rg-az5-lb --nic-name ${v}VMNic --ip-config-name ipconfig$v --lb-name lb-web --address-pool be-web -o none; done
$ az network lb show -g rg-az5-lb -n lb-web --query "{sku:sku.name, frontend:frontendIPConfigurations[0].name, pool:backendAddressPools[0].backendIPConfigurations[].id, rule:loadBalancingRules[0].{fe:frontendPort, be:backendPort, idle:idleTimeoutInMinutes}}" -o json
{
"frontend": "fe-web",
"pool": [
"/subscriptions/00000000-1111-2222-3333-444444444444/resourceGroups/rg-az5-lb/providers/Microsoft.Network/networkInterfaces/vm-web1VMNic/ipConfigurations/ipconfigvm-web1",
"/subscriptions/00000000-1111-2222-3333-444444444444/resourceGroups/rg-az5-lb/providers/Microsoft.Network/networkInterfaces/vm-web2VMNic/ipConfigurations/ipconfigvm-web2"
],
"rule": {
"be": 80,
"fe": 80,
"idle": 15
},
"sku": "Standard"
}
- Frontend: the public IP (Standard, zone-redundant) clients connect to.
- Backend pool: NIC IP configurations (or IP addresses in the VNet). VMs in the pool need no public IP; with no outbound rule or NAT gateway they also get no internet access out (Standard LB is closed by default).
- Health probe: every 5 seconds by default; an HTTP probe needs a 200; the probe comes from 168.63.129.16.
- Rule: frontend port -> backend port, with a probe. Distribution is a 5-tuple hash (source IP and port, destination IP and port, protocol);
--load-distribution SourceIPgives session persistence. Idle timeout (4-100 minutes) and TCP reset decide what happens to quiet connections - the cause of many "connection reset by peer" mysteries.
$ LBIP=$(az network public-ip show -g rg-az5-lb -n pip-lb-web --query ipAddress -o tsv)
$ curl -s -m 5 -o /dev/null -w "%{http_code}\n" http://$LBIP/
000
$ az monitor metrics list --resource $(az network lb show -g rg-az5-lb -n lb-web --query id -o tsv) --metric DipAvailability --offset 5m --interval PT1M --query "value[0].timeseries[0].data[-1].average" -o tsv
0
000 and a health probe status of 0: both backends are marked down. The probe asks for /healthz, and nginx on these VMs has no such page - it answers 404, and an HTTP probe accepts only
- Point the probe at a path that exists (or deploy a real health endpoint):
$ az network lb probe update -g rg-az5-lb --lb-name lb-web -n hp-web --path / -o none
$ curl -s -m 5 -o /dev/null -w "%{http_code}\n" http://$LBIP/
200
$ az monitor metrics list --resource $(az network lb show -g rg-az5-lb -n lb-web --query id -o tsv) --metric DipAvailability --offset 5m --interval PT1M --query "value[0].timeseries[0].data[-1].average" -o tsv
100
An internal load balancer
An internal LB has a private frontend in a subnet: for the tier behind the web servers, or for an appliance pair. Same pieces, no public IP:
$ az network lb create -g rg-az5-lb -n lb-api --sku Standard --vnet-name vnet-lb --subnet snet-web --private-ip-address 10.97.1.100 --frontend-ip-name fe-api --backend-pool-name be-api -o none
$ az network lb frontend-ip list -g rg-az5-lb --lb-name lb-api --query "[].{name:name, privateIp:privateIPAddress, allocation:privateIPAllocationMethod}" -o table
Name PrivateIp Allocation
------ ----------- ----------
fe-api 10.97.1.100 Static
A static frontend IP (--private-ip-address) is what clients and DNS records will point at. An internal Standard LB also offers HA ports (a rule with protocol All and port 0: every port, every protocol) - how network virtual appliances are made redundant. Clients in peered VNets can reach an internal LB; on-premises clients too, over VPN/ExpressRoute.
Troubleshooting a load balancer
Symptoms from the client side: connections time out (no healthy backend, or the frontend port has no rule), or some requests fail (one backend unhealthy, or not in the pool). Work through it in order:
| check | how | what it means |
|---|---|---|
| health probe status | metric DipAvailability per backend IP (dimension BackendIPAddress) | 0 = the probe fails on that backend |
| data path | metric VipAvailability | 0 = the frontend itself does not deliver |
| is the probe allowed? | az network watcher test-ip-flow from 168.63.129.16 to the probe port | an NSG must allow the AzureLoadBalancer tag |
| does the probe's question have a good answer? | curl the probe path on the VM (Bastion, run-command) | HTTP probes need 200, not 301 or 404 |
| is the VM in the pool? | az network lb show ... backendAddressPools[].backendIPConfigurations | a new VM nobody added gets no traffic |
| rule and ports | az network lb rule list | frontend port with no rule = timeout |
The second classic, an NSG that blocks the probe. Someone saw an unknown address, 168.63.129.16, "hitting port 80 every five seconds" in the nginx logs and blocked it:
$ az network nsg rule create -g rg-az5-lb --nsg-name nsg-web -n deny-scanner --priority 150 --source-address-prefixes 168.63.129.16/32 --destination-port-ranges 80 --protocol Tcp --access Deny -o none
$ az network watcher test-ip-flow -g rg-az5-lb --vm vm-web1 --direction Inbound --protocol TCP --local 10.97.1.4:80 --remote 168.63.129.16:51000 -o table
Access RuleName
------ --------------------------
Deny securityRules/deny-scanner
$ curl -s -m 5 -o /dev/null -w "%{http_code}\n" http://$LBIP/
000
$ az network nsg rule delete -g rg-az5-lb --nsg-name nsg-web -n deny-scanner
$ curl -s -m 5 -o /dev/null -w "%{http_code}\n" http://$LBIP/
200
That "scanner" was the load balancer's health probe. With it denied at 150 - above the default AllowAzureLoadBalancerInBound (65001) - every probe failed, both backends were marked down and the whole site timed out, although the VMs, nginx and the client rules were fine. 168.63.129.16 is Azure's platform address (probes, DHCP, DNS, the VM agent's wire server): never block it. If a subnet must deny broadly, put an explicit allow for the AzureLoadBalancer tag above the deny.
On the exam (AZ-104 4.3.1-4.3.3): "the domain does not resolve after creating the zone" = update the NS records at the registrar; "point the apex at a public IP that may change" = alias record; "load balance HTTP with URL paths or TLS offload" = Application Gateway, not a load balancer; "private frontend" = internal LB; "backends unhealthy although the app runs" = the probe (path, port, protocol) or an NSG blocking 168.63.129.16; "only some VMs get traffic" = check the backend pool and per-backend health probe status.
In an interview: "Users time out on a load-balanced service but every VM looks fine. What do you check?" - "Health probe status per backend: if they're all down, the probe is failing - wrong path or port, a 301 instead of a 200, or an NSG that blocks 168.63.129.16. Then whether the VMs are actually in the backend pool and the frontend port has a rule."