Network Watcher and Connection monitor
The networking lessons used Network Watcher's tools one at a time to answer one question at a time. This lesson looks at Network Watcher as a toolbox - what each tool answers, so you reach for the right one - and at Connection monitor, which turns a one-off connectivity check into a test that runs every minute and writes its results to Log Analytics, so that "is the app able to reach its database?" becomes a graph and an alert instead of a 2 am SSH session.
Need to know: Network Watcher is a regional service, enabled automatically per region (one NetworkWatcher_<region> resource in NetworkWatcherRG). Its diagnostic tools are IP flow verify (would an NSG allow this 5-tuple, and which rule decides), NSG diagnostics / effective security rules, Next hop (where does the route table send this packet), Connection troubleshoot (test-connectivity: an actual probe from a VM, hop by hop, with the issue found), packet capture, VPN troubleshoot, and flow logs (VNet flow logs - NSG flow logs can no longer be created - with Traffic analytics on top). Connection monitor runs tests from source endpoints (VMs with the Network Watcher agent extension, on-premises machines) to destinations over TCP, HTTP or ICMP on a schedule, with thresholds, and stores results in the NWConnectionMonitorTestResult table.
The toolbox
| tool | question it answers | command |
|---|---|---|
| IP flow verify | does an NSG allow this flow to/from this VM, and which rule? | az network watcher test-ip-flow |
| Effective security rules | every NSG rule that applies to this NIC, merged | az network nic list-effective-nsg |
| Next hop | where does a packet from this VM to that IP go? | az network watcher show-next-hop |
| Effective routes | the whole route table of this NIC | az network nic show-effective-route-table |
| Connection troubleshoot | can this VM actually connect to that host:port right now? | az network watcher test-connectivity |
| Packet capture | what is on the wire? | az network watcher packet-capture create (portal) |
| Flow logs + Traffic analytics | which flows happened, allowed or denied, over time? | az network watcher flow-log create --vnet |
| Connection monitor | is this path healthy every minute, and since when not? | az network watcher connection-monitor create |
The first five are point-in-time. IP flow verify and next hop only read configuration (they work even when the VMs are idle); connection troubleshoot sends real probes from the VM (through the Network Watcher agent extension) and reports per hop.
$ az network watcher list --query "[].{name:name, location:location, group:resourceGroup}" -o table
Name Location Group
------------------------- ---------- ----------------
NetworkWatcher_westeurope westeurope NetworkWatcherRG
$ az network watcher test-ip-flow -g rg-az5-net2 --vm vm-app --direction Outbound --protocol TCP --local 10.91.1.4:50000 --remote 10.92.1.4:5432 -o table
Access RuleName
------ --------------------------------------
Allow defaultSecurityRules/AllowVnetOutBound
$ az network watcher show-next-hop -g rg-az5-net2 --vm vm-app --source-ip 10.91.1.4 --dest-ip 10.92.1.4 -o table
NextHopIpAddress NextHopType RouteTableId
---------------- ----------- ------------
VNetPeering System Route
$ az network watcher test-connectivity -g rg-az5-net2 --source-resource vm-app --dest-address 10.92.1.4 --dest-port 5432 --query "{status:connectionStatus, avgLatencyMs:avgLatencyInMs, hops:hops[].{type:type, address:address, issues:length(issues)}}" -o json
{
"avgLatencyMs": 5,
"hops": [
{
"address": "10.91.1.4",
"issues": 0,
"type": "Source"
},
{
"address": "10.92.1.4",
"issues": 0,
"type": "VirtualMachine"
}
],
"status": "Reachable"
}
In this lab the spokes are directly peered, so everything is green: the outbound flow is allowed, the next hop is the peering, the probes get through with a latency.
Connection monitor: the same check, every minute
A connectivity check you run once tells you about now. A connection monitor keeps running it and records every result, so you see when it started failing and can alert on it:
$ APP=$(az vm show -g rg-az5-net2 -n vm-app --query id -o tsv); DB=$(az vm show -g rg-az5-net2 -n vm-db --query id -o tsv)
$ az network watcher connection-monitor create -n cm-app-to-db -l westeurope --endpoint-source-name vm-app --endpoint-source-resource-id $APP --endpoint-dest-name vm-db --endpoint-dest-resource-id $DB --test-config-name pg-5432 --protocol Tcp --tcp-port 5432 --test-group-name app-db --workspace-ids $(az monitor log-analytics workspace show -g rg-oncall-lab -n law-sysop --query id -o tsv) --query "{name:name, status:monitoringStatus, tests:testGroups[0].name}" -o json
{
"name": "cm-app-to-db",
"status": "Running",
"tests": "app-db"
}
$ az network watcher connection-monitor list -l westeurope --query "[].{name:name, status:monitoringStatus}" -o table
Name Status
------------ -------
cm-legacy Running
cm-app-to-db Running
The pieces of a connection monitor:
- endpoints - sources (Azure VMs or scale sets with the Network Watcher agent, on-premises machines with the Azure Monitor agent) and destinations (VMs, addresses, URLs, Azure resources);
- test configurations - protocol (TCP port, HTTP with method/path/expected status codes, ICMP), the test frequency (default 30 seconds) and thresholds (checks failed %, round-trip time) that decide pass or fail;
- test groups - which sources test which destinations with which configurations;
- the workspace that stores the results (
NWConnectionMonitorTestResult,NWConnectionMonitorPathResult).
Five minutes later there is history. Then someone tightens the database subnet:
$ sleep 300
$ WS=$(az monitor log-analytics workspace show -g rg-oncall-lab -n law-sysop --query customerId -o tsv)
$ az monitor log-analytics query -w $WS --analytics-query "NWConnectionMonitorTestResult | where TestGroupName == 'app-db' | summarize checks=count() by TestResult" -o table
TableName TestResult Checks
------------- ---------- ------
PrimaryResult Pass 5
$ az network nsg rule create -g rg-az5-net2 --nsg-name vm-dbNSG -n deny-app-subnet --priority 300 --source-address-prefixes 10.91.1.0/24 --destination-port-ranges 5432 --protocol Tcp --access Deny -o none
$ sleep 300
$ az monitor log-analytics query -w $WS --analytics-query "NWConnectionMonitorTestResult | where TestGroupName == 'app-db' | summarize first_fail=minif(TimeGenerated, TestResult == 'Fail'), last_pass=maxif(TimeGenerated, TestResult == 'Pass'), fails=countif(TestResult == 'Fail')" -o table
TableName Fails First_fail Last_pass
------------- ----- ------------------------ ------------------------
PrimaryResult 4 2026-09-22T20:06:18.200Z 2026-09-22T20:05:18.200Z
first_fail is the minute the NSG change landed - which the activity log can then attribute to a person or a pipeline (az monitor activity-log list --offset 15m). A log search alert on NWConnectionMonitorTestResult (or the built-in connection monitor metrics, ChecksFailedPercent and RoundTripTimeMs) pages someone the first time the path breaks, instead of when the users notice.
Flow logs and Traffic analytics
To see which flows happened - who talked to whom, allowed or denied - enable VNet flow logs (storage account for the raw logs, optionally Traffic analytics into a workspace for the aggregated view):
$ az network watcher flow-log create -g rg-az5-net2 -n fl-nsg -l westeurope --nsg vm-dbNSG --storage-account stflowaz5
ERROR: (simulator) NSG flow logs can no longer be created: creation stopped on 30 June 2025 and NSG flow logs retire on 30 September 2027. Create a virtual network flow log instead: --vnet <name> (it covers every NIC in the VNet, and records whether the traffic was allowed or denied by the NSGs).
$ az network watcher flow-log create -g rg-az5-net2 -n fl-vnet-data -l westeurope --vnet vnet-data --storage-account stflowaz5 --traffic-analytics true --workspace $(az monitor log-analytics workspace show -g rg-oncall-lab -n law-sysop --query id -o tsv) --query "{name:name, target:targetResourceId, enabled:enabled}" -o json
{
"enabled": true,
"name": "fl-vnet-data",
"target": "/subscriptions/00000000-1111-2222-3333-444444444444/resourceGroups/rg-az5-net2/providers/Microsoft.Network/virtualNetworks/vnet-data"
}
NSG flow logs stopped being creatable on 30 June 2025 and retire on 30 September 2027; VNet flow logs cover every NIC in the VNet (or a subnet or a NIC), record flows whether or not an NSG is attached, and include the encryption state. They are not real time (minutes of delay) and cost per GB collected.
On the exam (AZ-104 5.1.6): "which NSG rule blocks this traffic" = IP flow verify (or effective security rules); "why does traffic go to the appliance" = next hop; "test reachability from a VM to a host and port, see the failing hop" = connection troubleshoot; "monitor connectivity between an on-premises server and an Azure VM continuously, with latency" = Connection monitor; "capture packets on a VM" = packet capture; "analyse flows across a VNet" = VNet flow logs + Traffic analytics.
In an interview: "How would you know a network path broke before users tell you?" - "A connection monitor test from the app's VMs to each dependency - database port, the payment API's URL - writing to Log Analytics, with an alert on failed checks or latency. When it fires, the results show the first failing minute, and the activity log shows what changed then."