Lesson 16.31 · Kubernetes: Networking & Storage · 14 min read

Designing policies: three tiers, namespaces, the dash trap

Q: How can I tell AND from OR quickly when reading YAML?

Count the dashes under from or to. Each dash starts a separate peer, and peers are ORed. Selectors under the same dash are ANDed. If the YAML indentation is hard to read, run k describe netpol: within a peer the selectors are listed together, and a ---------- line separates ORed peers.

Q: Why use kubernetes.io/metadata.name instead of my own namespace labels?

The API server sets kubernetes.io/metadata.name on every namespace automatically, equal to its name, and you can't change it to something else. Custom labels like team: edge are only as trustworthy as whoever can label namespaces: anyone with that permission can make their namespace match your allow rule. For "exactly this namespace", use the automatic label.

Q: Can I allow an external SaaS API by hostname?

Not with core NetworkPolicy: it has only ipBlock, no FQDN rules. SaaS IPs change, so an IP list rots. Calico (GlobalNetworkPolicy with domains) and Cilium (CiliumNetworkPolicy with toFQDNs) add DNS-based rules in their own CRDs, or you route egress through an egress gateway or proxy that enforces an allow-list of hostnames.

Q: Should I use ipBlock for pods I know the IP of?

No. Pod IPs change every time a pod is recreated, so an ipBlock for a pod IP silently becomes wrong. Select pods by labels, optionally with a namespace selector. ipBlock is for addresses outside the cluster, like a partner API or an on-prem database subnet.

Q: What is the difference between namespaceSelector and podSelector in one "from" item, versus in two items?

YAML's leading - starts a new list item, and in a NetworkPolicy that one dash changes the meaning: One item: AND - only app=gateway pods in namespaces labelled team=edge. Two items: OR - every pod in the edge namespaces, or app=gateway pods in the policy's own namespace. It passes review and functional tests, and lets far more in. Read policies with k describe netpol: a ---------- separator between peers is the OR. And select "this exact namespace" with the automatic kubernetes.io/metadata.name label - custom namespace labels are only as trustworthy as whoever can set them.

In plain words

Imagine a party invitation list that says: "people from the Edge family who are also named Gateway". That's one person type, with two conditions that must both be true. Now imagine the list says: "people from the Edge family; also, anyone named Gateway who lives here". That's two separate groups, and suddenly the whole Edge family gets in. The difference on paper is one tiny dash.

In a NetworkPolicy, a namespaceSelector and podSelector in the same list item (one dash) mean AND; as two items (two dashes) they mean OR. kubectl describe netpol shows the OR as a ---------- separator. This lesson uses that, and the three-tier shop (ingress controller to frontend to backend to db), to build real policies.

Why design, not just write

One policy is easy. A real app is a chain - the ingress controller, a frontend, a backend, a database, an outside partner API - spread over namespaces owned by different teams. Get one selector subtly wrong and either the site is down (504s everywhere) or, worse, everything works and the database is open to every pod in another namespace. This lesson is about reading and designing policies so neither happens.

What you need to know already: NetworkPolicy, isolation, peers, default deny and the DNS rule (16.29), the Ingress controller as a proxy in its own namespace (16.19), the 504 status (16.19), CIDR (8.3), YAML lists (a - starts a new list item).

The dash trap

In YAML a leading - starts a new list item. These two policies differ by one dash and mean very different things:

  ingress:
  - from:
    - namespaceSelector:              # ONE peer: pods labelled app=gateway
        matchLabels:                  #   IN namespaces labelled team=edge
          team: edge
      podSelector:
        matchLabels:
          app: gateway

  ingress:
  - from:
    - namespaceSelector:              # TWO peers (note the second dash):
        matchLabels:                  #   ANY pod in team=edge namespaces
          team: edge                  # OR
    - podSelector:                    #   app=gateway pods in THIS namespace
        matchLabels:
          app: gateway

One dash = AND inside a peer. Two dashes = OR between peers. The second form passes review ("it mentions edge and gateway"), passes every functional test (the gateway can connect), and lets every pod of the edge namespaces in. Read policies with describe, which renders the structure explicitly:

  Allowing ingress traffic:
    To Port: 8080/TCP
    From:
      NamespaceSelector: team=edge
      PodSelector: app=gateway

versus

    From:
      NamespaceSelector: team=edge
    ----------
      PodSelector: app=gateway

The ---------- separator is the OR. When you review a policy, run describe and look for it.

Selecting namespaces

Custom namespace labels (team: edge) are only as trustworthy as whoever can label namespaces: anyone allowed to add team=edge to their namespace gets in. For "this exact namespace" use the automatic label, which holds the namespace's name and cannot be changed:

namespaceSelector:
  matchLabels:
    kubernetes.io/metadata.name: ingress-nginx

namespaceSelector: {} = all namespaces (the whole cluster).

A three-tier baseline

A three-tier app is the classic shape: a frontend (web pages), a backend (business logic, an API) and a database, each only talking to its neighbour:

internet -> ingress-nginx -> frontend:8080 -> backend:8080 -> db:6379
                                              backend -> 10.0.3.20:443 (partner API)

Policies in namespace shop, each small and named after its edge (one allowed connection between two parts). Shown compressed in YAML's one-line {...} form:

# 1. isolate everything
spec: {podSelector: {}, policyTypes: [Ingress, Egress]}
# 2. DNS for everyone (as in the previous lesson)
# 3. frontend accepts the ingress controller
spec:
  podSelector: {matchLabels: {tier: frontend}}
  ingress:
  - from:
    - namespaceSelector: {matchLabels: {kubernetes.io/metadata.name: ingress-nginx}}
      podSelector: {matchLabels: {app.kubernetes.io/name: ingress-nginx}}
    ports: [{port: 8080}]
# 4. frontend -> backend (egress on frontend, ingress on backend)
# 5. backend -> db     (egress on backend, ingress on db)
# 6. backend -> the partner API outside the cluster
spec:
  podSelector: {matchLabels: {tier: backend}}
  policyTypes: [Egress]
  egress:
  - to:
    - ipBlock: {cidr: 10.0.3.20/32}
    ports: [{port: 443}]

Points that decide whether this works:

The ingress controller is a client pod in another namespace. After the default deny, every Ingress for this namespace returns 504 (the controller's connection is dropped, not refused) until rule 3 exists.
Traffic from the controller comes from the controller pod IP, not from the end user - you allow the controller, and restrict users at the Ingress (whitelist-source-range) or the load balancer.
External egress is by IP (ipBlock). Kubernetes NetworkPolicy has no rules by DNS name (an FQDN, "fully qualified domain name" like api.partner.com). Calico and Cilium add those in their own CRDs (16.26) (GlobalNetworkPolicy, CiliumNetworkPolicy with toFQDNs). For a hosted outside API whose IPs keep changing, that - or an egress gateway, one fixed proxy all outbound traffic must pass - is the real answer.
ipBlock for pod IPs is a trap - pod IPs change. Select pods by label.

Why a policy with only ingress rules does not restrict egress

Isolation is per direction. policyTypes: [Ingress] isolates ingress only; the pod can still connect anywhere. A compromised frontend with an ingress-only policy can still send stolen data out to the internet, or probe every other service in the cluster. Baselines always isolate both directions, and allow egress explicitly.

The debugging checklist

1. which policies select the destination pod?   k get netpol -n NS; describe each;
                                                 compare podSelector with pod labels
2. which select the source pod (egress)?         same, in the source namespace
3. does a rule match the peer?                   namespace labels (k get ns --show-labels),
                                                 pod labels, AND vs OR
4. does it match the port?                       the POD port, protocol (UDP for DNS!)
5. is DNS allowed?                               nslookup from the source pod
6. does the CNI enforce policies at all?         a connection that must fail - does it?

AdminNetworkPolicy / BaselineAdminNetworkPolicy (newer, cluster-scoped policies from the Kubernetes project, with real deny rules and priorities) exist for platform-wide guardrails; know the name, the everyday tool is still namespaced NetworkPolicy.

What you can now do:

spot the one-dash / two-dash difference, in YAML and in describe
design a default-deny baseline for a three-tier app behind an ingress controller
walk the six-step checklist when "the policy blocks something it should not"

Why it helps

The dash trap is the NetworkPolicy bug that passes review and every functional test: the gateway can connect, so everyone is happy, while every pod in the edge namespaces can also reach your backend. Spotting it in a PR is exactly the kind of thing a platform engineer gets trusted for, and it's a common exam and interview gotcha.

Designing a real baseline also forces the decisions you'll face on a bank's platform: allow the ingress controller pods rather than end users, select namespaces by the immutable kubernetes.io/metadata.name label rather than a label anyone can set, and handle external APIs by IP or by the CNI's FQDN policies. The debugging checklist in this lesson is what you'll run when a team says "your policy broke us".

FAQ

How can I tell AND from OR quickly when reading YAML?

Count the dashes under from or to. Each dash starts a separate peer, and peers are ORed. Selectors under the same dash are ANDed. If the YAML indentation is hard to read, run k describe netpol: within a peer the selectors are listed together, and a ---------- line separates ORed peers.

Why use kubernetes.io/metadata.name instead of my own namespace labels?

The API server sets kubernetes.io/metadata.name on every namespace automatically, equal to its name, and you can't change it to something else. Custom labels like team: edge are only as trustworthy as whoever can label namespaces: anyone with that permission can make their namespace match your allow rule. For "exactly this namespace", use the automatic label.

Why do my Ingresses return 504 after the default deny?

The ingress controller is a client pod in another namespace, and its connections to your pods are now dropped (not refused), so it times out waiting and returns 504. Allow ingress to your frontends from the controller pods, selected by namespace and pod labels in one peer, on the pod port.

Can I allow an external SaaS API by hostname?

Not with core NetworkPolicy: it has only ipBlock, no FQDN rules. SaaS IPs change, so an IP list rots. Calico (GlobalNetworkPolicy with domains) and Cilium (CiliumNetworkPolicy with toFQDNs) add DNS-based rules in their own CRDs, or you route egress through an egress gateway or proxy that enforces an allow-list of hostnames.

Should I use ipBlock for pods I know the IP of?

No. Pod IPs change every time a pod is recreated, so an ipBlock for a pod IP silently becomes wrong. Select pods by labels, optionally with a namespace selector. ipBlock is for addresses outside the cluster, like a partner API or an on-prem database subnet.

In an interview Mid

What is the difference between namespaceSelector and podSelector in one "from" item, versus in two items?

YAML's leading - starts a new list item, and in a NetworkPolicy that one dash changes the meaning:

- from:
  - namespaceSelector: {matchLabels: {team: edge}}
    podSelector: {matchLabels: {app: gateway}}

One item: AND - only app=gateway pods in namespaces labelled team=edge.

- from:
  - namespaceSelector: {matchLabels: {team: edge}}
  - podSelector: {matchLabels: {app: gateway}}

Two items: OR - every pod in the edge namespaces, or app=gateway pods in the policy's own namespace. It passes review and functional tests, and lets far more in.

Read policies with k describe netpol: a ---------- separator between peers is the OR. And select "this exact namespace" with the automatic kubernetes.io/metadata.name label - custom namespace labels are only as trustworthy as whoever can set them.

Also asked: Design NetworkPolicies for a three-tier app behind an ingress controller. · Why does a policy with only ingress rules not restrict egress? · How would you debug "the NetworkPolicy is blocking us"?

Practise this lesson in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.