OnCallReadyStart learning

Infrastructure as code Lesson 42.9 · Platform Engineering: Backstage & Crossplane · 14 min read

Compositions and function-patch-and-transform

In plain words

A Composition is a recipe and the functions are the cooks. The recipe says: start from this base dish, take the size the customer asked for and turn it into the right pan, write the customer's name on the box, and when the dish is ready, tell the customer the table number. Every order goes through the same recipe, so every dish comes out the same way.

Compositions: what to build for an XR

An XRD says what a Database request looks like. A Composition says what Crossplane builds for it. In v2 every Composition is a pipeline: an ordered list of steps, each calling a function. The function receives the XR and everything already composed, and returns the desired composed resources; Crossplane applies them.

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: database-rds
spec:
  compositeTypeRef:
    apiVersion: platform.lab/v1alpha1
    kind: Database
  mode: Pipeline
  pipeline:
  - step: patch-and-transform
    functionRef:
      name: function-patch-and-transform
    input:
      apiVersion: pt.fn.crossplane.io/v1beta1
      kind: Resources
      resources:
      - name: db
        base:
          apiVersion: rds.aws.m.upbound.io/v1beta1
          kind: Instance
          spec:
            forProvider:
              region: eu-central-1
              engine: postgres
              engineVersion: "17"
              username: app
              allocatedStorage: 20
              autoGeneratePassword: true
              passwordSecretRef:
                key: password
                name: placeholder
              skipFinalSnapshot: true
        patches:
        - fromFieldPath: spec.size
          toFieldPath: spec.forProvider.instanceClass
          transforms:
          - type: map
            map:
              small: db.t4g.micro
              medium: db.t4g.medium
              large: db.r7g.large
        - fromFieldPath: spec.storageGB
          toFieldPath: spec.forProvider.allocatedStorage
        - type: CombineFromComposite
          combine:
            variables:
            - fromFieldPath: metadata.name
            strategy: string
            string:
              fmt: "%s-db-password"
          toFieldPath: spec.forProvider.passwordSecretRef.name
        - type: ToCompositeFieldPath
          fromFieldPath: status.atProvider.address
          toFieldPath: status.address
  • compositeTypeRef: the XR kind and apiVersion this Composition serves. It must match exactly - a Composition for platform.lab/v1alpha2 will never be chosen for a v1alpha1 XR.
  • mode: Pipeline is the only mode in v2 (and the default).
  • pipeline[].step names the step (it appears in errors), functionRef names an installed Function, input is whatever that function expects.

The Composition is applied by the platform team once. A team then creates XRs:

apiVersion: platform.lab/v1alpha1
kind: Database
metadata:
  name: orders
  namespace: default
spec:
  size: small

function-patch-and-transform

The function used here takes a list of resources. Each has a name (the composition resource name, used to recognise it on every later reconcile), a base (the starting manifest) and patches that copy values between the XR and the resource:

Patch typeCopies
FromCompositeFieldPath (the default)XR field -> composed resource field
ToCompositeFieldPathcomposed resource field -> XR (usually its status)
CombineFromCompositeseveral XR fields, formatted together -> composed field
PatchSeta named, reusable group of patches (patchSets: at the top of the input)

Transforms change the value on the way:

TransformExample
mapsmall -> db.t4g.micro (a key that is not in the map is an error)
matchliteral or regexp patterns with a fallbackValue (no error on a miss)
stringtype: Format with fmt: "%s-logs", or Convert (ToUpper, ToLower, ToBase64...), TrimPrefix, Regexp, Join, Replace
converttoType: string, int64, float64, bool
mathMultiply, ClampMin, ClampMax

Field paths use dots and brackets: spec.template.spec.containers[0].image, metadata.labels[app.kubernetes.io/name].

Two behaviours catch everyone at least once:

  1. A patch whose fromFieldPath does not exist is skipped silently. A typo like spec.parameters.region simply never patches anything. That is deliberate (optional fields), and it is why a typo shows up later as a strange composed resource rather than as an error. Set policy: {fromFieldPath: Required} on a patch to make a missing value hold the resource back (with a warning event) until the field exists.
  2. Values arrive as JSON types. Functions receive the XR as a protobuf Struct, where every number is a float64. A string Format transform with fmt: "%d" on a number prints %!d(float64=20); use %v, or convert to int64 first.

Readiness

The XR is Ready only when every composed resource is ready. function-patch-and- transform decides that per resource with readinessChecks. With none, a resource is ready when its own Ready condition is True - right for managed resources, wrong for plain Kubernetes objects: a Deployment has Available and Progressing conditions but no Ready, so it would never count as ready. Tell the function what "ready" means:

        readinessChecks:
        - type: MatchCondition
          matchCondition:
            type: Available
            status: "True"

Other check types: MatchString, MatchInteger, MatchTrue, MatchFalse (on a fieldPath), NonEmpty, and None (always ready; for ConfigMaps and Secrets).

Composing plain Kubernetes objects

v2 compositions can create any Kubernetes object, not just managed resources - an App XR that composes a Deployment and a Service is a few lines. Crossplane's own ServiceAccount must be allowed to manage the kind. Out of the box it may manage every provider and XR kind (the RBAC manager grants those), plus Deployments, Services, ConfigMaps, Secrets and ServiceAccounts. For anything else

  • an Ingress, a NetworkPolicy, another operator's custom resource - grant it with a

ClusterRole that aggregates into Crossplane's:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: ingress:aggregate-to-crossplane
  labels:
    rbac.crossplane.io/aggregate-to-crossplane: "true"
rules:
- apiGroups: ["networking.k8s.io"]
  resources: ["ingresses"]
  verbs: ["*"]

Without it the XR fails with the API server's RBAC error and a RoleBasedAccessControl event.

What Crossplane does with the result

On every reconcile of an XR, Crossplane:

  1. selects a Composition (once): spec.crossplane.compositionRef if set, else one matching compositionSelector labels, else any whose compositeTypeRef matches - none is cannot select Composition: no compatible Compositions found;
  2. picks the CompositionRevision (more in the lifecycle lesson);
  3. runs the pipeline;
  4. garbage collects composed resources that are no longer in the desired state;
  5. applies each desired resource with server-side apply, named <xr-name>-<hash> where the 12-character hash comes from the XR's UID and the composition resource name - stable, so the same XR always maps to the same objects;
  6. writes the resources into spec.crossplane.resourceRefs, copies the desired status onto the XR, and rolls up the conditions.

Every composed resource carries the label crossplane.io/composite: <xr-name>, the annotation crossplane.io/composition-resource-name: <name> and an owner reference to the XR (deleting the XR deletes them).

$ crossplane resource trace database orders
NAME                                        SYNCED   READY   STATUS
Database/orders (default)                   True     True    Available
└─ Instance/orders-f29bfca80b19 (default)   True     True    Available

Render before you apply

crossplane composition render runs the pipeline locally (the functions in Docker, no cluster needed) and prints the XR and the resources it would compose. It is the fastest way to test a Composition, and the way to review a change to one:

$ crossplane composition render db.yaml dbcomp.yaml fn.yaml
---
apiVersion: platform.lab/v1alpha1
kind: Database
metadata:
  name: orders
---
apiVersion: rds.aws.m.upbound.io/v1beta1
kind: Instance
metadata:
  annotations:
    crossplane.io/composition-resource-name: db
  generateName: orders-
  labels:
    crossplane.io/composite: orders
  namespace: default
  ownerReferences:
  - apiVersion: platform.lab/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: Database
    name: orders
    uid: ""
spec:
  forProvider:
    allocatedStorage: 20
    autoGeneratePassword: true
    engine: postgres
    engineVersion: "17"
    instanceClass: db.t4g.micro
    passwordSecretRef:
      key: password
      name: orders-db-password
    region: eu-central-1
    skipFinalSnapshot: true
    username: app

The third argument is a file with the Function objects the pipeline uses (the same YAML you install them with).

In an interview: a Composition is a pipeline of functions for one XR kind; function-patch-and-transform takes base resources and patches values between the XR and them, with transforms; Crossplane names, applies, garbage-collects and tracks the results, and the XR is Ready when every composed resource passes its readiness check.

Why it helps

Compositions are where the platform team's decisions live, and most composition bugs are silent: a skipped patch, a readiness check that waits forever, a kind Crossplane may not create. Knowing how function-patch-and-transform patches, transforms and decides readiness, and how Crossplane names and applies the result, is what lets you write compositions that work and review other people's changes with confidence.

On the blog

Commands in this lesson

crossplane

FAQ

What happens when a fromFieldPath does not exist?

The patch is skipped silently. That is deliberate for optional fields, and it is why a typo shows up later as a strange or invalid composed resource instead of an error. Set policy fromFieldPath Required to hold a new resource back until the field exists.

Why does my number print as %!d(float64=20)?

Functions receive the XR as a protobuf Struct, where every number is a float64. A string Format transform with %d on a float prints that error text; use %v, or convert the value to int64 first.

Why does an XR composing a Deployment never become Ready?

Without readinessChecks a resource counts as ready when its Ready condition is True. A Deployment has Available and Progressing conditions but no Ready, so it never counts. Add a MatchCondition check for Available True, and None for Services or ConfigMaps.

How are composed resources named?

The XR's name followed by twelve hex characters derived from the XR's UID and the composition resource name. The same XR always maps to the same objects, and renaming a resource in the composition gives a new object while the old one is garbage collected.

How do I test a composition without a cluster?

crossplane composition render xr.yaml composition.yaml functions.yaml runs the pipeline locally with the functions in Docker and prints the XR and the resources it would create. Add --observed-resources to simulate resources that already exist and -r to see the functions' warnings.

In an interview Mid

How does a Crossplane Composition turn a request into resources?

A Composition in Pipeline mode lists steps that call functions. With function-patch-and-transform, each resource has a name, a base manifest and patches such as FromCompositeFieldPath and ToCompositeFieldPath with transforms like map or string Format. Crossplane runs the pipeline, garbage collects resources that are no longer desired, applies the rest with server-side apply under stable names, and marks the XR Ready when every resource passes its readiness check. I test changes with crossplane composition render first.

Also asked: What does the map transform do with an unknown key? · Why would a composition fail with an RBAC error? · What is a PatchSet?

Practise this lesson in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.