OnCallReadyStart learning

Platform engineeringKubernetes · 6 min read

Crossplane function-patch-and-transform: patches, transforms and the three traps

Patch and transform in a Crossplane v2 pipeline: patch types, map and string transforms, readinessChecks, and testing it all with composition render.

Crossplane v2 removed the "native" patch and transform composition that most v1 tutorials use (it had been deprecated since v1.17). Every Composition is now a pipeline of composition functions, and the function that does what the old resources: section did is function-patch-and-transform. The ideas are the same, but they live in a different place, and a few behaviours surprise everyone once.

The examples are Crossplane v2.4, function-patch-and-transform v0.11 and the crossplane CLI v2.5. Every output below comes from crossplane composition render, which runs a Composition locally, without a cluster, and prints what Crossplane would create.

Where the patches live now

yaml
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: database-rds
spec:
  compositeTypeRef:
    apiVersion: platform.lab/v1alpha1
    kind: Database
  mode: Pipeline
  pipeline:
  - step: patch-and-transform
    functionRef:
      name: function-patch-and-transform
    input:
      apiVersion: pt.fn.crossplane.io/v1beta1
      kind: Resources
      resources:
      - name: db
        base:
          apiVersion: rds.aws.m.upbound.io/v1beta1
          kind: Instance
          spec:
            forProvider:
              engine: postgres
              engineVersion: "17"
              username: app
              allocatedStorage: 20
              autoGeneratePassword: true
              passwordSecretRef:
                key: password
                name: placeholder
              skipFinalSnapshot: true
        patches:
        - fromFieldPath: spec.size
          toFieldPath: spec.forProvider.instanceClass
          transforms:
          - type: map
            map:
              small: db.t4g.micro
              medium: db.t4g.medium
        - fromFieldPath: spec.region
          toFieldPath: spec.forProvider.region
        - fromFieldPath: spec.storageGB
          toFieldPath: spec.forProvider.allocatedStorage
        - fromFieldPath: spec.storageGB
          toFieldPath: spec.forProvider.tags.storage
          transforms:
          - type: string
            string:
              type: Format
              fmt: "%dGB"
        - type: CombineFromComposite
          combine:
            variables:
            - fromFieldPath: metadata.namespace
            - fromFieldPath: metadata.name
            strategy: string
            string:
              fmt: "%s-%s-db-password"
          toFieldPath: spec.forProvider.passwordSecretRef.name
        - type: ToCompositeFieldPath
          fromFieldPath: status.atProvider.address
          toFieldPath: status.address
  • pipeline[].input is the function's input, pt.fn.crossplane.io/v1beta1, kind: Resources. The old resources: list now lives in it.
  • Each resource has a name (db). That is the composition resource name Crossplane uses to recognise the resource on every reconcile, and the name you see in Unready resources: db. Renaming it in a Composition means "delete db, create a new one".
  • base is the starting manifest, and patches copy values between the XR and the resource.

And the request a team creates:

yaml
apiVersion: platform.lab/v1alpha1
kind: Database
metadata:
  name: orders
  namespace: team-a
spec:
  size: large
  region: eu-central-1
  storageGB: 50

The patch types

TypeCopies
FromCompositeFieldPath (the default when type is left out)an XR field -> the composed resource
ToCompositeFieldPatha composed resource field -> the XR, usually into its status
CombineFromCompositeseveral XR fields, formatted together -> the composed resource
CombineToCompositeseveral composed resource fields -> the XR
PatchSeta named group of patches defined once under patchSets:

There are also FromEnvironmentFieldPath / ToEnvironmentFieldPath and the Combine...Environment variants for the pipeline's environment. Field paths use dots and brackets: spec.template.spec.containers[0].image, metadata.labels[app.kubernetes.io/name].

The transforms

TransformDoesOn a miss
maplooks the value up in a fixed table: small -> db.t4g.microerror
matchliteral or regexp patterns, with a fallbackValuethe fallback
stringFormat (fmt: "%s-logs"), Convert (ToUpper, ToLower, ToBase64, ...), TrimPrefix, TrimSuffix, Regexp, Join, Replace
converttoType: string, int64, float64, bool, ...
mathMultiply, ClampMin, ClampMax

Trap 1: a map without the key fails the whole XR

Render the Composition above with the team's request:

terminal
$ crossplane composition render xr.yaml composition.yaml functions.yaml 2>&1 | grep -o 'pipeline step.*'
pipeline step "patch-and-transform" returned a fatal result: cannot render composed resource "db" "FromCompositeFieldPath" patch at index 0: transform at index 0 returned error: map transform could not resolve: key large is not found in map

The XRD allows size: large, but the map only knows small and medium. A map miss is fatal: nothing is composed for this XR, and in the cluster the same text shows up as the XR's Synced=False message. Read it as coordinates: resource db, patch index 0, transform index 0. Keep the XRD's enum and the map's keys in sync, or use match with a fallbackValue when a default makes sense. Add the missing key and render again:

terminal
$ sed -i 's/              medium: db.t4g.medium/              medium: db.t4g.medium\n              large: db.r7g.large/' composition.yaml
$ crossplane composition render xr.yaml composition.yaml functions.yaml | sed -n '/^spec:/,$p'
spec:
  forProvider:
    allocatedStorage: 50
    autoGeneratePassword: true
    engine: postgres
    engineVersion: "17"
    instanceClass: db.r7g.large
    passwordSecretRef:
      key: password
      name: team-a-orders-db-password
    region: eu-central-1
    skipFinalSnapshot: true
    tags:
      storage: '%!d(float64=50)GB'
    username: app

Every patch landed: the size went through the map, region and allocatedStorage were copied, and the password Secret name was combined from namespace and name. Except one.

Trap 2: numbers arrive as float64

storage: '%!d(float64=50)GB' is Go's fmt saying "you asked for an integer, I got a float". Functions receive the XR as a protobuf Struct, where every number is a float64, and the string Format transform passes the value straight to fmt.Sprintf. Use %v, or put a convert transform with toType: int64 before the string one:

terminal
$ sed -i 's/fmt: "%dGB"/fmt: "%vGB"/' composition.yaml
$ crossplane composition render xr.yaml composition.yaml functions.yaml | grep storage
      storage: 50GB

Trap 3: a typo in fromFieldPath is not an error

terminal
$ sed -i 's/fromFieldPath: spec.region/fromFieldPath: spec.regoin/' composition.yaml
$ crossplane composition render xr.yaml composition.yaml functions.yaml | grep region

Nothing, and no error either. A fromFieldPath that does not exist on the XR is skipped by default (policy.fromFieldPath: Optional), because optional fields are normal. The cost is that a typo surfaces somewhere else entirely: here the RDS Instance reaches the API server without a region and is refused (spec.forProvider.region: Required value), so the XR says Unsynced resources: db (how to debug an XR that is not Ready walks that chain). When a value must be there, say so:

yaml
        - fromFieldPath: spec.region
          toFieldPath: spec.forProvider.region
          policy:
            fromFieldPath: Required

With Required, the function leaves the new resource out of the desired state and returns a warning naming the patch, instead of creating something half-filled.

Readiness: why your XR never becomes Ready

An XR is Ready when every composed resource is ready. By default a composed resource counts as ready when its own Ready condition is True. That works for managed resources, which all have one. It never works for a Deployment:

terminal
$ crossplane resource trace app web -n team-a
NAME                                      SYNCED   READY   STATUS
App/web (team-a)                          True     False   Creating: Unready resources: deployment
├─ Deployment/web-7c202d4108f2 (team-a)   -        -       
└─ Service/web-bafc3c78a008 (team-a)      -        -       
$ kubectl get deploy -n team-a -o jsonpath='{.items[0].status.conditions[*].type}'; echo
Available Progressing

The pods run and the Service has endpoints, but a Deployment has Available and Progressing and no Ready, so the App waits forever, and so does every pipeline that waits on it. (Kubernetes itself has the same split: a pod can run without being ready for traffic.) Tell the function what ready means, per resource:

yaml
      - name: deployment
        ...
        readinessChecks:
        - type: MatchCondition
          matchCondition:
            type: Available
            status: "True"
      - name: service
        ...
        readinessChecks:
        - type: None
terminal
$ kubectl apply -f ~/oncall-lab/labs/platform/app-composition.yaml
...
composition.apiextensions.crossplane.io/app-basic configured
$ kubectl get app web -n team-a
NAME   SYNCED   READY   COMPOSITION   AGE
web    True     True    app-basic     60m
$ kubectl get app web -n team-a -o jsonpath='{.status.availableReplicas}'; echo
2

None means "ready as soon as it exists", which is right for a Service, a ConfigMap or a Secret. The other check types are MatchString, MatchInteger, MatchTrue and MatchFalse on a fieldPath, NonEmpty and MatchCondition. The last command shows the ToCompositeFieldPath patch at work: the Deployment's status.availableReplicas copied onto the App, where the team can see it.

Render before every apply

terminal
$ crossplane composition render xr.yaml composition.yaml functions.yaml | head -5
---
apiVersion: platform.lab/v1alpha1
kind: Database
metadata:
  name: orders

The third file holds the Function objects the pipeline uses, the same YAML you install them with. render needs Docker (it runs the functions in containers). Add -r to include the functions' results (warnings like the Required one), and -o (--observed-resources) with existing composed resources when you test ToCompositeFieldPath patches. In CI it turns "apply and wait for the XR to go Ready" into a diff you can review.

Most platforms put a Backstage Software Template in front of these XRs, so a developer fills a form instead of writing YAML. The template's own entity has to pass the catalog's checks: catalog-info.yaml validation errors.

Practise it

In the Platform Engineering: Backstage & Crossplane chapter, the lesson Compositions and function-patch-and-transform covers every patch and transform type. The lab An App that is never Ready is the readiness trap, the drill Write the patch the platform team asks for gives you a new patch-and-transform task each round, and The database that never comes has the silent fromFieldPath typo waiting in a live cluster.

OnCallReady is free, with no ads and no tracking. RSS · All posts