OnCallReadyStart learning

Platform engineering · 6 min read

Backstage catalog-info.yaml validation errors, and how to fix each one

Policy check failed, must have required property 'owner', missing or empty kind, Unknown field: what each Backstage catalog error means and how to fix it.

You pushed a catalog-info.yaml, registered it, and the service never shows up in Backstage. The catalog does not reject a bad file at the door. It reads it in the background, and an entity that fails validation simply never appears. The error is attached to the location, or you get it straight back if you ask the right way.

This post fixes one file, one error at a time, with the messages exactly as Backstage 1.56 prints them. Each fix is a commit and a push, and each check is a dry run against the catalog's REST API:

terminal
$ TOKEN=$(cat ~/oncall-lab/labs/platform/backstage-token)
$ curl -s -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d '{"type":"url","target":"https://git.lab/platform/payments/-/blob/main/catalog-info.yaml"}' "http://backstage.lab:7007/api/catalog/locations?dryRun=true" | jq -r .error.message

POST /api/catalog/locations?dryRun=true reads and processes the file exactly as a registration would, and creates nothing. (The token is a static token from backend.auth.externalAccess in app-config.yaml. Calls without one are refused with a 401.)

The file we start with:

yaml
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: Payments API
  tags:
    - Java
spec:
  type: service
  lifecycle: production
  system: payments
  dependsOn:
    - payments-db

It looks reasonable, and it has four mistakes.

1. "metadata.name" is not valid

output
InputError: Policy check failed for component:default/payments api; caused by Error: "metadata.name" is not valid; expected a string that is sequences of [a-zA-Z0-9] separated by any of [-_.], at most 63 characters in total but found "Payments API". To learn more about catalog file format, visit: https://github.com/backstage/backstage/blob/master/docs/architecture-decisions/adr002-default-catalog-file-format.md

metadata.name is an identifier, like a Kubernetes object name: letters and digits separated by -, _ or ., at most 63 characters, no spaces. Notice the entity reference in the message, component:default/payments api. References are always lowercased, so Payments-API and payments-api are the same entity. The human-friendly name goes in metadata.title:

yaml
metadata:
  name: payments-api
  title: Payments API

2. "tags.0" is not valid

output
InputError: Policy check failed for component:default/payments-api; caused by Error: "tags.0" is not valid; expected a string that is sequences of [a-z0-9+#] separated by [-], at most 63 characters in total but found "Java". To learn more about catalog file format, visit: https://github.com/backstage/backstage/blob/master/docs/architecture-decisions/adr002-default-catalog-file-format.md

Tags are stricter than names: lowercase only, words joined by -, with + and # allowed (c++, c#). tags.0 is the first tag, counting from zero. Java must be java, and spring boot must be spring-boot.

Both errors so far are policy failures ("Policy check failed for ..."). Policies run first, on every entity of every kind. They check the envelope (apiVersion, kind, metadata), the field formats, and that the root has only apiVersion, kind, metadata and spec. A typo at the root fails here too:

output
InputError: Policy check failed for component:default/payments-api; caused by Error: Unknown field specs

3. /spec must have required property 'owner'

output
InputError: Processor BuiltinKindsEntityProcessor threw an error while validating the entity component:default/payments-api; caused by TypeError: /spec must have required property 'owner' - missingProperty: owner

Now the envelope is fine and the kind validation runs: each built-in kind has a JSON schema for its spec. The message is the schema validator's: a path (/spec), what is wrong, then the details (missingProperty: owner). The required fields per kind:

KindRequired in spec
Componenttype, lifecycle, owner
APItype, lifecycle, owner, definition
Resourcetype, owner
System, Domainowner
Grouptype, children (may be [])
UsermemberOf (may be [])

owner: team-payments fixes it. Short references have a default kind: for owner it is Group, so this means group:default/team-payments.

4. Entity reference had missing or empty kind

output
InputError: Processor BuiltinKindsEntityProcessor threw an error while postprocessing; caused by Error: Entity reference "payments-db" had missing or empty kind (e.g. did not start with "component:" or similar)

This one comes later, while the catalog turns spec fields into relations. Most fields have a default kind (owner is a Group, system a System, providesApis an API), so a bare name works there. dependsOn and dependencyOf do not, because they can point at a Component or a Resource. You have to write the kind:

yaml
  dependsOn:
    - resource:payments-db

Now the dry run lists what would be created, a Location for the file plus the entity:

terminal
$ curl -s -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d '{"type":"url","target":"https://git.lab/platform/payments/-/blob/main/catalog-info.yaml"}' "http://backstage.lab:7007/api/catalog/locations?dryRun=true" | jq -r '.entities[] | .kind + " " + .metadata.name'
Location generated-c1a27c313491bda2411b03813caaddaab59dff6e
Component payments-api

Register it for real (the same POST without ?dryRun=true) and read the relations back:

terminal
$ curl -s -H "Authorization: Bearer $TOKEN" http://backstage.lab:7007/api/catalog/entities/by-name/component/default/payments-api | jq -c '.relations[]'
{"type":"dependsOn","targetRef":"resource:default/payments-db"}
{"type":"ownedBy","targetRef":"group:default/team-payments"}
{"type":"partOf","targetRef":"system:default/payments"}

A relation to an entity that does not exist is not an error. If there is no Group team-payments in the catalog, the Component still appears, with an owner link that leads nowhere. That is worth checking, because ownership is the first thing anyone looks for during an incident.

Validate one entity before you push

The dry run reads from git, so the file must be pushed first. To check an entity before you commit, post it to validate-entity as JSON. The response is 200 with no body when it is valid, and 400 with every error when it is not:

terminal
$ curl -s -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d '{"location":"url:https://git.lab/platform/payments/-/blob/main/catalog-info.yaml","entity":{"apiVersion":"backstage.io/v1alpha1","kind":"Component","metadata":{"name":"payments-api"},"spec":{"type":"service","lifecycle":"production","system":"payments"}}}' http://backstage.lab:7007/api/catalog/validate-entity | jq .
{
  "errors": [
    {
      "name": "InputError",
      "message": "Processor BuiltinKindsEntityProcessor threw an error while validating the entity component:default/payments-api; caused by TypeError: /spec must have required property 'owner' - missingProperty: owner",
      "cause": {
        "name": "TypeError",
        "message": "/spec must have required property 'owner' - missingProperty: owner"
      }
    }
  ]
}

That makes a good CI step on any repository with a catalog-info.yaml.

Two errors that are not about the YAML

The catalog cannot read the file. A wrong branch (here master instead of main), a wrong path, or a file that was never pushed:

output
NotFoundError: Unable to read url, https://git.lab/platform/payments/-/blob/master/catalog-info.yaml could not be read as https://git.lab/api/v4/projects/platform%2Fpayments/repository/files/catalog-info.yaml/raw?ref=master, 404 Not Found

The second URL is the one the catalog actually fetched, through the GitLab API. Open it with the same token the catalog uses, and you will see whether the problem is the path or the permissions.

The location is already registered. That is a 409 (ConflictError):

terminal
$ curl -s -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d '{"type":"url","target":"https://git.lab/platform/payments/-/blob/main/catalog-info.yaml"}' http://backstage.lab:7007/api/catalog/locations | jq .
{
  "error": {
    "name": "ConflictError",
    "message": "Location url:https://git.lab/platform/payments/-/blob/main/catalog-info.yaml already exists"
  },
  "request": {
    "method": "POST",
    "url": "/locations"
  },
  "response": {
    "statusCode": 409
  }
}

You do not need to register it again. The catalog re-reads registered locations on its own schedule (a pull from git, like Argo CD's sync loop), and POST /api/catalog/refresh with {"entityRef":"component:default/payments-api"} asks for a refresh now.

Why the entity "never shows up"

The catalog processes entities in a loop, in this order: the policies, the kind validation, then the processors that build relations. The first failure stops that entity. If the entity is new, it never appears. If an older version is already in the catalog, that version stays and carries the error, so the catalog shows yesterday's data and nothing looks broken. A Software Template has the same trap: its catalog:register step registers the location for real first and only then dry-runs it, so a template that generates a bad catalog-info.yaml "succeeds" at creating the repository and then leaves a location whose entity never appears.

The checklist

  • metadata.name: letters and digits separated by -, _ or ., no spaces (lowercase by convention). The display name goes in title.
  • tags: lowercase, joined with -.
  • Component: type, lifecycle and owner, all three.
  • dependsOn / dependencyOf: always with a kind (resource:, component:).
  • Only apiVersion, kind, metadata and spec at the root.
  • Dry-run the location (?dryRun=true) or validate-entity before you register.

On a platform built with Crossplane, the same component often has a Database or a bucket behind it. When that request is the part that never arrives, see Crossplane resource not Ready, and function-patch-and-transform for the Composition side.

Practise it

In the Platform Engineering: Backstage & Crossplane chapter, the lesson Backstage and the software catalog covers entities, references and the REST API. In the lab Put payments in the catalog you write and register a Component, API and Resource, the drill The catalog refuses the entity gives you a different broken file every round, and the incident the new service that never shows up is the template trap above.

OnCallReady is free, with no ads and no tracking. RSS · All posts