Lesson 35.5 · AWS I: CLI, IAM, S3 & KMS · 21 min read

Output, --query, pagination and reading errors

Q: How do you get one value out of the AWS CLI into a shell variable?

Select the field with JMESPath in --query and print it without quotes or JSON with --output text, for example ARN=$(aws iam get-role --role-name app --query Role.Arn --output text). The query runs on the client after the CLI has fetched every page, so for big lists add the operation's own server-side filter (such as --prefix) too. Check the exit code: 254 means AWS answered with an error, 252 that the command line itself was wrong.

In plain words

Asking the AWS CLI a question is like asking a librarian for a list of books. By default you get the full catalogue cards for every book (JSON). You can ask for a neat table to read, or plain lines to feed into another tool. And you can say "only the titles of the red books", and the librarian filters the cards for you before handing them over (--query).

If the list is very long, the librarian brings it in batches; the CLI keeps going back for the next batch unless you tell it to stop.

Output, --query, pagination and errors

Every AWS command answers with JSON shaped exactly like the API response. On call you rarely want all of it: you want one ARN in a variable, a table of who has which key, or the reason a call failed. This lesson is the toolkit for turning responses into answers - the output formats, JMESPath queries, pagination - and for reading the CLI's errors and exit codes, which are more precise than most people realise.

Need to know: --output json|text|table|yaml changes only the rendering. --query is JMESPath (the same language az uses), evaluated by the CLI after it has fetched every page. --output text with a --query is how you feed shell loops. The CLI pages through results for you; --max-items + NextToken + --starting-token page by hand. Errors start with aws: [ERROR]: and exit with 252 (bad command line), 253 (configuration or credentials), 254 (the service said no) or 255 (anything else).

Four ways to print the same answer

$ aws iam list-users --query 'Users[?starts_with(UserName, `svc-`)]'
[
    {
        "Path": "/",
        "UserName": "svc-orders",
        "UserId": "AIDASWXVBJWYSQHFHCEBG",
        "Arn": "arn:aws:iam::111122223333:user/svc-orders",
        "CreateDate": "2026-02-14T20:00:03+00:00"
    },
    {
        "Path": "/",
        "UserName": "svc-payments",
        "UserId": "AIDALZHYHVCXLVHKUGWVM",
        "Arn": "arn:aws:iam::111122223333:user/svc-payments",
        "CreateDate": "2026-03-26T20:00:03+00:00"
    },
    {
        "Path": "/",
        "UserName": "svc-search",
        "UserId": "AIDAJWNQRKXQVDUSPZPZP",
        "Arn": "arn:aws:iam::111122223333:user/svc-search",
        "CreateDate": "2026-06-19T20:00:03+00:00"
    }
]
$ aws iam list-users --query 'Users[].[UserName,CreateDate]' --output text
ana	2026-08-23T20:00:03+00:00
ci-deploy	2026-07-24T20:00:03+00:00
learner	2026-08-23T20:00:03+00:00
mihai	2026-09-10T20:00:03+00:00
svc-orders	2026-02-14T20:00:03+00:00
svc-payments	2026-03-26T20:00:03+00:00
svc-search	2026-06-19T20:00:03+00:00
$ aws iam list-users --query 'Users[].{User:UserName,Created:CreateDate}' --output table
-----------------------------------------------
|                  ListUsers                  |
+----------------------------+----------------+
|           Created          |     User       |
+----------------------------+----------------+
|  2026-08-23T20:00:03+00:00 |  ana           |
|  2026-07-24T20:00:03+00:00 |  ci-deploy     |
|  2026-08-23T20:00:03+00:00 |  learner       |
|  2026-09-10T20:00:03+00:00 |  mihai         |
|  2026-02-14T20:00:03+00:00 |  svc-orders    |
|  2026-03-26T20:00:03+00:00 |  svc-payments  |
|  2026-06-19T20:00:03+00:00 |  svc-search    |
+----------------------------+----------------+
$ aws sts get-caller-identity --output yaml
UserId: AIDAEVXTZTVCSMBGJ2WDW
Account: '111122223333'
Arn: arn:aws:iam::111122223333:user/learner

json (the default): the response, members in the API's own order, 4-space indent.
text: tab-separated values, one row per list element - the format for cut, while read and $(...). Keys are dropped, so always pick the fields with --query.
table: for humans; the title is the API operation name (ListUsers).
yaml / yaml-stream: handy for policy documents. --output off prints nothing (the exit code still tells you whether it worked).

The default comes from the profile's output setting or AWS_DEFAULT_OUTPUT.

--query: JMESPath on the client

The expression runs on the JSON response, before it is printed:

Expression	Means
`Users[].UserName`	the UserName of every element (a projection)
`Users[0]` / `Users[-1]`	the first / last element
`Users[?UserName=='ana']`	a filter (string literals in single quotes)
``Users[?starts_with(UserName,` svc-`)]``	a function in a filter (backticks = a JSON literal)
`Users[].[UserName,Arn]`	several fields as a list per element
`Users[].{Name:UserName,Id:UserId}`	renamed fields (table and json)
`length(Users)`	count
`sort_by(Users, &CreateDate)[-1].UserName`	the newest user
`Contents[?Size >` 50000`].Key`	numbers as backtick literals

Quote the whole expression in single quotes so bash leaves the brackets, ? and * alone.

$ aws iam list-users --query 'length(Users)'
7
$ aws iam list-users --query 'sort_by(Users, &CreateDate)[-1].UserName' --output text
mihai
$ for u in $(aws iam list-users --query 'Users[].UserName' --output text); do echo "$u: $(aws iam list-access-keys --user-name $u --query 'length(AccessKeyMetadata)')"; done
ana: 0
ci-deploy: 1
learner: 1
mihai: 0
svc-orders: 1
svc-payments: 1
svc-search: 1

--query filters after the data arrived. Many APIs can filter on the server too, which is faster on big accounts and the only way on huge ones: IAM's --path-prefix, S3's --prefix and --delimiter, EC2's --filters. Use the server filter to fetch less, --query to shape what came back.

$ aws s3api list-objects-v2 --bucket oncall-lab-logs-111122223333 --prefix nginx/2026/10/08/ --query 'Contents[].[Key,Size]' --output text
nginx/2026/10/08/access-00.log.gz	81352
nginx/2026/10/08/access-04.log.gz	96081
nginx/2026/10/08/access-08.log.gz	20810
nginx/2026/10/08/access-12.log.gz	35539
nginx/2026/10/08/access-16.log.gz	50268
$ aws s3api list-objects-v2 --bucket oncall-lab-logs-111122223333 --prefix nginx/2026/10/ --delimiter / --query 'CommonPrefixes[].Prefix'
[
    "nginx/2026/10/01/",
    "nginx/2026/10/02/",
    "nginx/2026/10/03/",
    "nginx/2026/10/04/",
    "nginx/2026/10/05/",
    "nginx/2026/10/06/",
    "nginx/2026/10/07/",
    "nginx/2026/10/08/"
]

Quick check

$ aws s3api list-objects-v2 --bucket oncall-lab-logs-111122223333 --query 'length(Contents)'

The bucket holds 40 log files and S3 returns at most 1,000 keys per call. What does this print?

40
1000
null, because --query only sees the first page

Answer

40 - The CLI fetched every page (here: one) and --query ran on the merged result.

Pagination

AWS APIs return results in pages (100 users per call for IAM, 1,000 keys for S3) with a marker for the next page. The CLI follows the markers for you and merges the pages, so aws iam list-users returns everyone. Three options change that:

--max-items N: print at most N items, plus a NextToken; pass it back with --starting-token to continue;
--page-size N: the size of each underlying API call - the CLI still fetches everything, in smaller calls (the cure for calls that time out on huge buckets);
--no-paginate: only the first API page, with the service's own marker (IsTruncated, Marker, NextContinuationToken).

$ aws s3api list-objects-v2 --bucket oncall-lab-logs-111122223333 --max-items 3 --query '{Keys: Contents[].Key, Next: NextToken}'
{
    "Keys": [
        "nginx/2026/10/01/access-00.log.gz",
        "nginx/2026/10/01/access-04.log.gz",
        "nginx/2026/10/01/access-08.log.gz"
    ],
    "Next": "eyJDb250aW51YXRpb25Ub2tlbiI6IG51bGwsICJib3RvX3RydW5jYXRlX2Ftb3VudCI6IDN9"
}
$ aws s3api list-objects-v2 --bucket oncall-lab-logs-111122223333 --max-items 3 --starting-token eyJDb250aW51YXRpb25Ub2tlbiI6IG51bGwsICJib3RvX3RydW5jYXRlX2Ftb3VudCI6IDN9 --query 'Contents[].Key'
[
    "nginx/2026/10/01/access-12.log.gz",
    "nginx/2026/10/01/access-16.log.gz",
    "nginx/2026/10/02/access-00.log.gz"
]

The token is opaque - here base64 of a small JSON with the position - and belongs to the CLI, not to the service. Note that --query runs on what --max-items kept: length(...) with --max-items 3 says 3.

Reading errors

A failed command prints one line that names the error code, the API operation and the service's message (the CLI's format since 2.34):

$ aws iam get-user --user-name ghost
aws: [ERROR]: An error occurred (NoSuchEntity) when calling the GetUser operation: The user with name ghost cannot be found.
$ echo $?
254
$ aws iam create-user
aws: [ERROR]: the following arguments are required: --user-name

usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help
$ echo $?
252
$ AWS_SHARED_CREDENTIALS_FILE=/dev/null aws sts get-caller-identity
aws: [ERROR]: Unable to locate credentials. You can configure credentials by running "aws configure".
$ echo $?
253
$ aws iam list-users --max-itemz 3

usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:

  aws help
  aws <command> help
  aws <command> <subcommand> help

Unknown options: --max-itemz, 3
$ echo $?
252

The return code tells a script what kind of failure it was:

Code	Meaning
0	success
1	an `aws s3` transfer failed (one or more files)
2	an `aws s3` command skipped files it could not read
130	interrupted (Ctrl+C)
252	the command line is wrong: unknown option, missing argument, bad JSON
253	the configuration is wrong or incomplete: no credentials, no Region, unknown profile
254	the request was sent and the service returned an error (AccessDenied, NoSuchEntity...)
255	anything else (connection errors, a failed assume-role, a bad --query)

So if aws iam get-user --user-name "$u" >/dev/null 2>&1 is not "the user exists": 254 can also be AccessDenied. Read the code - or better, the message. The error format is configurable (--cli-error-format, or cli_error_format in the profile):

$ aws iam get-user --user-name ghost --cli-error-format json
{
    "Code": "NoSuchEntity",
    "Message": "The user with name ghost cannot be found."
}
$ aws iam get-user --user-name ghost --cli-error-format legacy

An error occurred (NoSuchEntity) when calling the GetUser operation: The user with name ghost cannot be found.

legacy is the format before 2.34 (no aws: [ERROR]: prefix): what most blog posts and old runbooks show.

Quick check

$ aws iam get-user --user-name ghost; echo "exit=$?"

What exit code does a NoSuchEntity error give?

exit=1
exit=252
exit=254

Answer

exit=254 - 254 = the request reached IAM and IAM answered with an error. 252 would mean the command line itself was wrong.

--debug: what the CLI actually did

--debug logs every step to stderr: the version, the arguments, the credential chain, the request and the response headers. The two lines you usually want:

$ aws sts get-caller-identity --debug 2>&1 | grep -E 'credentials|Making request' | cut -c 1-150
2026-09-22 20:00:06,106 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: env
2026-09-22 20:00:06,109 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role
2026-09-22 20:00:06,112 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role-with-web-identity
2026-09-22 20:00:06,115 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: sso
2026-09-22 20:00:06,118 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: shared-credentials-file
2026-09-22 20:00:06,121 - MainThread - botocore.credentials - INFO - Found credentials in shared credentials file: ~/.aws/credentials
2026-09-22 20:00:06,124 - MainThread - botocore.endpoint - DEBUG - Making request for OperationModel(name=GetCallerIdentity) with params: {'url_path':

Scripts: the pager, skeletons, input files

The pager. On a terminal, CLI v2 sends long output through less. In scripts, cron and CI set AWS_PAGER="" (or cli_pager = in the profile, or --no-cli-pager), or a job can hang waiting for q.
Skeletons. --generate-cli-skeleton prints every parameter of an operation as JSON; fill it in and pass it back with --cli-input-json file://params.json - handy for operations with big nested parameters.
Files as parameters. Any parameter value can come from a file: file://policy.json (text) or fileb://blob.bin (binary). Policies are almost always passed this way.

$ aws iam create-user --generate-cli-skeleton
{
    "UserName": "",
    "Path": "",
    "PermissionsBoundary": "",
    "Tags": [
        {}
    ]
}

In an interview: "How do you get one value out of the AWS CLI into a shell variable?" - "With --query to select the field and --output text to print it bare: ARN=$(aws iam get-role --role-name app --query Role.Arn --output text). The query runs on the client, after the CLI fetched every page."

You can now: pick an output format for a human or a script, write JMESPath queries with projections, filters and functions, page through results by hand, tell a 252 from a 253 from a 254, and use --debug to see which credentials and endpoint the CLI used.

Why it helps

Most real use of the CLI is in scripts: get an ARN into a variable, loop over users, count objects. Doing that with grep on JSON breaks the first time a field moves. --query with --output text gives you exactly the value, every time.

Pagination and exit codes matter just as much: a script that only sees the first page, or that treats "you typed the option wrong" the same as "AWS said no", silently does the wrong thing. Reading the error format quickly is also the fastest way to tell a typo from a permission problem.

Commands in this lesson

aws echo

FAQ

Does --query reduce what AWS sends back?

No. --query is JMESPath evaluated by the CLI on your machine after it fetched every page. To make AWS send less, use the operation's own filter parameters, such as --prefix on list-objects-v2. Use both: the server-side filter for speed, --query to shape the output.

Why does --output text print None?

Because the field you asked for is missing in that item. In text output a missing value prints as None, in JSON as null. Either filter those items out in the query (for example with a ? filter) or handle None in the script.

What do the exit codes 252, 253, 254 and 255 mean?

252: the command line could not be parsed (an unknown option, a missing argument, bad JSON). 253: the configuration or credentials are invalid. 254: the request reached AWS and the service returned an error, such as AccessDenied. 255: anything else. The s3 high-level commands also use 1 and 2. Scripts can react differently to each.

Why did my script hang waiting for input?

The CLI pipes long output through a pager (less) when it thinks it talks to a terminal. In scripts set AWS_PAGER to an empty string, or pass --no-cli-pager, or set cli_pager in the config. In this lab the job scripts export AWS_PAGER="" for that reason.

How do I page through results by hand?

Use --max-items to limit how many items the CLI prints; if there are more, the output contains a NextToken. Pass it to the next call with --starting-token. --page-size only changes how many items each underlying API call fetches, not how many you see, and --no-paginate returns just the first page from the service.

In an interview Junior

How do you get one value out of the AWS CLI into a shell variable?

Select the field with JMESPath in --query and print it without quotes or JSON with --output text, for example ARN=$(aws iam get-role --role-name app --query Role.Arn --output text). The query runs on the client after the CLI has fetched every page, so for big lists add the operation's own server-side filter (such as --prefix) too. Check the exit code: 254 means AWS answered with an error, 252 that the command line itself was wrong.

Also asked: What is the difference between --query and a server-side filter? · How does pagination work in the AWS CLI? · How do you make AWS CLI commands safe to use in scripts?

Practise this lesson in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.