OnCallReadyStart learning

AWS Lesson 38.19 · AWS V: Data, Resilience & the Edge · 14 min read

S3 storage classes, lifecycle and Glacier restores

In plain words

Storage classes are like where you keep things at home. The desk drawer is Standard: instant and the priciest space. The cupboard is Infrequent Access: cheaper, but each time you take something out costs a little. The attic is Glacier: you must fetch the ladder first, which takes minutes or hours. The storage unit across town is Deep Archive: the cheapest, and it takes half a day to get anything back.

The further away, the cheaper the rent, but every place has a minimum rental period, and you pay each time you fetch something.

S3 storage classes, lifecycle and Glacier restores

Every S3 object has a storage class, and the class decides three things: what a gigabyte costs per month, what a read costs, and how fast you can get the data back. Standard is right for data that is read often. A backup read once a year in an emergency is cheapest in Glacier Deep Archive, and it takes twelve hours to come back. That delay is part of your RTO. AWS I set up versioning and a first lifecycle rule; this lesson picks classes by access pattern, reads what a lifecycle did overnight, and restores an archived object.

Need to know: all classes keep 11 nines of durability across at least three AZs, except the One Zone ones. Cheaper classes charge per GB retrieved and have a minimum storage duration (30, 90 or 180 days) and, for IA, a minimum billable size of 128 KB. Glacier Flexible Retrieval and Deep Archive objects cannot be read until restore-object makes a temporary copy. Lifecycle rules move or expire objects once a day, asynchronously.

The classes, by access pattern

classforfirst byteminimum durationnotes
STANDARDhot datams-the default
INTELLIGENT_TIERINGunknown or changing patternsms-moves objects between tiers by access, small monitoring fee per object
STANDARD_IAread rarely, needed fastms30 daysretrieval fee per GB, 128 KB minimum size
ONEZONE_IAre-creatable copiesms30 daysone AZ: lost if the AZ is lost
GLACIER_IR (Instant Retrieval)archives read a few times a yearms90 dayshigher retrieval fee
GLACIER (Flexible Retrieval)backups, archivesminutes to hours (restore)90 daysExpedited 1-5 min, Standard 3-5 h, Bulk 5-12 h
DEEP_ARCHIVEcompliance archives, last copies12-48 h (restore)180 daysthe cheapest storage on AWS

The trap in this table is the minimums. Moving millions of 10 KB objects to Standard-IA costs more than leaving them, because each is billed as 128 KB. Deleting a Deep Archive object after a month is billed as 180 days.

What the lifecycle did last night

The lab bucket oncall-lab-archive-111122223333 holds exports, logs and invoices of various ages, with a lifecycle configuration:

$ aws s3api get-bucket-lifecycle-configuration --bucket oncall-lab-archive-111122223333 --query 'Rules[].{id: ID, prefix: Filter.Prefix, transitions: Transitions[].[Days,StorageClass], expire: Expiration.Days}'
[
    {
        "id": "exports-to-ia",
        "prefix": "exports/",
        "transitions": [
            [
                30,
                "STANDARD_IA"
            ]
        ],
        "expire": null
    },
    {
        "id": "logs-archive-and-expire",
        "prefix": "logs/",
        "transitions": [
            [
                90,
                "GLACIER"
            ]
        ],
        "expire": 400
    },
    {
        "id": "invoices-deep-archive",
        "prefix": "invoices/",
        "transitions": [
            [
                365,
                "DEEP_ARCHIVE"
            ]
        ],
        "expire": null
    }
]
$ aws s3api list-objects-v2 --bucket oncall-lab-archive-111122223333 --query 'Contents[].[Key,StorageClass,LastModified]' --output text
exports/2026-08-orders.csv	STANDARD_IA	2026-08-08T20:00:03+00:00
exports/2026-09-orders.csv	STANDARD	2026-09-12T20:00:03+00:00
invoices/2025-08/INV-88231.pdf	DEEP_ARCHIVE	2025-08-28T20:00:03+00:00
invoices/2026-08/INV-90412.pdf	STANDARD	2026-08-28T20:00:03+00:00
logs/2026-05-nginx.log.gz	GLACIER	2026-05-05T20:00:03+00:00
logs/2026-08-nginx.log.gz	STANDARD	2026-08-23T20:00:03+00:00

Each rule matched by prefix and moved objects older than its days: exports to Standard-IA after 30 days, logs to Glacier after 90 and deleted after 400, invoices to Deep Archive after 365 (kept for ten years by law). Lifecycle runs once a day, so a new rule shows its effect the next morning, not when you save it.

Reading an archived object

$ aws s3api get-object --bucket oncall-lab-archive-111122223333 --key exports/2026-08-orders.csv /tmp/export.csv --query '[StorageClass,ContentLength]' --output text
STANDARD_IA	48
$ aws s3api get-object --bucket oncall-lab-archive-111122223333 --key logs/2026-05-nginx.log.gz /tmp/old.log.gz
aws: [ERROR]: An error occurred (InvalidObjectState) when calling the GetObject operation: The operation is not valid for the object's storage class

Additional error details:
StorageClass: GLACIER
$ aws s3api head-object --bucket oncall-lab-archive-111122223333 --key logs/2026-05-nginx.log.gz --query '[StorageClass,Restore]' --output text
GLACIER	None

Standard-IA reads like Standard (a retrieval fee is added to the bill). A Glacier object refuses with InvalidObjectState: the bytes are not online. restore-object asks for a temporary copy, for a number of days, at a retrieval tier:

$ aws s3api restore-object --bucket oncall-lab-archive-111122223333 --key logs/2026-05-nginx.log.gz --restore-request 'Days=2,GlacierJobParameters={Tier=Expedited}'
$ aws s3api head-object --bucket oncall-lab-archive-111122223333 --key logs/2026-05-nginx.log.gz --query '[StorageClass,Restore]' --output text
GLACIER	ongoing-request="true"
$ sleep 190
$ aws s3api head-object --bucket oncall-lab-archive-111122223333 --key logs/2026-05-nginx.log.gz --query '[StorageClass,Restore]' --output text
GLACIER	ongoing-request="false", expiry-date="Fri, 25 Sep 2026 00:00:00 GMT"
$ aws s3api get-object --bucket oncall-lab-archive-111122223333 --key logs/2026-05-nginx.log.gz /tmp/old.log.gz --query StorageClass --output text
GLACIER

While ongoing-request="true" the object is still unreadable. Afterwards the Restore header shows the expiry date of the temporary copy (the archived object itself stays in Glacier; you pay for both copies while the restore lasts). Tiers: Expedited (1-5 minutes, Flexible Retrieval only, expensive), Standard (3-5 hours; 12 hours for Deep Archive), Bulk (5-12 hours; 48 for Deep Archive, cheapest). If a recovery plan says "restore yesterday's logs from Deep Archive within an hour", the plan is wrong.

Choosing a class (and when not to)

$ aws s3api put-object --bucket oncall-lab-archive-111122223333 --key try/thumb-small.png --body /etc/hostname --storage-class STANDARD_IA --query 'ETag' --output text
"498513bec1338d638e88c971592ec34b"
$ aws s3api put-bucket-lifecycle-configuration --bucket oncall-lab-archive-111122223333 --lifecycle-configuration file://$HOME/oncall-lab/labs/aws/try5/lifecycle.json
{
    "TransitionDefaultMinimumObjectSize": "all_storage_classes_128K"
}
$ aws s3api get-bucket-lifecycle-configuration --bucket oncall-lab-archive-111122223333 --query 'Rules[].ID' --output text
exports-to-ia	logs-archive-and-expire	invoices-deep-archive	uploads-intelligent-tiering	abort-incomplete-mpu

put-bucket-lifecycle-configuration replaces the whole configuration: the file must contain every rule you want to keep. That is a classic way to lose the log-expiry rule while adding another. The lab file adds Intelligent-Tiering for the uploads/ prefix and an AbortIncompleteMultipartUpload rule. Failed multipart uploads leave invisible parts that you pay for until something aborts them.

Rules of thumb:

  • unknown or changing access: Intelligent-Tiering (no retrieval fees, a small monitoring fee per object above 128 KB);
  • known pattern: lifecycle transitions by age (Standard -> IA after 30 days -> Glacier after 90...);
  • many tiny objects: leave them in Standard, or bundle them first;
  • data you can re-create (thumbnails, replicas): One Zone-IA;
  • versioned buckets: add noncurrent version transitions and expirations, or old versions grow forever.

Big objects and long distances: multipart upload (the CLI does it automatically above 8 MB, in parallel, and a failed part is retried alone) and S3 Transfer Acceleration (uploads enter AWS at the nearest CloudFront edge location: put-bucket-accelerate-configuration, then the <bucket>.s3-accelerate.amazonaws.com endpoint). DataSync moves large data sets from on premises or between storage services, and Storage Gateway gives on-premises servers file, volume or tape interfaces backed by S3.

On the exam (SAA-C03 3.1 and 4.1, SOA-C03 1.3, CLF-C02 3.6): "unknown or changing access patterns" = Intelligent-Tiering; "rarely accessed, needed in milliseconds" = Standard-IA (or Glacier Instant Retrieval if quarterly); "archive, retrieval within hours" = Glacier Flexible Retrieval; "cheapest, retrieval within 12-48 hours, kept for years" = Deep Archive; "re-creatable data, cheapest fast access" = One Zone-IA. "Faster uploads from far away" = Transfer Acceleration; "large files" = multipart upload. Lifecycle cannot move objects to a class and back up again.

In an interview: "How would you cut an S3 bill of 200 TB?" - "Find what is in it first: S3 Storage Lens or an inventory report, grouped by prefix, age and last access. Then expire what nobody needs (old versions, incomplete multipart uploads, logs past retention), move cold data with lifecycle rules to IA or Glacier where the object sizes and minimum durations make it cheaper, and use Intelligent-Tiering where nobody knows the pattern. Check that restore times still meet the recovery requirements."

You can now: pick a storage class from an access pattern (and see when a cheaper class costs more), read what lifecycle rules did, restore an object from Glacier with the right tier and read its Restore header, and avoid the classic lifecycle mistakes.

Why it helps

The storage class decides what a gigabyte costs, what a read costs and how fast the data comes back, and the traps are in the small print: minimum storage durations, a 128 KB minimum billable size, retrieval fees, and restore times that become part of your RTO.

Lifecycle rules do the moving for you once a day, and one careless put-bucket-lifecycle-configuration can drop the rule that expires your logs. Knowing all this is the difference between a cheaper bill and a surprise one.

Commands in this lesson

aws sleep

FAQ

Why does get-object fail with InvalidObjectState?

The object is in Glacier Flexible Retrieval or Deep Archive, and its bytes are not online. restore-object asks for a temporary copy for a number of days at a retrieval tier. head-object shows ongoing-request="true" while it runs, and the expiry date of the copy afterwards; get-object works once the restore is done.

How long does a Glacier restore take?

Expedited takes 1 to 5 minutes and exists only for Flexible Retrieval. Standard takes 3 to 5 hours, or 12 hours for Deep Archive. Bulk takes 5 to 12 hours, or up to 48 for Deep Archive, and is the cheapest. While the restore lasts you pay for both copies. A plan that restores from Deep Archive within an hour is wrong.

When does a new lifecycle rule take effect?

Lifecycle runs once a day, asynchronously, so a new rule shows its effect the next morning rather than when you save it. Rules match by prefix or tags, transition objects older than their days and can expire them. Lifecycle cannot move objects to a class and back up again.

Why did adding a lifecycle rule remove my other rules?

put-bucket-lifecycle-configuration replaces the whole configuration. The file you send must contain every rule you want to keep, or the log-expiry rule quietly disappears. Read the current rules with get-bucket-lifecycle-configuration first, and add AbortIncompleteMultipartUpload while you are there: failed uploads leave invisible parts you pay for.

When is Intelligent-Tiering the right choice?

When nobody knows the access pattern, or it changes. It moves objects between tiers by access, with no retrieval fees and a small monitoring fee per object above 128 KB. When the pattern is known, plain lifecycle transitions by age are cheaper, and many tiny objects are best left in Standard or bundled first.

In an interview Mid

How would you cut an S3 bill of 200 TB?

Find out what is in it first: S3 Storage Lens or an inventory report, grouped by prefix, age and last access. Then expire what nobody needs: old noncurrent versions, incomplete multipart uploads (AbortIncompleteMultipartUpload), logs past their retention. Move cold data with lifecycle rules to IA or Glacier where object sizes and minimum durations make it cheaper, and use Intelligent-Tiering where nobody knows the pattern. Finally, check that the restore times of the new classes still meet the recovery requirements.

Also asked: What is the difference between S3 Standard-IA and One Zone-IA? · How do you read an object that lifecycle moved to Glacier Deep Archive? · Why can moving small objects to Standard-IA cost more than leaving them?

Practise this lesson in the terminal Free, in your browser - a real Ubuntu terminal to try it in, with missions that check your work.