Overview
Storage costs accumulate quietly. A bucket that started as a landing zone for application logs or nightly backups grows steadily, and most of that data is never accessed again after the first few days. AWS S3 lifecycle rules exist precisely to handle this pattern: you define a policy once, and S3 automatically moves objects through storage classes — from Standard to Glacier Instant Retrieval, then to Glacier, then to Deep Archive — based on object age, prefix, or tag.
Glacier Instant Retrieval costs roughly one-third of S3 Standard for storage, with millisecond retrieval. Glacier (Flexible Retrieval) drops further, and Deep Archive is the cheapest option AWS offers for long-term retention, at the cost of retrieval times measured in hours. The minimum storage duration for Glacier Instant Retrieval is 90 days; for Deep Archive it is 180 days — AWS charges you for that minimum even if you delete the object earlier, so tiering decisions need to account for that.
This tutorial covers writing a multi-rule lifecycle configuration JSON, attaching it to a bucket with the AWS CLI, and then confirming that transitions are working through object metadata and CloudWatch metrics. We will also handle the versioned-bucket case, which trips up a lot of engineers the first time.
Before You Start
Before writing a single line of JSON, make sure the following are in place.
- AWS CLI installed and configured. Version 2 is recommended. Run
aws --versionto confirm. - IAM permissions. The identity running these commands needs
s3:PutLifecycleConfiguration,s3:GetLifecycleConfiguration, ands3:GetBucketVersioning. Without all three, you will hit silent permission errors or incomplete reads. - An existing S3 bucket. Know whether versioning is enabled — this changes which rule blocks you need.
- Storage class pricing awareness. Review the AWS S3 pricing page for your region before committing to transitions. The per-request retrieval costs for Glacier tiers can outweigh storage savings if objects are accessed frequently.
- Objects larger than 128 KB. AWS does not transition objects smaller than 128 KB to any Glacier class via lifecycle rules. Those objects stay in their origin storage class regardless of what the policy says. This is a hard platform limit, not a configuration option.
Step 1: Create the Lifecycle Configuration JSON
The lifecycle policy is a JSON document that contains one or more rules. Each rule targets objects by prefix, tag, or both, and defines what actions to take at what age. We will build a configuration that covers three common real-world scenarios: tiering timestamped logs, handling noncurrent versions in a versioned backup bucket, and cleaning up incomplete multipart uploads that silently accumulate storage charges.
Save the following as ./lifecycle.json:
{
"Rules": [
{
"ID": "move-logs-to-glacier-instant",
"Status": "Enabled",
"Filter": {
"Prefix": "logs/"
},
"Transitions": [
{
"Days": 30,
"StorageClass": "GLACIER_IR"
},
{
"Days": 90,
"StorageClass": "GLACIER"
},
{
"Days": 365,
"StorageClass": "DEEP_ARCHIVE"
}
],
"Expiration": {
"Days": 1825
}
},
{
"ID": "move-backups-noncurrent-versions",
"Status": "Enabled",
"Filter": {
"Prefix": "backups/"
},
"NoncurrentVersionTransitions": [
{
"NoncurrentDays": 30,
"StorageClass": "GLACIER"
},
{
"NoncurrentDays": 180,
"StorageClass": "DEEP_ARCHIVE"
}
],
"NoncurrentVersionExpiration": {
"NoncurrentDays": 730
}
},
{
"ID": "abort-incomplete-multipart-uploads",
"Status": "Enabled",
"Filter": {
"Prefix": ""
},
"AbortIncompleteMultipartUpload": {
"DaysAfterInitiation": 7
}
}
]
}A few things worth understanding before applying this:
- The
Daysfield underTransitionsis counted from the object’s creation date, not its last-modified date. If you re-upload an object, the clock resets. - The minimum value for
Daysin any transition is 1. Setting it to 0 will cause the API call to fail with a validation error — a common mistake when copying examples from documentation. - The second rule uses
NoncurrentVersionTransitionsrather thanTransitions. On a versioned bucket, standardTransitionsrules only affect the current version. Old versions are completely ignored unless you add theNoncurrentVersion*blocks explicitly. - The multipart upload cleanup rule applies bucket-wide (empty prefix string). Orphaned multipart uploads are invisible in the console but show up on your bill. Seven days is a reasonable window for any legitimate large upload to complete.
Step 2: Apply the Lifecycle Policy to Your Bucket
With the JSON file ready, apply it to the target bucket using the put-bucket-lifecycle-configuration command. Replace BUCKET_NAME with your actual bucket name:
aws s3api put-bucket-lifecycle-configuration \
--bucket BUCKET_NAME \
--lifecycle-configuration file://lifecycle.jsonA successful call returns no output. To confirm the policy was accepted and is stored correctly, read it back immediately:
aws s3api get-bucket-lifecycle-configuration --bucket BUCKET_NAMEThe response should mirror your JSON structure. If you see all three rules with "Status": "Enabled", the policy is active. AWS evaluates lifecycle rules once per day, so transitions will not happen instantly — allow up to 24 hours for the first evaluation cycle to run.
If you prefer the console, navigate to the bucket → Management tab → Lifecycle rules → Create lifecycle rule. The console wizard maps directly to the same JSON fields, but the CLI is reproducible and scriptable, which matters when you are managing multiple buckets or want to store this configuration in version control.
Step 3: Monitor Transitions and Validate Glacier Storage
Once enough time has passed for objects to age into the first transition threshold, verify that the moves are actually happening. The most direct method is checking individual object metadata:
aws s3api head-object \
--bucket BUCKET_NAME \
--key logs/2025-01-15/app.log.gzLook for the StorageClass field in the response. If the object has transitioned, you will see GLACIER_IR, GLACIER, or DEEP_ARCHIVE rather than STANDARD. If the field is absent, the object is still in Standard.
For bucket-wide visibility, enable S3 Inventory. It produces daily or weekly CSV or Parquet reports listing every object along with its storage class, size, and last-modified date. This is the only scalable way to audit transitions across millions of objects.
CloudWatch also exposes S3 storage metrics broken down by storage class. Navigate to CloudWatch → Metrics → S3 → Storage Metrics, and look for BucketSizeBytes filtered by StorageType. Watching GlacierInstantRetrievalSizeBytes grow over time while StandardStorage shrinks is a straightforward confirmation that tiering is working.
If you need to access a Glacier or Deep Archive object before it transitions back automatically, you must issue a restore request first. The object is not directly readable until the restore completes:
aws s3api restore-object \
--bucket BUCKET_NAME \
--key logs/2025-01-15/app.log.gz \
--restore-request Days=7,GlacierJobParameters={Tier=Standard}Standard tier restores from Glacier take 3–5 hours. Deep Archive Standard restores take up to 12 hours. The Days=7 parameter controls how long the restored copy remains accessible in Standard before it reverts to Glacier storage.
Troubleshooting
Policy applied but no transitions after 48 hours. First, confirm the objects actually match the rule filter. A prefix of logs/ will not match an object stored at Logs/app.log — S3 prefixes are case-sensitive. Also verify object age: the Days counter starts from creation date. Use head-object to check LastModified and calculate whether the threshold has been crossed.
Objects smaller than 128 KB never transition. This is expected behavior and cannot be overridden. If your log files are small, consider aggregating them before upload, or accept that those objects will remain in Standard indefinitely.
Versioned bucket — only current versions are moving. Standard Transitions rules do not apply to noncurrent versions. You must add NoncurrentVersionTransitions blocks, as shown in the second rule of the configuration above. Run aws s3api get-bucket-versioning --bucket BUCKET_NAME to confirm versioning status before debugging further.
Validation error on put-bucket-lifecycle-configuration. The most frequent cause is a Days: 0 value in a transition rule. AWS requires a minimum of 1 day. Also check that transition days are in ascending order within a single rule — you cannot transition to GLACIER on day 90 and GLACIER_IR on day 30 if GLACIER_IR appears second in the array.
Restore request returns RestoreAlreadyInProgress. Another restore is already running for that object. Wait for it to complete, then retry the access. Check restore status with head-object and look for the Restore field in the response headers.
Unexpected charges for early deletion. If you delete a Glacier Instant Retrieval object before 90 days or a Deep Archive object before 180 days, AWS charges you for the remaining minimum storage duration. Review your deletion policies and expiration rules carefully against these minimums before enabling them in production.
Going Further
The configuration in this tutorial is a solid foundation, but there are several directions worth exploring depending on your use case.
S3 Intelligent-Tiering removes the need to predict access patterns entirely. Objects are monitored automatically and moved between access tiers based on actual usage. It adds a small per-object monitoring charge but eliminates the guesswork for workloads with unpredictable access. You can combine it with lifecycle rules by transitioning objects to INTELLIGENT_TIERING storage class after an initial period.
S3 Batch Operations is the right tool when you need to apply a storage class change to existing objects retroactively. Lifecycle rules only affect objects going forward from the rule’s creation date. If you have years of existing data in Standard that should already be in Glacier, Batch Operations can move it in bulk using an S3 Inventory manifest as input.
AWS Cost Explorer with S3 storage class breakdown gives you the feedback loop to validate that your tiering strategy is actually reducing spend. Filter by service and usage type to see Standard versus Glacier storage costs month over month.
Terraform-managed lifecycle rules are the production-grade approach for teams managing infrastructure as code. The aws_s3_bucket_lifecycle_configuration resource maps directly to the JSON structure used here, and storing lifecycle policies in version control means changes are reviewed, audited, and reproducible across environments. If you are already using Terraform for your S3 infrastructure, there is no good reason to manage lifecycle rules out-of-band through the CLI.
