This Lambda function is triggered by S3 object events and rebuilds an index.7z archive from objects in the same folder. It skips existing archive files, creates a temporary working directory, and uploads the resulting archive back to S3.
Workflow Summary
- Read bucket/key from S3 event record
- Resolve folder prefix from incoming object key
- Download folder files to unique
/tmppath - Create 7z archive via
/opt/bin/7z - Upload as
index.7zand cleanup temporary files
Lambda Handler (Python)
import os
import boto3
import subprocess
import shutil
import logging
# Configure logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()
s3_client = boto3.client('s3')
def lambda_handler(event, context):
for record in event['Records']:
bucket = record['s3']['bucket']['name']
key = record['s3']['object']['key']
# Check folder where index.zip is located
if '/' in key:
prefix = key.rsplit('/', 1)[0] + '/'
else:
prefix = ''
# List all objects
response = s3_client.list_objects_v2(Bucket=bucket, Prefix=prefix)
if 'Contents' not in response:
logger.info(f"No objects found in {prefix}")
continue
# Unique TMP dir
work_dir = f"/tmp/game_folder_{context.aws_request_id}"
if os.path.exists(work_dir):
shutil.rmtree(work_dir)
os.makedirs(work_dir)
# Download files except index.zip, index.7z, and directories
for obj in response['Contents']:
obj_key = obj['Key']
if (obj_key.endswith("index.zip") or
obj_key.endswith("index.7z") or
obj_key.endswith('/')):
continue
relative_path = obj_key[len(prefix):]
local_file_path = os.path.join(work_dir, relative_path)
local_dir = os.path.dirname(local_file_path)
if not os.path.exists(local_dir):
os.makedirs(local_dir)
logger.info(f"Downloading s3://{bucket}/{obj_key} to {local_file_path}")
try:
s3_client.download_file(bucket, obj_key, local_file_path)
except Exception as e:
logger.error(f"Error downloading {obj_key}: {e}")
raise
# Define the path of the archive
if not os.path.exists('/opt/bin/7z'):
logger.error("7z binary not found at /opt/bin/7z")
raise FileNotFoundError("7z binary missing")
local_archive = f"/tmp/index_{context.aws_request_id}.7z"
try:
subprocess.check_call(
['/opt/bin/7z', 'a', local_archive, '.'],
cwd=work_dir
)
except subprocess.CalledProcessError as e:
logger.error(f"Error during archive creation: {e}")
raise e
# Upload the 7z archive to S3
new_key = prefix + "index.7z"
logger.info(f"Uploading archive to s3://{bucket}/{new_key}")
s3_client.upload_file(local_archive, bucket, new_key)
# Cleanup
shutil.rmtree(work_dir)
if os.path.exists(local_archive):
os.remove(local_archive)
return {"statusCode": 200, "body": "index.7z created and uploaded successfully."}
Operational Notes
- Package
7zinto a Lambda layer and verify binary path/opt/bin/7z. - Set timeout/memory high enough for archive size.
- Ensure IAM policy allows
s3:ListBucket,s3:GetObject, ands3:PutObject. - Consider object versioning for safer overwrite behavior of
index.7z.
Official Documentation
- AWS Lambda Developer Guide: https://docs.aws.amazon.com/lambda/latest/dg/welcome.html
- Amazon S3 Event Notifications: https://docs.aws.amazon.com/AmazonS3/latest/userguide/EventNotifications.html
- Boto3 S3 Client Reference: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html
