The script printed nothing and exited clean. Then I added --debug and saw botocore.exceptions.NoRegionError: You must specify a region buried three lines up. The --region arg existed. I just never passed it into the Session constructor.
That cost me twenty minutes on a Friday. The fix is one word, but getting the whole thing right , profiles, paginators, log noise, CI exit codes , took a few more iterations than I’d like to admit.
build_args and the NoRegionError
The argparse setup uses formatter_class=argparse.RawDescriptionHelpFormatter so the epilog examples don’t get word-wrapped into garbage when someone runs --help.
parser = argparse.ArgumentParser(
description="List running EC2 instances",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="Examples:\n %(prog)s --profile staging --region us-east-1\n %(prog)s --all-regions",
)--profile defaults to "default" so it works without any flag, but pass --profile staging and you get a completely different credential set. That profile name goes directly into boto3.Session(profile_name=args.profile, region_name=args.region). The second argument is what I’d dropped. Without it, if the profile’s config doesn’t declare a default region, boto3 throws NoRegionError the moment you try to build a client.
--output uses argparse.FileType("w") with a default of "-", meaning stdout unless you pass a filename. Open and close are handled automatically, including the - case. I’d been doing that by hand in an earlier version and it was pointless boilerplate.
Logging setup before any boto3 call
Order matters here. logging.basicConfig has to run before boto3 does anything , otherwise the SDK’s own handlers register first and your format string gets ignored in some environments. Set the root level to DEBUG when args.debug is true, INFO otherwise. Then immediately suppress the child loggers you don’t want:
logging.basicConfig(format="%(asctime)s %(levelname)s %(message)s", level=level)
logging.getLogger("botocore").setLevel(logging.WARNING)
logging.getLogger("boto3").setLevel(logging.WARNING)Skip those two setLevel calls and running at DEBUG floods the terminal with HTTP wire logs from every single API call. Useless when you’re just trying to see which regions came back.
For structured debug output I’m using logging.debug('%s', json.dumps(result, default=str)). The default=str handles datetime fields that boto3 returns as Python objects instead of strings. Leave it out and json.dumps throws a TypeError on LaunchTime.
Paginators and the full script
describe_instances is paginated. Calling it directly and reading response["Reservations"] will silently drop instances once you’re past the first page. Use the paginator every time.
paginator = client.get_paginator("describe_instances")
for page in paginator.paginate(Filters=[{"Name": "instance-state-name", "Values": ["running"]}]):
for reservation in page["Reservations"]:
for inst in reservation["Instances"]:
...Three levels of nesting because that’s how EC2 structures it: pages contain reservations, reservations contain instances. Can’t flatten it without losing data.
Errors from boto3 get caught as BotoCoreError or ClientError, logged, then the process exits with code 1. That matters when this runs inside a CI job that needs to detect failures.
#!/usr/bin/env python3
"""List running EC2 instances across one or all regions."""
import argparse
import json
import logging
import sys
import boto3
from botocore.exceptions import BotoCoreError, ClientError
def build_args():
parser = argparse.ArgumentParser(
description="List running EC2 instances",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="Examples:\n %(prog)s --profile staging --region us-east-1\n %(prog)s --all-regions",
)
parser.add_argument("--profile", default="default", help="AWS profile name")
parser.add_argument("--region", default="us-east-1", help="AWS region")
parser.add_argument("--all-regions", action="store_true", help="Scan every region")
parser.add_argument("--output", type=argparse.FileType("w"), default="-")
parser.add_argument("--debug", action="store_true")
return parser.parse_args()
def get_regions(session):
ec2 = session.client("ec2", region_name="us-east-1")
resp = ec2.describe_regions(Filters=[{"Name": "opt-in-status", "Values": ["opt-in-not-required", "opted-in"]}])
return [r["RegionName"] for r in resp["Regions"]]
def list_instances(session, region):
client = session.client("ec2", region_name=region)
paginator = client.get_paginator("describe_instances")
instances = []
for page in paginator.paginate(Filters=[{"Name": "instance-state-name", "Values": ["running"]}]):
for reservation in page["Reservations"]:
for inst in reservation["Instances"]:
instances.append({
"id": inst["InstanceId"],
"type": inst["InstanceType"],
"region": region,
"launch": inst["LaunchTime"].isoformat(),
})
return instances
def main():
args = build_args()
level = logging.DEBUG if args.debug else logging.INFO
logging.basicConfig(format="%(asctime)s %(levelname)s %(message)s", level=level)
logging.getLogger("botocore").setLevel(logging.WARNING)
logging.getLogger("boto3").setLevel(logging.WARNING)
try:
session = boto3.Session(profile_name=args.profile, region_name=args.region)
regions = get_regions(session) if args.all_regions else [args.region]
results = []
for region in regions:
logging.debug("scanning region %s", region)
results.extend(list_instances(session, region))
logging.info("found %d running instances", len(results))
json.dump(results, args.output, indent=2)
args.output.write("\n")
except (BotoCoreError, ClientError) as exc:
logging.error("AWS error: %s", exc)
sys.exit(1)
if __name__ == "__main__":
main()When scanning all regions, get_regions pulls the list dynamically from describe_regions, filtered to regions that are either enabled by default or explicitly opted in. New regions don’t require a code change.
