EventBridge Pipe: Route S3 DataSync Task Events to SNS Alerts

What We Are Building

Every time an AWS DataSync task run completes, DataSync emits an event to EventBridge. Most teams either ignore those events or bolt on a Lambda function just to forward them somewhere useful. This tutorial skips the Lambda middleman entirely by using an EventBridge Pipe — a fully managed point-to-point integration that reads the DataSync event, optionally filters it, and pushes a formatted message straight into an SNS topic. The result is a near-real-time alert pipeline with zero custom runtime code to maintain.

Prerequisites

You need an AWS account with an existing DataSync task (even a simple S3-to-S3 task is fine for testing), the AWS CLI v2 configured, and permission to create EventBridge Pipes, SNS topics, and IAM roles. All resources below live in us-east-1 — adjust the region flag as needed.

Step 1 — Create the SNS Topic

This topic is the destination. Subscribers (email, Slack via HTTP endpoint, PagerDuty, etc.) attach here later.

aws sns create-topic \
  --name datasync-alerts \
  --region us-east-1

Note the TopicArn returned. It looks like arn:aws:sns:us-east-1:123456789012:datasync-alerts. Save it — you will use it in every subsequent command.

Step 2 — Create the IAM Role for the Pipe

EventBridge Pipes needs permission to read from the source (the default event bus acts as source here via a filter, so the pipe actually uses an SQS queue as a buffered source — more on that in a moment) and to publish to SNS. Create the trust policy file first.

# trust-policy.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "pipes.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

aws iam create-role \
  --role-name datasync-pipe-role \
  --assume-role-policy-document file://trust-policy.json

# Inline permission: read SQS + publish SNS
aws iam put-role-policy \
  --role-name datasync-pipe-role \
  --policy-name datasync-pipe-policy \
  --policy-document '{
    "Version":"2012-10-17",
    "Statement":[
      {
        "Effect":"Allow",
        "Action":[
          "sqs:ReceiveMessage",
          "sqs:DeleteMessage",
          "sqs:GetQueueAttributes"
        ],
        "Resource":"arn:aws:sqs:us-east-1:123456789012:datasync-events-queue"
      },
      {
        "Effect":"Allow",
        "Action":"sns:Publish",
        "Resource":"arn:aws:sns:us-east-1:123456789012:datasync-alerts"
      }
    ]
  }'

Replace the account ID 123456789012 with your own throughout.

Step 3 — Create the SQS Queue (Pipe Source)

EventBridge Pipes currently supports SQS as a polling source. We route DataSync events from the default event bus into this queue using a standard EventBridge rule, and the Pipe drains the queue and forwards to SNS. This pattern also gives you a built-in buffer if SNS is temporarily unreachable.

aws sqs create-queue \
  --queue-name datasync-events-queue \
  --region us-east-1

Note the QueueUrl and derive the ARN: arn:aws:sqs:us-east-1:123456789012:datasync-events-queue.

Step 4 — Allow EventBridge to Send Messages to the Queue

Without this queue policy the EventBridge rule will silently drop messages.

aws sqs set-queue-attributes \
  --queue-url https://sqs.us-east-1.amazonaws.com/123456789012/datasync-events-queue \
  --attributes '{
    "Policy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"events.amazonaws.com\"},\"Action\":\"sqs:SendMessage\",\"Resource\":\"arn:aws:sqs:us-east-1:123456789012:datasync-events-queue\"}]}"
  }'

Step 5 — Create the EventBridge Rule That Captures DataSync Events

DataSync publishes task execution state changes under the detail-type DataSync Task Execution State Change. The rule below captures only terminal states — SUCCESS and ERROR — so you are not flooded with intermediate progress events.

aws events put-rule \
  --name datasync-terminal-states \
  --event-pattern '{
    "source": ["aws.datasync"],
    "detail-type": ["DataSync Task Execution State Change"],
    "detail": {
      "State": ["SUCCESS", "ERROR"]
    }
  }' \
  --state ENABLED \
  --region us-east-1

aws events put-targets \
  --rule datasync-terminal-states \
  --targets '[{
    "Id": "datasync-sqs-target",
    "Arn": "arn:aws:sqs:us-east-1:123456789012:datasync-events-queue"
  }]' \
  --region us-east-1

Step 6 — Create the EventBridge Pipe

The Pipe reads from SQS, applies an input transformation to produce a human-readable SNS message body, and publishes to the topic. The InputTemplate field lets you shape the payload without any Lambda code.

aws pipes create-pipe \
  --name datasync-to-sns \
  --role-arn arn:aws:iam::123456789012:role/datasync-pipe-role \
  --source arn:aws:sqs:us-east-1:123456789012:datasync-events-queue \
  --target arn:aws:sns:us-east-1:123456789012:datasync-alerts \
  --target-parameters '{
    "InputTemplate": "DataSync task <$.body.detail.TaskExecutionArn> finished with state <$.body.detail.State> at <$.body.time>. Bytes transferred: <$.body.detail.BytesTransferred>."
  }' \
  --region us-east-1

The $.body prefix is required because SQS wraps the original EventBridge event inside a body field when it stores the message. The Pipe exposes that raw SQS record, so you must navigate one level deeper to reach the actual event fields.

Step 7 — Subscribe Your Email to the SNS Topic

aws sns subscribe \
  --topic-arn arn:aws:sns:us-east-1:123456789012:datasync-alerts \
  --protocol email \
  --notification-endpoint [email protected] \
  --region us-east-1

Confirm the subscription from your inbox before testing.

Step 8 — Test the Pipeline End to End

Start any DataSync task run and wait for it to complete, or inject a synthetic event directly into the event bus to verify the wiring without waiting for a real transfer.

aws events put-events \
  --entries '[{
    "Source": "aws.datasync",
    "DetailType": "DataSync Task Execution State Change",
    "Detail": "{\"TaskExecutionArn\":\"arn:aws:datasync:us-east-1:123456789012:task/task-01234567890abcdef/execution/exec-01234567890abcdef\",\"State\":\"SUCCESS\",\"BytesTransferred\":104857600}",
    "EventBusName": "default"
  }]' \
  --region us-east-1

Within a few seconds the SQS queue receives the message, the Pipe polls it (default batch window is up to 20 seconds), transforms the body, and SNS delivers the formatted alert to your email.

Verify the Pipe Is Running

# Check pipe state — should show RUNNING
aws pipes describe-pipe \
  --name datasync-to-sns \
  --region us-east-1 \
  --query "CurrentState"

# Check SQS for stuck messages (should be 0 after pipe drains)
aws sqs get-queue-attributes \
  --queue-url https://sqs.us-east-1.amazonaws.com/123456789012/datasync-events-queue \
  --attribute-names ApproximateNumberOfMessages \
  --region us-east-1

Cost Considerations

EventBridge Pipes charges per event processed ($0.40 per million events). For typical DataSync workloads running a handful of tasks per day the monthly cost rounds to zero. SQS standard queue pricing for this volume is similarly negligible. There is no Lambda execution cost because no Lambda exists in this stack.

Extending the Pattern

Once the Pipe is in place you can enrich it without changing the rule or the queue. Add an enrichment Lambda to the Pipe definition if you need to look up the task name from the ARN before formatting the message. Swap the SNS target for an EventBridge API destination to push directly to a Slack webhook or a PagerDuty endpoint. You can also add a filter inside the Pipe itself (separate from the EventBridge rule) to further narrow which states trigger an alert — for example, forward only ERROR events to an on-call SNS topic and SUCCESS events to a low-priority audit topic, all from the same SQS source queue using two Pipes.

official docs