AWS offers a dizzying array of targets for EventBridge Rules, but even if you set them up through the console, the documentation for each target type can be a bit light.
SQS, being an early AWS service, can come with some extra quirks that make integrations a bit trickier, especially when it comes authorization or configuration.
In this article, I show you how to set up SQS queues and SQS FIFO queues as a target of an Event Bridge Rule using CloudFormation syntax.
NOTE: I've assumed you're familiar with EventBridge events and SQS, and just want to get up and running.
Event Bridge to SQS Queue - the simple case
There is nothing special in the EventBridge rule - just the target ARN of the Queue:
MyQueueTarget:
Type: AWS::Events::Rule
Properties:
# Configure the event pattern to filter your events (https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-event-patterns.html)
EventPattern:
source: ["com.mycompany.events"]
detailType: ["CustomerPurchase"]
Targets:
- Arn: !GetAtt MyQueue.Arn
Id: QueueTarget # set as needed
You don't need a role ARN because the permissions are handled using queue policies
Queue and Queue Policy
Define your queue - it requires no special properties:
MyQueue:
Type: AWS::SQS::Queue
and define a queue policy, which will permit EventBridge to write events to your queue:
MyQueueEventBridgePolicy:
Type: AWS::SQS::QueuePolicy
Properties:
PolicyDocument:
Version: 2012-10-17
Statement:
-
Effect: Allow
Principal: { Service: events.amazonaws.com }
Action: SQS:SendMessage
Resource: !GetAtt MyQueue.Arn
Queues:
- !Ref MyQueue
You can further restrict the policy with a condition key limiting access to the event rule that invoked the policy (you shouldn't consider this optional, otherwise you're granting access to any EventBridge rule):
Statement:
-
...
Condition:
ArnEquals: { "aws:SourceArn": !GetAtt MyQueueTarget.Arn }
Event Bridge to FIFO Queue - the complex case
Similar to the normal SQS queue case, you define your event bridge rule:
MyFifoQueueTarget:
Type: AWS::Events::Rule
Properties:
# Configure the event pattern to filter your events (https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-event-patterns.html)
EventPattern:
source: ["com.mycompany.events"]
detailType: ["CustomerPurchase"]
Targets:
- Arn: !GetAtt MyFifoQueue.Arn
Id: QueueTarget # set as needed
SqsParameters:
MessageGroupId: myMessageGroupId
Similar to SQS queues, you don't need a Role ARN, but you do need to specify a MessageGroupId. This value is a fixed string - there is currently no support to make this dependent on a value in the incoming event, which means all your messages will be in the same message group.
Queue and Queue Policy
Your queue is similar, but will need to be a FIFO queue, and have content-based deduplication turned on (this seems to mandatory to avoid silent failures):
MyFifoQueue:
Type: AWS::SQS::Queue
Properties:
FifoQueue: true
ContentBasedDeduplication: true
The queue policy is the same (see the previous section for an example).
What about encryption?
SQS supports two types of encryption at rest:
- SSE-SQS - SQS manages the encryption for you. This currently isn't available with CloudFormation, so I haven't tested its use with EventBridge
- SSE-SQS - KMS is used to perform encryption, either with a AWS-managed key or a Customer-Managed Key (CMK)
Transport encryption still relies on TLS, and IAM (via queue policies) is used to perform authorization and control access to the queue.
Using the AWS-managed key with SQS (by specifying KMSMasterKeyId: alias/aws/sqs) doesn't appear to work, and this is probably because specific access hasn't been granted to EventBridge to access the key, and probably can't be made to work because AWS managed keys do not let you edit their IAM key policy.
A customer managed key may be able to be made to work - see this FAQ for an example.
Hopefully once SSE-SQS is available in CloudFormation, it should be simple matter of enabling it so encryption can be enabled in SQS when used as an EventBridge target.
Processing messages
The records that arrive in your SQS queue will have a body of the whole EventBridge event by default (unless you configure your EventBridge rule to transform, or select part of the event, or pass a different JSON entirely using the InputTransformer/InputPath/Input properties - see this article for details on using InputTransformer).
If you process your "EventBridge over SQS" events with AWS Lambda, your input event will look something like:
{
"Records": [
{
"messageId": "9c3aeeeef-6eee-4a18-9abc-89753969a233",
"receiptHandle": "AQEBUjLMNIEkASgzMS9sSrfQdkZZa7tfH0sLH/KEXpr3P3c+Aen0WQ3x3gZoKXycvhkjU3F6vA+VwRky3JOCRxUN5o1d97y52Iqw0ID/HUflIXDoJWEUCRDUIRCONTEtbYW39EE5Fxxz+rc4YyBw5v8dlxCDtXdgdyEucUVIdQt4K+94YJdz3GTpQF2ASg0YdhRQvWQb4to+wfJLVW/C0f/cgY43zFTNHrRCSuRKBMNEUONSKntc0ubh7QHyBjNoIUC+2QXGq2ECdkBlBJ9BVDQZLXuObgjcoL1hi2XiLebTKCRT0Jo0rMEfs17giLHIFwy1ZrKeOqim",
"body": "{\"version\":\"0\",\"id\":\"5b2fcd39-5fd6-b4ef-51a6-7d5624ced5da\",\"detail-type\":\"CustomerPurchase\",\"source\":\"com.mycompany.events\",\"account\":\"3829362938291\",\"time\":\"2021-12-02T06:01:09Z\",\"region\":\"us-east-1\",\"resources\":[],\"detail\":{\"type\":\"order\",\"orderNumber\":\"2928\"}}",
"eventSource": "aws:sqs",
"eventSourceARN": "arn:aws:sqs:us-east-1:3829362938291:my-queue.fifo",
"awsRegion": "us-east-1"
}
]
}
If you don't care about the EventBridge envelope, and would prefer to receive the raw detail component of the message, the simplest input transformation is to set InputPath to $.detail.
MyFifoQueueTarget:
Type: AWS::Events::Rule
Properties:
....
Targets:
-
Id: QueueTarget # set as needed
...
InputPath: $.detail # just send the event detail
This is most useful when porting existing integrations (e.g. SNS -> SQS with raw message delivery), when you're using your own events, and/or when your event rule selects the Event Source and Event Detail Type such that you know the schema of the detail component without needing the EventBridge metadata to distinguish it.
Top comments (0)