SQS Queues as an EventBridge Rule Target (with CloudFormation)

AWS offers a dizzying array of targets for EventBridge Rules, but even if you set them up through the console, the documentation for each target type can be a bit light.

SQS, being an early AWS service, can come with some extra quirks that make integrations a bit trickier, especially when it comes authorization or configuration.

In this article, I show you how to set up SQS queues and SQS FIFO queues as a target of an Event Bridge Rule using CloudFormation syntax.

NOTE: I've assumed you're familiar with EventBridge events and SQS, and just want to get up and running.

Event Bridge to SQS Queue - the simple case

There is nothing special in the EventBridge rule - just the target ARN of the Queue:

  MyQueueTarget:
    Type: AWS::Events::Rule
    Properties:
      # Configure the event pattern to filter your events (https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-event-patterns.html)
      EventPattern:
        source: ["com.mycompany.events"]
        detailType: ["CustomerPurchase"]
      Targets:
        - Arn: !GetAtt MyQueue.Arn
          Id: QueueTarget # set as needed

You don't need a role ARN because the permissions are handled using queue policies

Queue and Queue Policy

Define your queue - it requires no special properties:

  MyQueue:
    Type: AWS::SQS::Queue

and define a queue policy, which will permit EventBridge to write events to your queue:

  MyQueueEventBridgePolicy:
    Type: AWS::SQS::QueuePolicy
    Properties:
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          -
            Effect: Allow
            Principal: { Service: events.amazonaws.com }
            Action: SQS:SendMessage
            Resource: !GetAtt MyQueue.Arn
      Queues:
        - !Ref MyQueue

You can further restrict the policy with a condition key limiting access to the event rule that invoked the policy (you shouldn't consider this optional, otherwise you're granting access to any EventBridge rule):

         Statement:
           -
            ...
            Condition:
              ArnEquals: { "aws:SourceArn": !GetAtt MyQueueTarget.Arn }

Event Bridge to FIFO Queue - the complex case

Similar to the normal SQS queue case, you define your event bridge rule:

  MyFifoQueueTarget:
    Type: AWS::Events::Rule
    Properties:
      # Configure the event pattern to filter your events (https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-event-patterns.html)
      EventPattern:
        source: ["com.mycompany.events"]
        detailType: ["CustomerPurchase"]
      Targets:
        - Arn: !GetAtt MyFifoQueue.Arn
          Id: QueueTarget # set as needed
          SqsParameters:
            MessageGroupId: myMessageGroupId

Similar to SQS queues, you don't need a Role ARN, but you do need to specify a MessageGroupId. This value is a fixed string - there is currently no support to make this dependent on a value in the incoming event, which means all your messages will be in the same message group.

Queue and Queue Policy

Your queue is similar, but will need to be a FIFO queue, and have content-based deduplication turned on (this seems to mandatory to avoid silent failures):

  MyFifoQueue:
    Type: AWS::SQS::Queue
    Properties:
      FifoQueue: true
      ContentBasedDeduplication: true

The queue policy is the same (see the previous section for an example).

What about encryption?

SQS supports two types of encryption at rest:

• SSE-SQS - SQS manages the encryption for you. This currently isn't available with CloudFormation, so I haven't tested its use with EventBridge
• SSE-SQS - KMS is used to perform encryption, either with a AWS-managed key or a Customer-Managed Key (CMK)

Transport encryption still relies on TLS, and IAM (via queue policies) is used to perform authorization and control access to the queue.

Using the AWS-managed key with SQS (by specifying KMSMasterKeyId: alias/aws/sqs) doesn't appear to work, and this is probably because specific access hasn't been granted to EventBridge to access the key, and probably can't be made to work because AWS managed keys do not let you edit their IAM key policy.

A customer managed key may be able to be made to work - see this FAQ for an example.

Hopefully once SSE-SQS is available in CloudFormation, it should be simple matter of enabling it so encryption can be enabled in SQS when used as an EventBridge target.

Processing messages

The records that arrive in your SQS queue will have a body of the whole EventBridge event by default (unless you configure your EventBridge rule to transform, or select part of the event, or pass a different JSON entirely using the InputTransformer/InputPath/Input properties - see this article for details on using InputTransformer).

If you process your "EventBridge over SQS" events with AWS Lambda, your input event will look something like:

{
    "Records": [
        {
            "messageId": "9c3aeeeef-6eee-4a18-9abc-89753969a233",
            "receiptHandle": "AQEBUjLMNIEkASgzMS9sSrfQdkZZa7tfH0sLH/KEXpr3P3c+Aen0WQ3x3gZoKXycvhkjU3F6vA+VwRky3JOCRxUN5o1d97y52Iqw0ID/HUflIXDoJWEUCRDUIRCONTEtbYW39EE5Fxxz+rc4YyBw5v8dlxCDtXdgdyEucUVIdQt4K+94YJdz3GTpQF2ASg0YdhRQvWQb4to+wfJLVW/C0f/cgY43zFTNHrRCSuRKBMNEUONSKntc0ubh7QHyBjNoIUC+2QXGq2ECdkBlBJ9BVDQZLXuObgjcoL1hi2XiLebTKCRT0Jo0rMEfs17giLHIFwy1ZrKeOqim",
            "body": "{\"version\":\"0\",\"id\":\"5b2fcd39-5fd6-b4ef-51a6-7d5624ced5da\",\"detail-type\":\"CustomerPurchase\",\"source\":\"com.mycompany.events\",\"account\":\"3829362938291\",\"time\":\"2021-12-02T06:01:09Z\",\"region\":\"us-east-1\",\"resources\":[],\"detail\":{\"type\":\"order\",\"orderNumber\":\"2928\"}}",
            "eventSource": "aws:sqs",
            "eventSourceARN": "arn:aws:sqs:us-east-1:3829362938291:my-queue.fifo",
            "awsRegion": "us-east-1"
        }
    ]
}

If you don't care about the EventBridge envelope, and would prefer to receive the raw detail component of the message, the simplest input transformation is to set InputPath to $.detail.

  MyFifoQueueTarget:
    Type: AWS::Events::Rule
    Properties:
      ....
      Targets:
        - 
          Id: QueueTarget # set as needed
          ...
          InputPath: $.detail # just send the event detail

This is most useful when porting existing integrations (e.g. SNS -> SQS with raw message delivery), when you're using your own events, and/or when your event rule selects the Event Source and Event Detail Type such that you know the schema of the detail component without needing the EventBridge metadata to distinguish it.