Introduction
This is the third in a series of posts looking at some of the core services, building blocks and approaches that will help you build out a multi-account best practice landing zone:
- Part 1: Initial setup up of a multi-account AWS environment
- Part 2: Adding AWS SSO and controlling permissions
- Part 3: Centralising compliance and incident detection
In this post we will focus on some of the services that will provide security, compliance and incident detection, starting with CloudTrail.
The source code used is available in this GitHub repository
CloudTrail
AWS CloudTrail is an AWS service that helps you enable governance, compliance, and operational and risk auditing of your AWS account. For this demo, we are going to create an Organization Trail. This is a trail that logs all events for all AWS accounts in the organization. They are automatically applied to all member accounts.
In order to set up an organization trail, you will first need to enable CloudTrail as a trusted service in you organization, else you will get an error message like the following:
ERROR: Resource OrgTrail failed because Resource handler returned
message: "Invalid request provided: The request could not be processed
because your organization hasn't enabled CloudTrail service access.
Enable service access for CloudTrail, and then try again.
To enable trusted access, you can run the following command using a profile in the management account:
aws organizations enable-aws-service-access --service-principal cloudtrail.amazonaws.com
Or you can setup directly from within AWS Organizations in the management account by clicking on services and selecting CloudTrail, and then enable trusted access.
Enabling trusted access automatically creates a service-linked role called AWSServiceRoleForCloudTrail
role in each account. This is required for CloudTrail to successfully log events for an organization.
An organization trail can only be setup in the management account, and so we use the following OrganizationBinding
that applies only to this account:
OrganizationBindings:
OrgTrailBinding:
IncludeMasterAccount: true
We also pass in a number of parameters to the template, which include specifying a name for the S3 bucket, the organization trail and the CloudWatch log group. In addition, if we have not already done so, we store the organizationId of our AWS Organization in the organization-parameters.yml file and pass this value in along with the resource-prefix.
Parameters:
orgBucketName: !Sub '${resourcePrefix}-orgtrail-${CurrentAccount.AccountId}'
resourcePrefix: !Ref resourcePrefix
organizationId: !Ref organizationId
trailName: central-orgtrail
logGroupName: OrgTrail/org-audit-log
Now we define the cloudformation in the main template itself.
Create an S3 bucket in management account
First we need to create an S3 bucket that will receive the log files for the organization trail. We use the PublicAccessBlockConfiguration
settings to block open public access to the bucket. We use the LifecycleConfiguration
settings to only store the objects for a specific period of time. We also setup server side encryption with S3-managed keys, and enable versioning.
OrgTrailBucket:
OrganizationBinding: !Ref OrgTrailBinding
Type: AWS::S3::Bucket
DeletionPolicy: Retain
UpdateReplacePolicy: Retain
Properties:
BucketName: !Ref orgBucketName
AccessControl: Private
PublicAccessBlockConfiguration:
BlockPublicAcls: true
BlockPublicPolicy: true
IgnorePublicAcls: true
RestrictPublicBuckets: true
LifecycleConfiguration:
Rules:
- ExpirationInDays: !Ref logDeletionDays
Id: 'orgtrail-bucket-lifecycle-configuration'
Status: Enabled
BucketEncryption:
ServerSideEncryptionConfiguration:
- ServerSideEncryptionByDefault:
SSEAlgorithm: AES256
VersioningConfiguration:
Status: Enabled
This bucket needs a bucket policy that allows CloudTrail to put the log files in the bucket for the organization. The resource with organizationId
allows logging for the organization trail. It also allows logging for the specific account itself in the event the trail is changed from an organization trail to a trail for that account only. The aws:SourceArn
condition helps ensure that CloudTrail can write to the S3 bucket only for the specific trail.
- Sid: AWSCloudTrailAclCheck
Effect: Allow
Principal:
Service: cloudtrail.amazonaws.com
Action: s3:GetBucketAcl
Resource: !GetAtt OrgTrailBucket.Arn
- Sid: AWSCloudTrailWrite
Effect: Allow
Principal:
Service: cloudtrail.amazonaws.com
Action: s3:PutObject
Resource:
- !Sub '${OrgTrailBucket.Arn}/AWSLogs/${AWS::AccountId}/*'
- !Sub '${OrgTrailBucket.Arn}/AWSLogs/${organizationId}/*'
Condition:
StringEquals:
s3:x-amz-acl: bucket-owner-full-control
AWS:SourceArn: !Sub 'arn:aws:cloudtrail:eu-west-2:${AWS::AccountId}:trail/${trailName}'
We also want our organization trail to support sending the events to a CloudWatch log group. Before we set up the trail, we need to set up the CloudWatch log role and the IAM role assumed by CloudTrail to write to CloudWatch.
We start off by creating the CloudWatch log group:
OrgTrailLogGroup:
OrganizationBinding: !Ref OrgTrailBinding
Type: 'AWS::Logs::LogGroup'
Properties:
RetentionInDays: 7
LogGroupName: !Ref logGroupName
We then create the IAM role that will be assumed by CloudTrail. This allows CloudTrail to create a log stream in the log group specified above, and to deliver events to that log stream for both trails in the specific AWS account and for organization trails created in this account (the management account) that are applied to the organization with the specific organizationId.
OrgTrailLogGroupRole:
OrganizationBinding: !Ref OrgTrailBinding
Type: 'AWS::IAM::Role'
Properties:
RoleName: orgtrail-publish-to-cloudwatch-log-group
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Sid: AssumeRole1
Effect: Allow
Principal:
Service: 'cloudtrail.amazonaws.com'
Action: 'sts:AssumeRole'
Policies:
- PolicyName: 'cloudtrail-policy'
PolicyDocument:
Version: '2012-10-17'
Statement:
- Sid: AWSOrgTrailCreateLogStream
Effect: Allow
Action:
- 'logs:CreateLogStream'
- 'logs:PutLogEvents'
Resource:
- !Sub 'arn:aws:logs:eu-west-2:${AWS::AccountId}:log-group:${logGroupName}:log-stream:${AWS::AccountId}_CloudTrail_eu-west-2*'
- !Sub 'arn:aws:logs:eu-west-2:${AWS::AccountId}:log-group:${logGroupName}:log-stream:${organizationId}_*'
Finally we create the trail providing all the relevant information
OrgTrail:
OrganizationBinding: !Ref OrgTrailBinding
Type: AWS::CloudTrail::Trail
DependsOn:
- OrgTrailBucketPolicy
- OrgTrailLogGroup
- OrgTrailLogGroupRole
Properties:
CloudWatchLogsLogGroupArn: !GetAtt 'OrgTrailLogGroup.Arn'
CloudWatchLogsRoleArn: !GetAtt 'OrgTrailLogGroupRole.Arn'
EnableLogFileValidation: true
IncludeGlobalServiceEvents: true
IsLogging: true
IsMultiRegionTrail: true
IsOrganizationTrail: true
S3BucketName: !Ref OrgTrailBucket
TrailName: !Ref trailName
By default, trails created without specific event selectors will be configured to log all read and write management events, and no data events. Data events provide insights into the resource (“data plane”) operations performed on or within the resource itself. Data events are often high volume activities and include operations such as Amazon S3 object level APIs and Lambda function invoke API.
Adding the following EventSelector to the properties section would log data events for all objects in all S3 buckets in your account, with the trail logging both read and write events as well as management events.
...
Properties:
EventSelectors:
- DataResources:
- Type: AWS::S3::Object
Values:
- !Sub "arn:aws:s3:::"
IncludeManagementEvents: true
ReadWriteType: All
When the pipeline runs and the organization trail is created, a trail with the given name will be created in every AWS account that belongs to the organization. Users with the relevant permissions will be able to see this trail in their member accounts, and will be able to view the event history directly in CloudTrail for their account. However, they will not be able to remove or modify the trail in any way. Any attempt to do so will show an error message like the one shown in the console below:
By default, the log files delivered by CloudTrail to your bucket are encrypted by Amazon server-side encryption with Amazon S3-managed encryption keys (SSE-S3). To provide a security layer that is directly manageable, you can instead use server-side encryption with AWS KMS–managed keys (SSE-KMS) for your CloudTrail log files, but that is currently outside the scope of this blog post.
CloudWatch Alarms
In our second post, we showed how to setup a cross-account role, so that a user in the IncidentResponse group in the Security account could jump across into a production account to investigate in the case of an incident. To add an additional level of security and audit, we will setup a CloudWatch alarm to alert us whenever the elevated role has been assumed.
To start off with we define a CloudWatch alarm that will exist in the management account with the centralised CloudWatch logs from CloudTrail. We specify the name of the metric associated with the alarm. Statistics are metric data aggregations over specified periods of time. For this alarm, we are simply summing the values of all the data points over a time period of 10 seconds. We are evaluating only over 1 period. This means that if one incident occurs in a 10 second period, the alarm will be triggered.
RoleAlarm:
Type: AWS::CloudWatch::Alarm
OrganizationBinding: !Ref OrgTrailBinding
Properties:
AlarmName: 'Security switched to Elevated Role in Prod'
AlarmDescription: 'Alarm on usage of elevated role in the Prod account'
MetricName: !Sub '${resourcePrefix}-switch-elevated-count'
Namespace: OrgTrailMetrics
Statistic: Sum
Period: 10
EvaluationPeriods: 1
Threshold: 1
TreatMissingData: notBreaching
AlarmActions:
- !Ref AlarmNotificationTopic
ComparisonOperator: GreaterThanOrEqualToThreshold
Then we define the MetricFilter. This filter searches the CloudWatch log group for any events where the event name is 'SwitchRole' and the assumed role is the 'elevated-security-role'
ProductionSupportRoleLoginsFilter:
Type: AWS::Logs::MetricFilter
OrganizationBinding: !Ref OrgTrailBinding
Properties:
LogGroupName: !Ref logGroupName
FilterPattern: '{($.eventName = "SwitchRole") && ($.userIdentity.arn = "arn:aws:sts::*:assumed-role/elevated-security-role/*") }'
MetricTransformations:
- MetricValue: '1'
MetricNamespace: OrgTrailMetrics
MetricName: !Sub '${resourcePrefix}-switch-elevated-count'
Finally, we define the SNS topic where the notification will be sent if the alarm is triggered. This is setup to send an email to our root email address.
AlarmNotificationTopic:
Type: AWS::SNS::Topic
OrganizationBinding: !Ref OrgTrailBinding
Properties:
DisplayName: !Sub 'Notifies when alarm on usage of elevated role goes off'
TopicName: !Sub '${resourcePrefix}-switch-elevatedrole-alarm-notification'
Subscription:
- Endpoint: !GetAtt MasterAccount.RootEmail
Protocol: email
We can now log into the Security account as a user in the IncidentResponse group, and switch role in the console to the elevated security role in the Production account. This will have the result of the triggering our alarm which we can see in the CloudWatch Alarm console:
And when we see that this is in alarm, we should also receive an email notifying us of the use of the elevated security role in production.
At this point we could use CloudTrail to view all the actions that were carried out by the user, or take some other action. This gives you an idea of the capability that exists using CloudTrail. Now we will move onto AWS Config.
AWS Config
AWS Config is a service that enables you to continually assess, audit, and evaluate the configurations of your AWS resources. This includes how the resources are related to one another, and how they were configured in the past and changed over time.
To start off with, we create an S3 bucket in the Log Archive account, and attach a bucket policy to it that allows the Config service to put objects into the bucket as shown here. To enable AWS Config, we must create a configuration recorder and a delivery channel. AWS Config uses the delivery channel to deliver the configuration changes to your Amazon S3 bucket
The configuration record describes the AWS resource types we want to record configuration changes for. In the recording group, we specify that AWS Config will record configuration changes for every supported type of regional resource. It will also include all supported types of global resources, such as IAM.
ConfigurationRecorder:
Type: 'AWS::Config::ConfigurationRecorder'
Properties:
RecordingGroup:
AllSupported: true
IncludeGlobalResourceTypes: true
RoleARN: !GetAtt ConfigurationRecorderRole.Arn
The delivery channel is used to deliver configuration information to our S3 bucket (it also supports SNS). We set it up to delivery configuration snapshots every hour to the S3 bucket.
DeliveryChannel:
Type: 'AWS::Config::DeliveryChannel'
Properties:
ConfigSnapshotDeliveryProperties:
DeliveryFrequency: One_Hour
S3BucketName: !Ref ConfigAuditBucket
We also configure the IAM role that Config will assume. This includes the AWSConfigRole
AWS managed policy, which will ensure that Config will have the right permissions to get configuration details whenever a new AWS resource type is supported. The policy also allows Config to write the details to the S3 bucket
ConfigurationRecorderRole:
Type: 'AWS::IAM::Role'
Properties:
ManagedPolicyArns:
- 'arn:aws:iam::aws:policy/service-role/AWSConfigRole'
AssumeRolePolicyDocument:
Version: '2012-10-17'
Statement:
- Sid: AssumeRole1
Effect: Allow
Principal:
Service: 'config.amazonaws.com'
Action: 'sts:AssumeRole'
Policies:
- PolicyName: 's3-policy'
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action: 's3:PutObject'
Resource: !Sub '${ConfigAuditBucket.Arn}/*'
Condition:
StringLike:
's3:x-amz-acl': 'bucket-owner-full-control'
- Effect: Allow
Action: 's3:GetBucketAcl'
Resource: !GetAtt ConfigAuditBucket.Arn
We can push the changes to deploy the pipeline, and now AWS Config will be enable and rolled out across all accounts in our organizations. You can go into any account and view the resources through a dashboard. However, there are currently no compliance checks as we have not defined any rules. So let's go and do that.
AWS Config Managed Rules
AWS Config provides AWS managed rules, which are predefined, customizable rules that AWS Config uses to evaluate whether your AWS resources comply with common best practices. After you activate a rule, AWS Config compares your resources to the conditions of the rule. After this initial evaluation, AWS Config continues to run evaluations each time one is triggered.
We will setup compliance with an AWS managed rule to check if the incoming SSH traffic for a security group is accessible. You can find here the list of AWS Config Managed Rules. We will use the restricted-ssh
rule which has the Identifier of INCOMING_SSH_DISABLED
. This is setup using cloudformation in the template below. It is rolled out to all accounts.
SSHOrganizationConfigRule:
Type: "AWS::Config::OrganizationConfigRule"
Properties:
OrganizationConfigRuleName: "OrganizationRestrictedSSH"
OrganizationManagedRuleMetadata:
RuleIdentifier: "INCOMING_SSH_DISABLED"
Description: "restricted-ssh"
Having to setup lots of individual managed rules can be tedious and error prone, so AWS also provide conformance packs, which we will take a look at now.
AWS Config Conformance Pack
A conformance pack is a collection of AWS Config rules and remediation actions that can be easily deployed as a single entity in an account and a Region or across an organization in AWS Organizations. Conformance packs are created by authoring a YAML template that contains the list of AWS Config managed or custom rules and remediation actions.
AWS provide a set of sample templates for conformance packs. We will use the 'Operational Best Practices for Security, Identity and Compliance Services'. The template is available in this GitHub repo
The steps to enable a conformance pack is straightforward. Firstly, we invoke a template that creates an S3 bucket in the management account.
CompliancePackBucket:
Type: AWS::S3::Bucket
DeletionPolicy: Retain
UpdateReplacePolicy: Retain
Properties:
BucketName: !Ref bucketName
AccessControl: Private
PublicAccessBlockConfiguration:
BlockPublicAcls: true
BlockPublicPolicy: true
IgnorePublicAcls: true
RestrictPublicBuckets: true
BucketEncryption:
ServerSideEncryptionConfiguration:
- ServerSideEncryptionByDefault:
SSEAlgorithm: AES256
We then copy the conformance pack from GitHub to a local yml file. Next we run a task to copy this file to the S3 bucket.
CopyToS3:
Type: copy-to-s3
DependsOn: ConfigCompliancePackBucket
LocalPath: ./Operational-Best-Practices-for-Security-Services.yml
RemotePath: !Sub 's3://${resourcePrefix}-conformance-pack/security-services.yml'
OrganizationBinding:
IncludeMasterAccount: true
Region: eu-west-2
Finally, we invoke a template that uses a OrganizationConformancePack
resource to deploy the template in the S3 bucket to the organization.
OrganizationConformancePack:
Type: AWS::Config::OrganizationConformancePack
Properties:
OrganizationConformancePackName: SecurityServices
TemplateS3Uri: !Ref templateURI
Once deployed, we have much richer information available to us on the compliance status of our resources.
AWS Config Aggregator
An aggregator is an AWS Config resource type that can collect AWS Config configuration and compliance data from an organization in AWS Organizations. This will allow us to centralise all Config findings in our Security account.
To setup an aggregator, we first have to go into AWS Organizations in the management account, click on services, select Config, and then enable trusted access.
Next we define the Security Account as the delegated administrator for Config using the following command in a terminal window:
aws organizations register-delegated-administrator --account-id <security-account-id> --service-principal config.amazonaws.com
We then define an IAM role to be assumed by the Config service. This role must have the AWSConfigRoleForOrganizations
policy attached. This role is targeted at the Security account:
ConfigAggregatorRole:
Type: AWS::IAM::Role
OrganizationBinding: !Ref SecurityBinding
Properties:
RoleName: !Sub '${resourcePrefix}-configaggregator-role'
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service: config.amazonaws.com
Action:
- sts:AssumeRole
Path: /
ManagedPolicyArns:
- 'arn:aws:iam::aws:policy/service-role/AWSConfigRoleForOrganizations'
Next we define the aggregator itself against the Security account. We use the OrganizationAggregationSource
property to define that this is for an organization. In our case, we are just going to collate data for a single region.
ConfigAggregator:
Type: AWS::Config::ConfigurationAggregator
OrganizationBinding: !Ref SecurityBinding
DependsOn: ConfigAggregatorRole
Properties:
OrganizationAggregationSource:
RoleArn: !GetAtt ConfigAggregatorRole.Arn
AwsRegions:
- !Ref aggregatorRegion
AllAwsRegions: false
ConfigurationAggregatorName: OrgConfigAggregator
Once deployed, the aggregator will appear in the Security account and start collecting the config data from other accounts. Now starts the hard work of ensuring your resources are compliant.
One of the rules that is marked as non-compliant is GuardDuty not being enabled, so that is what we will look at now.
Amazon GuardDuty
Amazon GuardDuty is a threat detection service that continuously monitors your AWS accounts and workloads for malicious activity and delivers detailed security findings for visibility and remediation.
We start off by configuring a new detector (an object that represents the GuardDuty service) which is required for GuardDuty to become operational. This is deployed to all accounts in the organization. The detector is setup to be enabled on creation, and will export updated findings every 15 minutes.
Detector:
Type: AWS::GuardDuty::Detector
OrganizationBinding: !Ref GuardDutyAllBinding
Properties:
Enable: true
FindingPublishingFrequency: FIFTEEN_MINUTES
Then we setup the AWS::GuardDuty::Master
resource in each GuardDuty member account to accept an invitation from the GuardDuty administrator account, which is designated as the Security account.
Master:
DependsOnAccount: !Ref SecurityAccount
Type: AWS::GuardDuty::Master
OrganizationBinding: !Ref GuardDutyMemberBinding
Properties:
DetectorId: !Ref Detector
MasterId: !Ref SecurityAccount
Finally, we setup the AWS::GuardDuty::Member
resource to add an AWS account as a GuardDuty member account to the GuardDuty administrator account. This is only deployed to the Security account, but loops through for all other AWS accounts and passes in their account IDs to be added as members.
Member:
Type: AWS::GuardDuty::Member
OrganizationBinding: !Ref GuardDutyMasterBinding
ForeachAccount: !Ref GuardDutyMemberBinding
Properties:
DetectorId: !Ref Detector
Email: !GetAtt CurrentAccount.RootEmail
MemberId: !Ref CurrentAccount
Status: Invited
DisableEmailNotification: true
Once deployed, we can go and look in the Accounts section of the GuardDuty console in the Security account, and we will see all other member accounts listed. GuardDuty findings are automatically sent to CloudWatch Events. Now we will look at how to send a simple notification when an incident takes place. We start off by specifying an event rule on the default Amazon EventBridge.
FindingRule:
Type: AWS::Events::Rule
DependsOn: FindingsTopicPolicy
OrganizationBinding: !Ref GuardDutyMasterBinding
Properties:
Name: !Sub '${resourcePrefix}-guardduty-findings-rule'
EventPattern:
source:
- aws.guardduty
Targets:
- Id: FindingsTopic
Arn: !Ref FindingsTopic
State: ENABLED
The rule above looks for any event that is published from the GuardDuty service. When a match is found, it will push an event onto an SNS topic. You can have fun looking at different rules with event patterns in EventBridge including using sample GuardDuty findings. You could use the pattern below to trigger a notification for a specific finding type:
EventPattern:
source:
- aws.guardduty
detail:
type:
- UnauthorizedAccess:EC2/MaliciousIPCaller.Custom
You could use a pattern like the one below to trigger a notification if the severity of the finding is above a certain threshold.
EventPattern:
source:
- aws.guardduty
detail.severity:
- numeric:
- ">"
- 6
Finally, we define an SNS topic in the cloudformation template, which will be used to trigger an email notification when a finding is received.
FindingsTopic:
Type: AWS::SNS::Topic
OrganizationBinding: !Ref GuardDutyMasterBinding
Properties:
DisplayName: GuardDuty Findings
TopicName: !Sub '${resourcePrefix}-guardduty-findings-notification'
Subscription:
- Protocol: email
Endpoint: !GetAtt SecurityAccount.RootEmail
We can test this out by logging into one of the AWS accounts using the root email address. This is something that should be avoided, and will trigger a GuardDuty finding for RootCredentialUsage.
This post has touched on a number of AWS services that help with audit and compliance as well as incident detection and response. It is a very broad topic with powerful features available. In the next post, we will start to look at budgets and the world of FinOps and sustainability using the Cost and Usage Reports.
Top comments (0)