🧽 Cleaning up Security Hub with AWS Resource Explorer 🫧

Config and Security Hub are probably one of the most used services to get an overview over the compliance of your resources and your overall security store.

Config lets you run predefined or custom rules over your AWS resources to check whether they are compliant or not.
Security Hub uses different Security Standards including predefined Severity categories to categorize findings and providing an overall overview over your Security status. Since some time already Config Results are automatically transferred to Security Hub giving you the possibility to just check one tool for your current security status.

Config and Security Hub as Mess-Makers

As AWS environments are hardly static in its behavior, a lot of resources will be removed, created or modified making the environment being in a constant state of change. If a resource which has been scanned by Config gets deleted, it will result in Security Hub in a NOT_AVAILABLE finding. The default view in Security Hub will still show you this finding as the findings Record State is still set to active.
If AWS realizes that the state of these NOT_AVAILABLE findings didn't change for over 90 days, they will get archived automatically, quite a long time period right?

Everyone working with the Security Hub in bigger environments using several Config Rules and Security Standards will know how overwhelming it feels like opening the Security Hub console feeling like you can never bear this amount of findings. But as written above not all of these findings belong to still existent findings.

If you're orderly like me, you would rather prefer a solution which will automatically check the existence resources with the NOT_AVAILABLE state and resolve the findings belonging to resources that are already deleted.

So lets dive into an early Spring Cleaning and clean up this mess with a simple lambda function!

What is the Resource Explorer?

This solution is leveraging the Resource Explorer services and is a prerequisite to keep this automation running.

The AWS Resource Explorer is a resource search and discovery service letting you gather all resources in your organization. You can provide a broad range of inputs like the ARN, a string or a tag key. The output provides you all information which you would also get when calling the resource.

So if we would search for a resource which doesn't exist, we just don't get any output. Makes sense right?

Before deploying the Lambda function, make sure to set up Resource Explorer correctly also including all relevant regions where you have deployed resources. All information regarding the setup can be found in the AWS Documentations

All set up? Let's look at the Lambda!

Cleaning up the mess

The function starts off with a Security Hub paginator to gather all current findings. We are filtering the output by the ComplianceStatus which should be NOT_AVAILABLE, the RecordState set to ACTIVE to not get already archived ones and the WorkflowStatus not being set to RESOLVED.
Then we start looping over the findings capturing some variables out of the output like the ARN of the resource belonging to the findings.

def lambda_handler(event, context):
    securityhub = boto3.client('securityhub')
    paginator = securityhub.get_paginator('get_findings')


    finding_filters = {
        'ComplianceStatus': [
            {
                'Value': "NOT_AVAILABLE",
                'Comparison': 'EQUALS'
            }
        ],
        'RecordState': [
            {
                'Value': "ACTIVE",
                'Comparison': 'EQUALS'
            }
        ],
        'WorkflowStatus': [
            {
                'Value': "RESOLVED",
                'Comparison': 'NOT_EQUALS'
            }
        ]
    }

    page_iterator = paginator.paginate(Filters=finding_filters)
    for page in page_iterator:
        for finding in page['Findings']:
            resource = finding['Resources'][0] 
            resource_id = resource['Id']

            exists = resource_exists(resource_id)

            if exists==False:
                resolve_sechub_finding(finding)


    return {
        'statusCode': 200,
        'body': json.dumps('Security Hub Finding Compliance Status Check Completed.')
    }

Then the resource_exists method gets called using the Resource Explorer to check whether the resource is still existent. As we learned earlier, if it has been deleted, the Resource Explorer just delivers no output, which will return a False boolean in our function.

def resource_exists(resource_id):

    resexp = boto3.client('resource-explorer-2')

    try:
        print(f"Search for resource {resource_id}")
        results = resexp.search(QueryString=resource_id)
        if resource_id in results['Resources'][0]['Arn']:
            print(f"Resource {resource_id} still exists.")
            return True
        else:
            print(f"Resource {resource_id} doesn't exist anymore.")
    except Exception as e:

        return False

    return False

The next step is probably obvious. We check whether the resource check returned a False, and if so, we set the WorkflowStatus of the corresponding finding to Resolved adding a note that this state change was executed by the automation.

def resolve_sechub_finding(finding):

    sechub = boto3.client('securityhub')

    try:
        print(f"Finding {finding['Id']} will be resolve as resource not exists any more.")
        response = sechub.batch_update_findings(
                    FindingIdentifiers = [{'Id': finding['Id'], 'ProductArn': finding['ProductArn']}],
                    Workflow = {
                        'Status': 'RESOLVED'
                    },
                    Note = {
                        "Text": "This resource no longer exists. Findings for this resource have been set to RESOLVED.",
                        "UpdatedBy": "DeletedresourceFindingResolver"
                    }
                )

    except Exception as e:
        print(e)

So if you now open the default view of the Security Hub Findings we can enjoy our tidied up results leaving off just the eligible Findings.

I would suggest to also add an Eventbridge Scheduled Rule to have this function running on a regular basis. In huge environments it may also make sense to play around with the Memory to not run into a timeout.

About Me

Hi! My name is Jana, I live in the Southwest of Germany and when I'm not smashing weights in the gym I love to architect solutions in AWS making my and the customers lives easier.

My computer science journey started as an On-Premise System Administrator over the time developing to an AWS Architect. As I know both the "old" and "new" world, I know common pain points in architectures and being able to provide solutions to solve them and making them not even more efficient but also cheaper!

I enjoy to learn and as the AWS portfolio is evolving all the time, I also try to stay up to date by getting certified and checking out newly launched products and services.

If you want to lift your environment either to the cloud or want to leverage your already migrated environment to use more of the cloud services, hit me up or check out Public Cloud Group GmbH!

About PCG

Public Cloud Group supports companies in their digital transformation through the use of public cloud solutions.

With a product portfolio designed to accompany organisations of all sizes in their cloud journey and competence that is a synonym for highly qualified staff that clients and partners like to work with, PCG is positioned as a reliable and trustworthy partner for the hyperscalers, relevant and with repeatedly validated competence and credibility.

We have the highest partnership status with the three relevant hyperscalers: Amazon Web Services (AWS), Google, and Microsoft. As experienced providers, we advise our customers independently with cloud implementation, application development, and managed services.