DEV Community

Cover image for Understand Amazon SSM Agent In 2 Minutes
馃殌  Vu Dao 馃殌
馃殌 Vu Dao 馃殌

Posted on • Updated on

Understand Amazon SSM Agent In 2 Minutes

馃殌 Install SSM Agent on Ubuntu Server instances

To install SSM Agent on Ubuntu Server 20.10 STR & 20.04, 18.04, and 16.04 LTS 64-bit instances (with Snap package)

~ $:/home/ubuntu# sudo snap install amazon-ssm-agent --classic
Enter fullscreen mode Exit fullscreen mode

馃殌 Check SSM Agent log

~ $:/home/ubuntu# systemctl restart snap.amazon-ssm-agent.amazon-ssm-agent.service                                                                                                                                                                                                 
~ $:/home/ubuntu# tail -f /var/log/amazon/ssm/amazon-ssm-agent.log                                                                                                                                                                                                                 
        status code: 400, request id: ea74ed4f-70d4-4610-8221-ce7868c3c9fb                                                                                                                                                                                                                
2021-01-08 08:40:20 INFO [amazon-ssm-agent] [SelfUpdate] Initializing self update ...
2021-01-08 08:40:20 INFO [amazon-ssm-agent] Starting Core Agent: amazon-ssm-agent - v3.0.161.0
2021-01-08 08:40:20 INFO [amazon-ssm-agent] OS: linux, Arch: amd64
2021-01-08 08:40:22 INFO [amazon-ssm-agent] [LongRunningWorkerContainer] [WorkerProvider] Worker ssm-agent-worker is not running, starting worker process
2021-01-08 08:40:22 INFO [amazon-ssm-agent] [LongRunningWorkerContainer] [WorkerProvider] Worker ssm-agent-worker (pid:11067) started
2021-01-08 08:40:22 ERROR Error adding the directory to watcher: no such file or directory
2021-01-08 08:40:22 INFO [amazon-ssm-agent] [LongRunningWorkerContainer] Monitor long running worker health every 60 seconds
2021-01-08 08:40:22 INFO [ssm-agent-worker] Dial to Core Agent broadcast channel
2021-01-08 08:40:22 INFO [ssm-agent-worker] Dial to Core Agent broadcast channel
2021-01-08 08:40:22 INFO [ssm-agent-worker] Create new startup processor
2021-01-08 08:40:22 INFO [ssm-agent-worker] Start to listen to Core Agent health channel
2021-01-08 08:40:22 INFO [ssm-agent-worker] Start to listen to Core Agent termination channel
2021-01-08 08:40:22 INFO [ssm-agent-worker] [StartupProcessor] Executing startup processor tasks
2021-01-08 08:40:22 INFO [ssm-agent-worker] [StartupProcessor] Write to serial port: Amazon SSM Agent v3.0.161.0 is running
2021-01-08 08:40:22 INFO [ssm-agent-worker] [StartupProcessor] Write to serial port: OsProductName: Ubuntu
2021-01-08 08:40:22 INFO [ssm-agent-worker] [StartupProcessor] Write to serial port: OsVersion: 18.04
2021-01-08 08:40:23 INFO [ssm-agent-worker] Entering SSM Agent hibernate - AccessDeniedException: User: arn:aws:sts::111111111111:assumed-role/SSMAutomation/i-06f3424a03d04c66d is not authorized to perform: ssm:UpdateInstanceInformation on resource: arn:aws:ec2:eu-central-1:111111111111:instance/i-06f3424a03d04c66d
        status code: 400, request id: f0580f42-a0c8-4038-8242-46e4818a391b
Enter fullscreen mode Exit fullscreen mode
  • It shows that the SSM agent does not have permission on SSM action such as UpdateInstanceInformation for the instance that it relies on

2021-01-08 08:40:23 INFO [ssm-agent-worker] Entering SSM Agent hibernate - AccessDeniedException: User: arn:aws:sts::111111111111:assumed-role/SSMAutomation/i-06f3424a03d04c66d is not authorized to perform: ssm:UpdateInstanceInformation on resource: arn:aws:ec2:eu-central-1:111111111111:instance/i-06f3424a03d04c66d

馃殌 Attach instance profile for ssm agent permission

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AmazonSSMtoEC2",
            "Effect": "Allow",
            "Action": [
                "ssm:*",
                "ssmmessages:CreateControlChannel",
                "ssmmessages:CreateDataChannel",
                "ssmmessages:OpenControlChannel",
                "ssmmessages:OpenDataChannel",
                "ec2messages:AcknowledgeMessage",
                "ec2messages:DeleteMessage",
                "ec2messages:FailMessage",
                "ec2messages:GetEndpoint",
                "ec2messages:GetMessages",
                "ec2messages:SendReply"
            ],
            "Resource": "*"
        }
    ]
}
Enter fullscreen mode Exit fullscreen mode
  • Restart the ssm agent for the role affects
2021-01-08 08:44:27 INFO [ssm-agent-worker] Starting SSM Agent Worker: amazon-ssm-agent - v3.0.161.0
2021-01-08 08:44:27 INFO [ssm-agent-worker] OS: linux, Arch: amd64
2021-01-08 08:44:27 INFO [ssm-agent-worker] [MessagingDeliveryService] Starting document processing engine...
2021-01-08 08:44:27 INFO [ssm-agent-worker] [OfflineService] Starting document processing engine...
2021-01-08 08:44:27 INFO [ssm-agent-worker] [OfflineService] [EngineProcessor] Starting
2021-01-08 08:44:27 INFO [ssm-agent-worker] [OfflineService] [EngineProcessor] Initial processing
2021-01-08 08:44:27 INFO [ssm-agent-worker] [HealthCheck] HealthCheck reporting agent health.
2021-01-08 08:44:27 INFO [ssm-agent-worker] [OfflineService] Starting message polling
2021-01-08 08:44:27 INFO [ssm-agent-worker] [OfflineService] Starting send replies to MDS
2021-01-08 08:44:27 INFO [ssm-agent-worker] [LongRunningPluginsManager] starting long running plugin manager
2021-01-08 08:44:27 INFO [ssm-agent-worker] [LongRunningPluginsManager] there aren't any long running plugin to execute
2021-01-08 08:44:27 INFO [ssm-agent-worker] [MessagingDeliveryService] [EngineProcessor] Starting
2021-01-08 08:44:27 INFO [ssm-agent-worker] [MessagingDeliveryService] [EngineProcessor] Initial processing
2021-01-08 08:44:27 INFO [ssm-agent-worker] [MessagingDeliveryService] Starting message polling
2021-01-08 08:44:27 INFO [ssm-agent-worker] [MessagingDeliveryService] Starting send replies to MDS
2021-01-08 08:44:27 INFO [ssm-agent-worker] [instanceID=i-06f3424a03d04c66d] Starting association polling
2021-01-08 08:44:27 INFO [ssm-agent-worker] [MessagingDeliveryService] [Association] [EngineProcessor] Starting
2021-01-08 08:44:27 INFO [ssm-agent-worker] [MessagingDeliveryService] [Association] Launching response handler
2021-01-08 08:44:27 INFO [ssm-agent-worker] [MessagingDeliveryService] [Association] [EngineProcessor] Initial processing
2021-01-08 08:44:27 INFO [ssm-agent-worker] [MessagingDeliveryService] [Association] Initializing association scheduling service
2021-01-08 08:44:27 INFO [ssm-agent-worker] [MessagingDeliveryService] [Association] Association scheduling service initialized
2021-01-08 08:44:27 INFO [ssm-agent-worker] [MessageGatewayService] Starting session document processing engine...
2021-01-08 08:44:27 INFO [ssm-agent-worker] [MessageGatewayService] [EngineProcessor] Starting
2021-01-08 08:44:27 INFO [ssm-agent-worker] [MessageGatewayService] SSM Agent is trying to setup control channel for Session Manager module.
2021-01-08 08:44:27 INFO [ssm-agent-worker] [MessageGatewayService] agent telemetry cloudwatch metrics disabled
2021-01-08 08:44:27 INFO [ssm-agent-worker] [MessageGatewayService] Setting up websocket for controlchannel for instance: i-06f3424a03d04c66d, requestId: 8ad8e181-fff5-4aee-81ca-4bbaf6542c3e
2021-01-08 08:44:27 INFO [ssm-agent-worker] [MessageGatewayService] listening reply.
Enter fullscreen mode Exit fullscreen mode

馃殌 Conclusion

  • For using AWS Systems Manager Run Command, not only set IAM policy for the instance sending SSM command but also the target need IAM policy so that its SSM agent can assume EC2 resource

Mirror

Read More

馃尃 Blog Web Linkedin Group Page Twitter 馃尃

Discussion (10)

Collapse
shokuninkishitsu profile image
Oleksiy Holubyev

Hmm, I have exactly this problem, but I don't understand where I should attach the instance profile. My EC2 instance already has AmazonSSMRoleForInstanceQuickSetup role attached, which has the AmazonSSMManagedInstanceCore policy, which should have all of the listed permissions. But I still get the error mentioned above.

Collapse
vumdao profile image
馃殌 Vu Dao 馃殌 Author

Attache instance profile which provide ssm:* permisson to the EC2 instance which you got the ssm-agent error

Collapse
shokuninkishitsu profile image
Oleksiy Holubyev

I have that. But I still get the error.

Thread Thread
vumdao profile image
馃殌 Vu Dao 馃殌 Author

Can you show you policy and the log

Thread Thread
shokuninkishitsu profile image
Oleksiy Holubyev

Sure. The role in the instance profile is AmazonSSMRoleForInstancesQuickSetup, the associated policy is arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore:

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ssm:DescribeAssociation",
"ssm:GetDeployablePatchSnapshotForInstance",
"ssm:GetDocument",
"ssm:DescribeDocument",
"ssm:GetManifest",
"ssm:GetParameter",
"ssm:GetParameters",
"ssm:ListAssociations",
"ssm:ListInstanceAssociations",
"ssm:PutInventory",
"ssm:PutComplianceItems",
"ssm:PutConfigurePackageResult",
"ssm:UpdateAssociationStatus",
"ssm:UpdateInstanceAssociationStatus",
"ssm:UpdateInstanceInformation"
],
"Resource": ""
},
{
"Effect": "Allow",
"Action": [
"ssmmessages:CreateControlChannel",
"ssmmessages:CreateDataChannel",
"ssmmessages:OpenControlChannel",
"ssmmessages:OpenDataChannel"
],
"Resource": "
"
},
{
"Effect": "Allow",
"Action": [
"ec2messages:AcknowledgeMessage",
"ec2messages:DeleteMessage",
"ec2messages:FailMessage",
"ec2messages:GetEndpoint",
"ec2messages:GetMessages",
"ec2messages:SendReply"
],
"Resource": "*"
}
]
}

and the error is:

Entering SSM Agent hibernate - AccessDeniedException: User: arn:aws:sts::111111111111:assumed-role/AmazonSSMRoleForInstancesQuickSetup/i-instance-id
is not authorized to perform: ssm:UpdateInstanceInformation on resource: arn:aws:ec2:eu-central-1:111111111111:instance/i-instance-id
status code: 400, request id:

Thread Thread
vumdao profile image
馃殌 Vu Dao 馃殌 Author

Why the resource is "" for ssm action?

Thread Thread
shokuninkishitsu profile image
Oleksiy Holubyev

It's just the text box interpreted the asterisk as italic formatting. The asterisk is there in the policy itself.

Collapse
routinggames profile image
Duy Nguy3n

It solved my problem with SSM agent :)

Collapse
vumdao profile image
馃殌 Vu Dao 馃殌 Author

Glad to hear that

Collapse
routinggames profile image
Duy Nguy3n

Thanks for sharing