Day 017 - 100DaysAWSIaCDevopsChallenge
Today in my series of 100 days of code challenge, I will show you how to decouple microservices developed with springboot using Amazon SQS.
What is Amazon SQS ?
Amazon SQS (Simple Queue Service) is a cloud service that helps applications communicate by sendyg, storing and receiving messages in a queue. It's like a waiting line where are stored until a consumer is ready to process them. This prevents systems from getting overhelmed and make sure no message is lost.
Consume a SQS Messages using Springboot application
To demonstrate how to consume SQS messages by creating a Spring Boot app that processes each message from an SQS queue. The infrastructure, built using CDK (Java), will include:
- a VPC with a public Subnet to host an EC2 instance where the Spring Boot app will run.
- An Internet Gateway for the EC2 instance to access the internet and download dependencies.
- A SQS Queue and Dead Letter Queue for message storage.
- An EC2 Instance for hosting SpringBoot App.
- An IAM Role to allow the EC2 instance to retrieve messages from the SQS queue (very important).
Create the Infrastructure
Set up the necessary infrastructure using CDK (Java)
VPC & Subne + Internet Gateway
public class NetworkContruct extends Construct {
private final IVpc vpc;
public NetworkContruct(Construct scope, String id, StackProps props) {
super(scope, id);
this.vpc =
new Vpc(
this,
"VpcResource",
VpcProps.builder()
.vpcName("my-vpc")
.enableDnsHostnames(true)
.enableDnsSupport(true)
.createInternetGateway(true)
.ipProtocol(IpProtocol.IPV4_ONLY)
.ipAddresses(IpAddresses.cidr("10.0.0.1/16"))
.maxAzs(1)
.subnetConfiguration(
List.of(
SubnetConfiguration.builder()
.name("Public-Subnet")
.mapPublicIpOnLaunch(true)
.subnetType(SubnetType.PUBLIC)
.build()))
.build());
}
public IVpc getVpc() {
return vpc;
}
}
This cdk construct will create:
👉🏽 A VPC named my-vpc and enable DNS hostname enabled.
👉🏽 A public subnet named Public-Subnet which allows resources to attach a public IP (if configured with one).
👉🏽 An Internet Gateway to enable internet traffic.
SQS Queue & Dead Letter Queue
public class QueueConstruct extends Construct {
private final IQueue queue;
public QueueConstruct(Construct scope, String id, IVpc vpc, StackProps props) {
super(scope, id);
IQueue dlq =
new Queue(
this,
"DeadLetterQueue",
QueueProps.builder()
.deliveryDelay(Duration.millis(0))
.retentionPeriod(Duration.days(10))
.queueName("my-queue-dlq")
.build());
DeadLetterQueue deadLetterQueue = DeadLetterQueue.builder()
.queue(dlq)
.maxReceiveCount(32)
.build();
this.queue =
new Queue(
this,
"SQSQueueResource",
QueueProps.builder()
.queueName("my-queue")
.retentionPeriod(Duration.minutes(15))
.visibilityTimeout(Duration.seconds(90))
.deadLetterQueue(deadLetterQueue)
.build());
}
public IQueue getQueue() {
return queue;
}
}
The above CDK construct will create the following resources:
- A Queue named
my-queue, which will be used in the Spring Boot app. - A DeadLetter Queue named
my-queue-dlqwhich captures failed messages so they can be analysed and fixed later, without blocking the main queue.
EC2 Instance for Hosting the Spring Boot Application
// ComputerProps.java
public record ComputerProps(IVpc vpc, String sqsQueueArn) {}
// ComputerConstruct.java
public class ComputerConstruct extends Construct {
private final IInstance computer;
public ComputerConstruct(
Construct scope, String id, ComputerProps computerProps, StackProps props) {
super(scope, id);
SecurityGroup securityGroup =
new SecurityGroup(
this,
"WebserverSGResource",
SecurityGroupProps.builder()
.allowAllOutbound(true)
.securityGroupName("Webserver-security-group")
.disableInlineRules(true)
.vpc(computerProps.vpc())
.description("Allow trafic from/to webserver instance")
.build());
securityGroup.addIngressRule(Peer.anyIpv4(), Port.SSH, "Allow ssh traffic");
securityGroup.addIngressRule(Peer.anyIpv4(), Port.tcp(8089), "Allow traffic from 8089 port");
KeyPair keyPair =
new KeyPair(
this,
"KeyPairResource",
KeyPairProps.builder()
.keyPairName("ws-keypair")
.account(Objects.requireNonNull(props.getEnv()).getAccount())
.type(KeyPairType.RSA)
.format(KeyPairFormat.PEM)
.build());
new CfnOutput(
this, "KeyPairId", CfnOutputProps.builder().value(keyPair.getKeyPairId()).build());
Instance ec2Instance =
new Instance(
this,
"WebServerInstanceResource",
InstanceProps.builder()
.securityGroup(securityGroup)
.keyPair(keyPair)
.instanceName("Webserver-Instance")
.machineImage(
MachineImage.lookup(
LookupMachineImageProps.builder()
.name("*ubuntu*")
.filters(
Map.ofEntries(
Map.entry("image-id", List.of("ami-0e86e20dae9224db8")),
Map.entry("architecture", List.of("x86_64"))))
.windows(false)
.build()))
.vpc(computerProps.vpc())
.role(buildInstanceRole(computerProps))
.instanceType(InstanceType.of(InstanceClass.T2, InstanceSize.MICRO))
.associatePublicIpAddress(true)
.blockDevices(
List.of(
BlockDevice.builder()
.mappingEnabled(true)
.deviceName("/dev/sda1")
.volume(
BlockDeviceVolume.ebs(
10,
EbsDeviceOptions.builder()
.deleteOnTermination(true)
.volumeType(EbsDeviceVolumeType.GP3)
.build()))
.build()))
.userDataCausesReplacement(true)
.vpcSubnets(SubnetSelection.builder().subnetType(SubnetType.PUBLIC).build())
.build());
UserData userData = UserData.forLinux();
userData.addCommands(readFile("./webserver-startup.sh"));
ec2Instance.addUserData(userData.render());
this.computer = ec2Instance;
}
public IInstance getComputer() {
return computer;
}
private String readFile(String filename) {
InputStream scriptFileStream = getClass().getClassLoader().getResourceAsStream(filename);
try {
assert scriptFileStream != null;
try (InputStreamReader isr = new InputStreamReader(scriptFileStream, StandardCharsets.UTF_8);
BufferedReader br = new BufferedReader(isr)) {
StringBuilder content = new StringBuilder();
String line;
while ((line = br.readLine()) != null) {
content.append(line).append("\n");
}
return content.toString();
}
} catch (IOException e) {
throw new RuntimeException(e.getMessage());
}
}
private IRole buildInstanceRole(ComputerProps props) {
return new Role(
this,
"WebserverInstanceRoleResource",
RoleProps.builder()
.roleName("webserver-role")
.assumedBy(new ServicePrincipal("ec2.amazonaws.com"))
.path("/")
.inlinePolicies(
Map.ofEntries(
Map.entry(
"sqs",
new PolicyDocument(
PolicyDocumentProps.builder()
.assignSids(true)
.statements(
List.of(
new PolicyStatement(
PolicyStatementProps.builder()
.effect(Effect.ALLOW)
.actions(
List.of(
"sqs:DeleteMessage",
"sqs:ReceiveMessage",
"sqs:SendMessage",
"sqs:GetQueueAttributes",
"sqs:GetQueueUrl"))
.resources(List.of(props.sqsQueueArn()))
.build())))
.build()))))
.build());
}
}
The above CDK construct will create the following resources:
- A Security Group named
Webserver-security-groupthat allow inbound traffic onPort 22for SSH connections and allow inbound traffic on Port 8089, which is the application connection port. - A Key Pair named
ws-keypairthat will be used to connect to the app host via SSH. Since we are using CDK to build the infrastructure, if you need to download the private key (PEM file) after deployment, refer to my previous article on How the retrieve the private key file PEM after Cloudformation or CDK stack creation[↗]. - An Ec2 Instance named
Webserver-Instance. - An IAM Role for the Ec2 Instance named
webserver-role, which allows the spring Boot application hosted on the Ec2 Instance to establish connections with the Amazon SQS Queue (already created) and perform actions:Delete Message,Receive Message,Send Message,Get Queue AttributesandGet Queue Url.
Create the stack
// MyStack.java
public class MyStack extends Stack {
public MyStack(final Construct scope, final String id, final StackProps props) {
super(scope, id, props);
IVpc vpc = new NetworkContruct(this, "NetworkResource", props).getVpc();
IQueue queue = new QueueConstruct(this, "QueueResource", vpc, props).getQueue();
IInstance webserver =
new ComputerConstruct(
this, "ComputerResource", new ComputerProps(vpc, queue.getQueueArn()), props)
.getComputer();
}
}
// Day17App.java
public class Day017App {
public static void main(final String[] args) {
App app = new App();
new MyStack(app,"Day017Stack",
StackProps.builder()
.env(
Environment.builder()
.account(System.getenv("CDK_DEFAULT_ACCOUNT"))
.region(System.getenv("CDK_DEFAULT_REGION"))
.build())
.build());
app.synth();
}
}
Create SpringBoot Consumer Application
To keep things simple and avoid complicating my life, I will use Spring Cloud AWS Docs[↗]
Spring Cloud AWS simplifies using AWS managed services in a Spring Framework and Spring Boot applications. It offers a convenient way to interact with AWS provided services using well-known Spring idioms and APIs.
To configure the SQS service in my project, I will add the following beans in the configuration class:
@Configuration
public class ApplicationConfiguration {
@Bean
public AwsRegionProvider customRegionProvider() {
return new InstanceProfileRegionProvider();
}
@Bean
public AwsCredentialsProvider customInstanceCredProvider() {
return InstanceProfileCredentialsProvider.builder()
.build();
}
}
And the listener that captures all new message and prints their content.
@Slf4j
@Component
public class ExampleSQSConsumer {
@SqsListener(queueNames = { "my-queue" }) // 👈👈
public void listen(String payload) {
log.info("******************* SQS Payload ***************");
log.info("Message Content: {}", payload);
log.info("Received At: {}", Date.from(Instant.now()));
log.info("************************************************");
}
}
You can find the full project in my GitHub repo[↗]
Deployment
⚠️⚠️ Before run the deployment commands ensure that you have java installed on your host machine. I used Java 21 under MacOs to build this insfrastructure.
Open the terminal anywhere and run the following commands:
git clone https://github.com/nivekalara237/100DaysTerraformAWSDevops.git
cd 100DaysTerraformAWSDevops/day_017
cdk bootstrap --profile cdk-user
cdk deploy --profile cdk-user Day017Stack
Resut
Your can find the full source code on GitHub Repo↗
Top comments (0)