Introduction
LocalStack is an open-source framework that allows us to emulate the major AWS services locally, making it easier to develop and test cloud applications without incurring the cost and complexity of deploying to a real cloud environment.
In this post, I'll show how to configure it to emulate S3 buckets and how to interact with those buckets from a C# application.
Running LocalStack
LocalStack can be run as a CLI or using a container. In this post, I'll explain how to run it in a container with docker run
and docker-compose
.
âšī¸ If you don't have docker or other container runtime installed, click here for Docker installation instructions or here for Podman installation instructions.
Container
To start a container with a LocalStack instance, run the following command:
docker run --rm -it -p 4566:4566 -p 4510-4559:4510-4559 -e EXTRA_CORS_ALLOWED_ORIGINS=https://app.localstack.cloud. localstack/localstack:1.3.1
Note that it is exposing port 4566
and ports 4510
to 4559
and allowing CORS access from https://app.localstack.cloud.
(to allow access from the LocalStack dashboard).
đ¨ It's recommended to use a specific version instead of
latest
to avoid problems with new versions updated automatically.
Docker compose
Starting LocalStack from docker compose is just as easy. Just add the localstack
service, as below, to a docker-compose.yaml
file:
version: "3.8"
services:
localstack:
container_name: "${LOCALSTACK_DOCKER_NAME-localstack_main}"
image: localstack/localstack
ports:
- "127.0.0.1:4566:4566" # LocalStack Gateway
- "127.0.0.1:4510-4559:4510-4559" # external services port range
environment:
- DEBUG=${DEBUG-}
- LAMBDA_EXECUTOR=${LAMBDA_EXECUTOR-}
- DOCKER_HOST=unix:///var/run/docker.sock
- EXTRA_CORS_ALLOWED_ORIGINS=https://app.localstack.cloud. # Enable access from the dashboard
volumes:
- "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack"
- "/var/run/docker.sock:/var/run/docker.sock"
Then run the following command:
docker-compose up
đ¨ It's recommended to use a specific version instead of
latest
to avoid problems with new versions updated automatically.â ī¸ I've added the environment variable
EXTRA_CORS_ALLOWED_ORIGINS
with the valuehttps://app.localstack.cloud.
to allow access from the LocalStack dashboard.đĄ The updated docker-compose.yaml file can be found here, in the LocalStack repo.
LocalStack Dashboard
LocalStack has a web-based dashboard that allows us to manage and configure its services and visualize its logs.
Access https://app.localstack.cloud. and it will connect to the LocalStack instance running locally.
It runs in the browser, so there is no need to expose any ports to the internet.
đ¨ If the dashboard says "Please start LocalStack to check the System Status" and the container log shows Blocked CORS request from forbidden origin https://app.localstack.cloud., the
EXTRA_CORS_ALLOWED_ORIGINS
environment variable was not correctly set tohttps://app.localstack.cloud.
. See here.
Interacting with LocalStack using the AWS CLI
We'll use the AWS CLI to interact with LocalStack. If you don't have the AWS CLI, look here for instructions on how to install it.
Configuring a profile for LocalStack
The AWS CLI requires credentials when running. LocalStack doesn't validate the credentials by default, so will create a profile with anything as access key and secret key just to make the CLI happy.
- In a terminal, type
aws configure --profile localstack
; - For
AWS Access Key ID [None]:
, type anything; - For
AWS Secret Access Key [None]:
, type anything; - For
Default region name [None]:
, type the region you prefer (for example,us-east-1
); - For
Default output format [None]:
, typejson
.
How to create a bucket using the AWS CLI
To create a S3 bucket in LocalStack, we'll use the aws s3 mb
command (mb is short for Make Bucket).
The command below will create a bucket with the name local-bucket-name
using the AWS CLI profile we previously created with the name localstack
. It's important to pass the --endpoint
parameter or else it will try to create the bucket in AWS.
aws s3 mb s3://local-bucket-name --endpoint http://localhost:4566 --profile localstack
Looking at LocalStack dashboard we can see the bucket was created:
How to list a bucket contents using the AWS CLI
To look the contents of a bucket, we can use the aws s3 ls
command:
aws s3 ls s3://local-bucket-name --endpoint http://localhost:4566 --profile localstack
Or use the LocalStack dashboard:
Accessing LocalStack from .NET
To access an S3 bucket from LocalStack in .NET we use the same libraries we use to access it in AWS.
In this example, I'll use the AWSSDK.S3
and the AWSSDK.Extensions.NETCore.Setup
NuGet packages, both from AWS.
How does the AmazonS3Client get AWS access data?
When running in AWS, the AmazonS3Client
will get its access data from the IAM Role attached to the service running it. When running locally, it will get from the AWS CLI profile named default
or from the settings we pass to it.
In the code below, I'm checking for a configuration section with the name AWS
, that is not present in the production environment. If it's found, I set the Region
, ServiceURL
and ForcePathStyle
properties of the AmazonS3Config
and pass it to the creation of the AmazonS3Client
.
Program.cs
var builder = WebApplication.CreateBuilder(args);
builder.AddAwsS3Service();
var app = builder.Build();
AwsExtensions.cs
public static class AwsExtensions
{
public static void AddAwsS3Service(this WebApplicationBuilder builder)
{
if (builder.Configuration.GetSection("AWS") is null)
{
builder.Services.AddAWSService<IAmazonS3>();
}
else
{
builder.Services.AddSingleton<IAmazonS3>(sc =>
{
var awsS3Config = new AmazonS3Config
{
RegionEndpoint = RegionEndpoint.GetBySystemName(builder.Configuration["AWS:Region"]),
ServiceURL = builder.Configuration["AWS:ServiceURL"],
ForcePathStyle = bool.Parse(builder.Configuration["AWS:ForcePathStyle"]!)
};
return new AmazonS3Client(awsS3Config);
});
}
}
}
The appsettings.Development.json
has the configurations pointing to the LocalStack instance:
appsettings.Development.json
{
"BucketName": "local-bucket-name",
"AWS": {
"Region": "us-east-1",
"ServiceURL": "http://localhost:4566",
"ForcePathStyle": "true"
}
}
â ī¸ The
ForcePathStyle
forces the use ofhttps://s3.amazonaws.com/<bucket-name>/<object-key>
styled URLs instead ofhttps://<bucket-name>.s3.amazonaws.com/<object-key>
URLs.The
ForcePathStyle
needs to be set totrue
for theAmazonS3Client
to work with LocalStack.
Upload an image to the S3 Bucket
Using Minimal APIs, I created an endpoint that receives a file and saves it in the S3 bucket.
The code is straight forward:
Program.cs
app.MapPost("/upload", async (IAmazonS3 s3Client, IFormFile file) =>
{
var bucketName = builder.Configuration["BucketName"]!;
var bucketExists = await s3Client.DoesS3BucketExistAsync(bucketName);
if (!bucketExists)
{
return Results.BadRequest($"Bucket {bucketName} does not exists.");
}
using var fileStream = file.OpenReadStream();
var putObjectRequest = new PutObjectRequest()
{
BucketName = bucketName,
Key = file.FileName,
InputStream = fileStream
};
putObjectRequest.Metadata.Add("Content-Type", file.ContentType);
var putResult = await s3Client.PutObjectAsync(putObjectRequest);
return Results.Ok($"File {file.FileName} uploaded to S3 successfully!");
});
Now, we can test it from Postman:
Get an image from the S3 Bucket
I also created an endpoint that returns the file with the key passed by parameter from the S3 bucket:
app.MapGet("/object/{key}", async (IAmazonS3 s3Client, string key) =>
{
var bucketName = builder.Configuration["BucketName"]!;
var bucketExists = await s3Client.DoesS3BucketExistAsync(bucketName);
if (!bucketExists)
{
return Results.BadRequest($"Bucket {bucketName} does not exists.");
}
try
{
var getObjectResponse = await s3Client.GetObjectAsync(bucketName,
key);
return Results.File(getObjectResponse.ResponseStream,
getObjectResponse.Headers.ContentType);
}
catch (AmazonS3Exception ex) when (ex.ErrorCode.Equals("NotFound", StringComparison.OrdinalIgnoreCase))
{
return Results.NotFound();
}
});
Testing from Postman, we can see the image previously uploaded:
Full source code
Liked this post?
I post extra content in my personal blog. Click here to see.
Top comments (0)