Building a Fargate API Server with Go, Gin, Docker, and AWS Copilot

Hello fellow tech enthusiasts!

As a self-taught engineer with over two decades of experience, starting back in the days the browser wars, my journey has been driven by curiosity, innovation, and a passion for understanding the why and how technology does its "magic".

I chose to use Golang over Python not just because it's fun, but because it presented an opportunity to demystify its "magic" and deepen my understanding (even though I've been using it since it was in beta). This became especially relevant during a conversation with a respected leader who posed a seemingly simple question about Go's struct vs. interface. As a self-taught engineer who's gone toe-to-toe with imposter syndrome, vocabulary questions like this give me sweaty palms. But I realized that this was a great opportunity to break down these concepts and really understand them at a deeper level.

(PS A struct is a composite data type that groups together variables under a single name, while an interface is a collection of method signatures that a type must implement. Understanding the difference between these two concepts is crucial for writing clean and efficient Go code.)

Langchain was another exciting venture, and I am thrilled that Harrison Chase figured this one out because chaining GPT-2 and GPT-3 calls in Python in 2021 was a pain the keister!

And then there's AWS Fargate, a game-changer in serverless container platforms. Having used Fargate at NBA Digital and NBATV to serve content to 50 million concurrent users at scale nightly, its efficiency and scalability are undeniable.

Through all these experiences, my hope is to inspire fellow self-taught engineers, innovators, and anyone with a thirst for knowledge. Let's explore, learn, and innovate together!

In this blog post, we'll walk through the process of building an API server using Go and the Gin framework, containerizing it with Docker, and deploying it to AWS Fargate using AWS Copilot. We'll also delve into the importance of environment management and testing. Let's get started.

Please refer to the code in this git repo: https://github.com/doctor-ew/go_skippy_lc

1. Docker: Containerizing the Go Application

Docker allows us to package our application and its dependencies into a container, ensuring consistent behavior across different environments. Let's break down the Dockerfile from the go_skippy_lc repository:

1.1. DockerfileCopy code

FROM golang:1.17-alpine

Explanation: This line specifies the base image for our Docker container. In this case, you're using the official Go image (golang) with version 1.17 based on the lightweight Alpine Linux distribution (alpine). This image will have the Go runtime and tools pre-installed.

1.2. DockerfileCopy code

WORKDIR /app

Explanation: This line sets the working directory inside the container to /app. All subsequent commands in the Dockerfile (like COPY or RUN) will be executed in this directory. Essentially, this directory will be the root for our application inside the container.

1.3. DockerfileCopy code

COPY go.mod . COPY go.sum .

Explanation: These lines copy the go.mod and go.sum files from our local machine (outside the container) to the current directory inside the container (/app). These files are essential for Go's module system, ensuring that the correct dependencies are used when building our application.

1.4. DockerfileCopy code

RUN go mod download

Explanation: This line runs the go mod download command inside the container. This command fetches all the dependencies listed in go.mod and go.sum, ensuring they're available in the container for the build process.

1.5. DockerfileCopy code

COPY . .

Explanation: This line copies everything from our current directory on our local machine (i.e., the root of our Go project) to the current directory inside the container (/app). This ensures that all our application's source code and other necessary files are available inside the container.

1.6. DockerfileCopy code

RUN go build -o ./out/myapi .

Explanation: This line runs the go build command inside the container to compile our Go application. The -o ./out/myapi flag specifies the output directory and name for the compiled binary. In this case, the binary will be named myapi and will be located in the /app/out/ directory inside the container.

1.7. DockerfileCopy code

CMD ["./out/myapi"]

Explanation: This line specifies the command that will be executed when the container starts. In this case, it's running the compiled Go application (./out/myapi). The CMD instruction allows the container to behave like an executable, meaning when you run the container, it will automatically start our Go application.

2. Setting Up the Gin Server, Environment Management, and .gitignore

Before diving into the code, it's essential to understand the importance of environment management and the role of .gitignore in safeguarding sensitive information.

2.1. Environment Files and .gitignore

Before we delve into the code, it's essential to mention the importance of environment files and .gitignore. Storing sensitive information, like API keys, in environment variables is a best practice. This ensures that these keys are not hard-coded into the application, reducing the risk of accidental exposure. The .gitignore file ensures that certain files, like the .env containing these keys, are not committed to version control, further safeguarding them.

2.2. Imports

Here's a brief overview of each import:

• Standard Library Imports:

  • context: Provides a way to carry deadlines, cancellations, and other request-scoped values across API boundaries.
  • fmt: Implements formatted I/O functions.
  • log: Provides logging capabilities.
  • net/http: Provides HTTP client and server implementations.
  • os: Provides a platform-independent interface to operating system functionality.
  • os/signal: Provides a way to intercept and act upon signals sent to the application.
  • syscall: Contains an interface to the low-level operating system primitives.
  • time: Provides functionality for measuring and displaying time.

• Third-party Imports:

  • github.com/gin-gonic/gin: Gin is a web framework for building APIs in Go. It's known for its performance and small memory footprint.
  • github.com/joho/godotenv: A package to load environment variables from a .env file.
  • github.com/tmc/langchaingo/llms: A library related to the language chain (from the context, it seems to be related to interfacing with OpenAI).
  • github.com/tmc/langchaingo/llms/openai: Specific OpenAI related functionalities for the language chain.
  • github.com/tmc/langchaingo/schema: Defines the schema or structure for the messages and responses with OpenAI.

2.3. Code Structure

a. Interface vs. Struct

• Interface:

  goCopy code

  type Chat interface { Call(ctx context.Context, messages []schema.ChatMessage, options ...llms.CallOption) (*schema.AIChatMessage, error) }

  Explanation: An interface Chat is defined, which any type must satisfy if it has a Call method with the specified signature. This allows for flexibility and can be used to mock the OpenAI chat for testing or to use different implementations.

• Struct:

  goCopy code

  type RequestBody struct { Message string `json:"message"` }

  Explanation: This struct RequestBody defines the structure of the request body that the /ask-skippy endpoint expects. The json:"message" tag indicates that when this struct is unmarshaled from a JSON object, the Message field corresponds to the message key in the JSON.

2.4. Functions

a. srv Function

This function sets up and starts the Gin server. It defines two endpoints: /ask-skippy for POST requests and / for GET requests. It also sets up graceful shutdown for the server when it receives an interrupt or termination signal.

b. askSkippy Function

This function interfaces with the OpenAI chat (or any other implementation that satisfies the Chat interface). It sends a system message to set the context for the AI and then sends the user's message. It then waits for a response from the AI and returns it.

c. main Function

This is the entry point of the application. It loads the environment variables from the .env file, retrieves the OpenAI API key, initializes the OpenAI chat, and then starts the server.

This breakdown provides a high-level overview of the code's structure and functionality. Each section and function plays a crucial role in setting up the server, interfacing with OpenAI, and serving responses to the user.

3. Testing: Ensuring Our Application Works as Expected

Testing is a crucial aspect of software development. It ensures that our application behaves as expected and helps catch issues early in the development process. Let's dive into the testing code for the askSkippy function:

3.1. Imports

• Standard Library Imports:

  • context: As before, this provides a way to carry request-scoped values.
  • errors: Provides functions to manipulate errors.
  • testing: The standard Go testing package.

• Third-party Imports:

  • github.com/stretchr/testify/mock: The testify library's mocking package. It provides utilities to easily mock interfaces for testing.
  • github.com/tmc/langchaingo/schema: Defines the schema or structure for the messages and responses with OpenAI.

3.2. MockChat Struct

goCopy code

type MockChat struct { mock.Mock }

Explanation: The MockChat struct embeds the mock.Mock type from the testify library. This allows MockChat to have all the methods and functionalities provided by mock.Mock, enabling easy setup of expected method calls and their return values.

3.3. Test Functions

a. TestAskSkippy

This test function checks the "happy path" scenario where everything works as expected.

• Mock Setup:

  goCopy code

  mockChat.On("Call", mock.Anything, mock.Anything, mock.Anything).Return(&schema.AIChatMessage{Content: "Mocked response"}, nil)

  Here, we're setting up the expectation that the Call method of mockChat will be invoked with any arguments (mock.Anything is a placeholder that matches any value). When invoked, it should return a mocked AI chat message with the content "Mocked response" and no error.

• Function Call:

  goCopy code

  response, err := askSkippy(context.Background(), mockChat, "English", "Test message for Skippy")

  We then call the askSkippy function with our mock chat and check the response.

• Assertions: The test checks two things:

1. That there's no error.


That the response matches the expected "Mocked response".

b. TestAskSkippy_Error

This test function checks the scenario where the Call method returns an error.

• Mock Setup:

  goCopy code

  mockChat.On("Call", mock.Anything, mock.Anything, mock.Anything).Return(nil, errors.New("Mocked error"))

  Here, we're setting up the expectation that the Call method of mockChat will be invoked and will return an error "Mocked error".

• Function Call:

  goCopy code

  _, err := askSkippy(context.Background(), mockChat, "English", "Test error message for Skippy")

  We then call the askSkippy function with our mock chat and check for an error.

• Assertion: The test checks that an error is returned.

Potential Issues and Solutions:

1. MockChat Implementation: The MockChat struct is defined, but its methods aren't. For the tests to work, the MockChat needs to have a Call method that uses the testify mock's functionalities. This method should look something like:

   goCopy code

   func (m *MockChat) Call(ctx context.Context, messages []schema.ChatMessage, options ...llms.CallOption) (*schema.AIChatMessage, error) { args := m.Called(ctx, messages, options) return args.Get(0).(*schema.AIChatMessage), args.Error(1) }

2. Dependencies: Ensure that the testify library is installed. If not, it can be added using:

   bashCopy code

   go get github.com/stretchr/testify

3. Integration with Main Code: Ensure that the Chat interface in the main code and the MockChat in the test code are in sync. If the interface changes, the mock and tests need to be updated accordingly.

This breakdown provides an overview of the testing code's structure and functionality. The tests are designed to validate the behavior of the askSkippy function in both normal and error scenarios.

4. AWS Fargate and Copilot: Deploying and Managing the Service

AWS Fargate and Copilot provide a seamless experience for deploying and managing containerized applications on AWS without the need to manage the underlying infrastructure. With AWS Copilot, you can define, release, and manage services using simple CLI commands. Let's dive into the Copilot configuration files from the go_skippy_lc/copilot directory to understand how the service and environment are set up:

4.1. Service Configuration: copilot/go-skippy-lc/manifest.yml

Overview:

This file defines the configuration for the go-skippy-lc service. It's a Load Balanced Web Service, which means it's a public-facing web service that's behind a load balancer.

Breakdown:

• name & type:

  • name: go-skippy-lc specifies the name of the service.
  • type: Load Balanced Web Service indicates the type of service.

• http:

  • path: '/' specifies the path that the load balancer should forward requests to.
  • alias: This is a list of domain names that should route to this service. This is useful for custom domain routing.
  • healthcheck: (commented out) would specify a custom health check path for the service.

• image:

  • build: Dockerfile specifies the Dockerfile to use for building the container image.
  • port: 80 is the port the container listens on.

• cpu, memory, count:

  • These fields specify the resources for the ECS task. It uses 256 CPU units, 512 MiB of memory, and there should always be 1 task running.

• exec:

  • exec: true allows you to run commands in our container.

• network:

  • connect: true enables Service Connect for intra-environment traffic between services.

• storage:

  • readonly_fs: (commented out) would limit the mounted root filesystems to read-only access.

• variables & secrets:

  • (Both commented out) would allow us to pass environment variables and secrets to our service.

• environments:

  • (Commented out) would allow us to override any of the above values for specific environments.

4.2. Environment Configuration: copilot/environments/test/manifest.yml

Overview:

This file defines the configuration for the test environment.

Breakdown:

• name & type:

  • name: test specifies the name of the environment.
  • type: Environment indicates the type of configuration.

• network:

  • (Commented out) would allow us to specify a custom VPC or configure how the VPC should be created.

• http:

  • public: Specifies the configuration for the public load balancer in the environment.
  • certificates: Lists the ARN of the SSL certificate to use with the load balancer.

• observability:

  • container_insights: false specifies that container insights (for monitoring) should not be enabled for this environment.

Manual Step:

I had to manually point the A Record to the ALB. This step is necessary because while AWS Copilot can automate the creation of resources like the ALB, the final step of updating DNS records to point to the ALB often requires manual intervention, especially if you're using a third-party DNS provider or have specific routing requirements.

Conclusion

As an architect or engineer, we need to think through the atoms that compose the thing we're building. In this case, a robust and scalable API server encompasses everything from server setup and environment management to containerization and rigorous testing. I've shared this guide to provide a solid foundation for deploying a Go-based API server to AWS Fargate.

My hope is that this not only serves as a practical guide but also inspires fellow self-taught engineers, innovators, and all curious minds. Let's continue to explore, learn, and push the boundaries of innovation together.

Excelsior!