DEV Community

Cover image for How to access Neon Postgres from AWS Lambda functions via serverless driver
Divine Orji for Hackmamba

Posted on

How to access Neon Postgres from AWS Lambda functions via serverless driver

Serverless is a computing paradigm that provides a pay-as-you-use model. It lets you deploy your projects easily and scale to meet large demands without the stress of maintaining or managing servers.

Wherever there are conversations on serverless, two important concepts often come up:

  • Compute — the processing power of a server.
  • Storage — the capacity to store data.

Traditionally, accessing a database from a serverless computing service like AWS Lambda is tricky due to the differences in connection protocols. Lambda functions are “ephemeral” — relying on short-lived, HTTP-centric connections, while databases utilize long-lived, TCP-centric connections.

It is possible to send a single-shot HTTP query to retrieve data from your database. However, maintaining a persistent connection (like streaming data to your clients) could lead to timeouts in your lambda runtime or compute costs that balloon unexpectedly.

One way to solve this problem is connecting via a gateway or proxy. This way, your client can establish a long-lived connection (using WebSockets) with the proxy, which then interacts with your backend using standard HTTP requests.

Neon’s serverless driver is a drop-in replacement for node-postgres that combines the benefits of WebSockets and HTTP into one simple package, optimizing the number of round trips required to interact with a Neon Postgres database.

This article will teach you how to access a Neon Postgres database from an AWS Lambda function via Neon’s serverless driver.

Data flow between Neon and AWS

Source code
Here’s a link to the GitHub repository for this article’s demo.

Prerequisites

  • A Neon account. Neon offers a generous free tier that you can start with. Get it here.
  • An AWS account. Explore its free tier here.
  • Node.js installed on your PC. It comes with npm, which you will use to add Neon’s serverless driver to your project.
  • A code editor. This article’s demo was built with Visual Studio Code.

Setting up a Neon database

Create a project

When you sign up on Neon, you’ll be prompted to create a project and a database like this:

Create Project page showing inputs for Project name, Database name, and Region

Once created, your project’s dashboard will contain a general overview of your database connection details, branches, compute, and storage data.

Neon dashboard showing Connection Details, Branches, Usage, and Storage details

Please take note of your database connection string because you will add it as an environment variable in your project.

Seed data

In your Neon project’s console sidebar, click on ‘SQL Editor’ and run the queries below to seed your database with some data:

    CREATE TABLE cars (
        id SERIAL PRIMARY KEY,
        car TEXT NOT NULL,
        model TEXT NOT NULL,
        created_at TIMESTAMPTZ DEFAULT NOW()
    );

    INSERT INTO cars (car, model)
    VALUES
        ('Toyota', 'Camry'),
        ('Honda', 'Civic'),
        ('Ford', 'Mustang'),
        ('Tesla', 'Model S'),
        ('Chevrolet', 'Silverado'),
        ('BMW', '3 Series'),
        ('Mercedes-Benz', 'E-Class'),
        ('Jeep', 'Wrangler'),
        ('Audi', 'Q5'),
        ('Subaru', 'Outback');
Enter fullscreen mode Exit fullscreen mode

This SQL script creates a cars table with columns for id, car, model, and created_at, then inserts the provided car makes and models into the table.

GIF demonstrating how to run an SQL script in Neon’s SQL editor

Accessing Neon’s database with serverless driver

Setup project

In your terminal, navigate to your project’s folder and start a new Node.js project by running the command below:

    mkdir neon-serverless-lambda
    cd neon-serverless-lambda
    npm init -y
Enter fullscreen mode Exit fullscreen mode

This will create a Node.js project with default values. Open your project in a code editor and update the package.json with the code below:

    {
      "name": "neon-serverless-lambda",
      "version": "1.0.0",
      "description": "",
      "main": "index.js",
      "type": "module",
      "scripts": {
        "test": "echo \\"Error: no test specified\\" && exit 1"
      },
      "keywords": [],
      "author": "",
      "license": "ISC"
    }
Enter fullscreen mode Exit fullscreen mode

Here, you added "type": "module" to enable EcmaScript’s import/export functionality.
In your project’s terminal, run the command below to install Neon’s serverless driver:

    npm install @neondatabase/serverless
Enter fullscreen mode Exit fullscreen mode

Write function

Open your project in a code editor, then create an index.js file and update it with the code below:

    // index.js (using `neon` object)
    import { neon } from '@neondatabase/serverless';

    export async function handler(event) {
      try {
        const sql = neon(process.env.DATABASE_URL);
        const rows = await sql('SELECT * from cars');

        console.log(rows);

        return {
          statusCode: 200,
          body: JSON.stringify({
            message: 'Successful',
            data: rows,
          }),
        };

      } catch (err) {
        console.log('Error', err);
      }
    }
Enter fullscreen mode Exit fullscreen mode

In the code above, you did the following:

  • Imported the neon object from @neondatabase/serverless. This object will allow you to connect to and run SQL queries on a Neon database using a connection string.
  • Created a handler() function with an event parameter.
  • In your handler() function, you wrote a try-catch block that attempts to establish a connection to the database using the neon object and the DATABASE_URL environment variable. It executes a SQL query (SELECT * from cars) to retrieve all rows from the "cars" table. Any errors during the database operation are logged using console.log().
  • After fetching the rows from the database, your function constructs and returns an HTTP response with a status code of 200 (indicating success) and a JSON body containing a success message ('Successful') along with the retrieved rows (data: rows).

Choosing between the Neon serverless driver’s neon or Pool/Client objects
If you want to execute multiple queries in a single connection, Neon’s Pool object uses WebSockets and can bring latencies down to 4ms once a connection is established. However, it is best to use the neon object for single-shot queries. Check out this blog post for more insights.

Prepare for deployment

In your PC’s file explorer, navigate to your project’s folder and zip all of its contents:

File explorer showing selected Neon code and .zip file

You will upload this .zip file to AWS while creating the Lambda.

Deploying to AWS

Create a Lambda function

In your browser, log into your AWS console, search for “Lambda,” and select it from the results:

AWS console with search results for Lambda

Click “Create function” and fill out the basic information for your new Lambda function:

Create function page with inputs for your Lambda’s basic information

After filling out the required basic information, scroll down and click “Create function” to finish creating your new Lambda function:

neon-serverless-lambda dashboard on AWS

Upload your code as a zip file

In your Lambda function’s dashboard, scroll down to “Code source”, click “Upload from”, and select “.zip file”:

Code source section in AWS Lambda

Navigate to your project’s folder and select the .zip file you created in the last section, then click “Save”:

AWS Lambda dashboard with .zip file selected

Set environment variables

In your Lambda function’s dashboard, click “Configuration”, then select “Environment variables” and click the “Edit” button:

Environment variables screen

In the “Edit environment variables” page, click “Add environment variable”, then input DATABASE_URL as the variable key and your Neon project’s database connection string as the value, then click “Save”:

Edit environment variables page with key and value inputs

Test your deployment

In your Lambda function, click “Test” to create a new test event, then fill out the event name, and click “Save”:

Test event screen with inputs to create a reusable test

After saving your test, click the “Test” button to run it. If it is successful, you will see something like this:

Successful test result on AWS Lambda

Creating and testing your API

Create an API with AWS API Gateway

In your Lambda’s “Function overview” section, click “Add trigger” and select “API Gateway”:

Add trigger page with API Gateway selected

Select “Create a new API”, specify “HTTP API” as the API type and the security mechanism as “Open”, then scroll down and click “Add”:

API Gateway Trigger configuration options

Once your API endpoint has been created, scroll down to “Triggers” to see its URL:

API Gateway trigger for neon-serverless-lambda showing its endpoint URL

Test your API endpoint

To quickly test your new endpoint, copy its URL and make a “GET” request in your preferred API testing platform:

Thunderclient UI to test API endpoint

Conclusion

Following this guide, you’ve had hands-on experience bridging the gap between AWS Lambda and Neon’s database using the Neon serverless driver. In this guide, you manually zipped and deployed your code; however, you can explore The Serverless Framework or AWS Serverless Application Model (SAM) for continuous integration and deployment.

Serverless computing can help you scale to meet enormous demands. When paired with a robust, scalable, serverless database service like Neon, the result is a seamless experience for your development team and end users. Neon also offers innovative features like database branching, autoscaling, and bottomless storage, saving you time and money. Click here to explore its free tier today.

Top comments (2)

Collapse
 
leob profile image
leob

Excellent post ... expert knowledge like this is the kind of content that I like to see on dev.to !

Collapse
 
dalu46 profile image
Dalu46

NICE READ