Neptune Serverless - "is it really Serverless?"

This is my longform article from a recent tweet thread covering some initial thoughts on "Amazon Neptune Serverless".

Table of Contents

 1. Neptune Serverless
 2. Serverless means "Scale to Zero"
       2.1. Neptune Serverless does not Scale to Zero.
 3. Comparison to other AWS Serverless Databases
       3.2. DynamoDB
       3.3. Aurora Serverless
 4. Operational Concerns for a Serverless database?
 5. Working around the (lack of) Scale to Zero
 6. Serverless or "Auto Scaling"?
 7. Neptune Serverless Use-cases

Neptune Serverless

Amazon recently introduced "Neptune Serverless".

You can see the full details on the launch blog.

In a nutshell - Neptune is a managed Graph Database from AWS and this "Serverless" offering autoscales based on demand so you are not stuck with the initial provisioned capacity.

BUT... the very first thought on reading the announcmeent was "Is it REALLY Serverless"?

Serverless means "Scale to Zero"

One of the key benefits of Serverless for me is "Scale to Zero" where the service is only running when there is actual load or the service is performing some workload. Lambda for example is a serverless compute platform. After the workload is completed (or the timeout of 15 minutes has passed), that specific Lambda execution is complete. You are no longer billed after that and there is nothing to do in terms of shutting down etc. It's upto the Cloud provider to manage the compute environment - whether they keep it around for reuse or delete it and assign the underlying hardware capacity elsewhere.

Neptune Serverless does not Scale to Zero.

As of now there is a minimum capacity of 2.5 NCU (Neptune Capacity Units - a combination of some amount of Compute CPU's and Memory).

The immediate impact is that there is a minimum charge of about USD 290 a month even if you did not perform a single query on the database.

Comparison to other AWS Serverless Databases

DynamoDB

A classic example of a serverless database is Amazon DynamoDB with the "Pay Per Use" mode where you only pay for active API Calls / Queries. You do not pay for anything when there is no usage of the database. But this is a key design advantage of a system like DynamoDB where you do not manage anything related to the database instances and only specify tables that you want to use. All the heavy lifting of managing capacity is AWS's responsibility and they can optimize the usage to allow the system to seeming appear that its shutdown by moving workloads around their servers.

Aurora Serverless

There are two other "serverless" databases from AWS which use more "traditional" database engines - Aurora Serverless v1 and Aurora Serverless v2 which are a lot more similar to Neptune than DynamoDB would be.

Aurora serverless v1 has a "Pause and Resume" feature where you can set a timeout after which the database would automatically shutdown. One of the more common use cases that I see for such databases are dev / testing systems. Once the development team has finished their active testing, the database shuts itself down and spins back up whenever testing resumes. This can save costs for the consumer and for the cloud provider, they can re-use or optimize the capacity elsewhere. This also helps avoid wastage from a sustainability perspective.

While Aurora Serverless v1 had this key feature, it struggled in other areas like scaling points and Aurora Serverless v2 improves on these limitations a LOT.

BUT... Aurora Serverless v2 removed this "Pause and Resume" feature. The smallest Aurora Serverless v2 capacity that you can define is 0.5 ACUs (Aurora Compute Units - a combination of some amount of Compute CPU's and Memory).

Keeping a minimum amount of compute around has a key benefit - there is no latency incurred in starting up the database back from its paused state. With Aurora Serverless v1, this used to be a few minutes. No "Pause feature" means databases like Aurora Serverless v2 and Neptune Serverless do not have this latency as they are ready to go anytime they are running.

Operational Concerns for a Serverless database?

There are other operational concerns with these type of Serverless databases too. For example, I tried to spin up a Neptune Serverless database in my test account and immediately hit an issue.

Before I could create the database, I needed to first setup a VPC with 2 subnets.

One of the other things you need to specify when you build the database is the Neptune engine version. This means in the future, you have to deal with upgrading engines.

Serverless systems need to abstract you away from networking concerns and operational concerns like managing software versions. One could argue that managing engine versions comes with having to deal with new features / breaking changes etc but I do not remember a time when I had to upgrade DynamoDB to take into account a new feature. Where Amazon is tracking compatibility to PostgreSQL / MYSQL features - version management may be envitable. But with a proprietary engine like Neptune - maybe this is something that should not require user level management.

Working around the (lack of) Scale to Zero

There is a dirty workaround to this scale to zero problem.
You build it yourself!

The Neptune database allows you to shutdown and startup the database programmatically. I tested this via the AWS CLI.

So you could schedule the database to shutdown automatically using a simple CloudWatch Trigger and a Lambda that calls this API.

AWS themselves offer a prebuilt "solution" called the Instance Scheduler

Using a combination of CloudWatch, DynamoDB and Lambda - the system can stop / start databases on a schedule. You can use resource tags to control schedules, exclude resources etc.

Now this prebuilt solution was built for fixed schedules and not really handle "Dynamic" setup like "Shutdown after 10 minutes of no activity". Checking the documentation, CloudWatch has CPUUtilization metrics for Neptune Databases and we could build this ourselves.

Shutting down on a schedule or when idle seem easy enough but starting up when there is a request looks a lot harder.

How do we even know that there is a database request pending?
One way is to encode intelligence into the Application as part of the connection handling logic. Use the API again to obtain the DBInstanceStatus and if database is shutdown, issue the command to start it up. It does take a few minutes for the database to fully start up - so the latency has to also be handled where the application waits till the database is fully up.

Serverless or "Auto Scaling"?

Aurora Serverless v2 and Neptune Serverless are answers to a totally different problem than scaling to zero. They are addresssing "provisioning for peak volume".

With traditional Aurora Or Neptune databases - you are setting a predefined instance type with a fixed vCPU Compute and Memory limit. Scaling beyond the limits of the instance type needs thinking differently and making the system scale in terms of "Compute Unit bundles" between a minimum and maximum threshold is what these serverless offerings bring.

In my mind this is basically "Autoscaling".

Autoscaling is used more when referring to "horizontal scaling" (adding more units of databases) rather than in "vertical scaling" (adding more cpu and memory to the existing database instance) and maybe that is one of the reasons why this is not called that.

Neptune Serverless Use-cases

Usecases for me for Neptune Serverless would be

• Consistent usage through the entire day with infrequent "Bursty" workloads. Classic Example : "we need X amount of cpu /memory during the day for normal queries and then 4X that amount for batch based workloads in the evening".
• Latency / wait to start database back up is not acceptable
• Dev / Test databases for these usecases in combination with my "poor man's scale to zero" workaround.

I think AWS should stop calling these systems "Serverless" and I look forward to the day when we can have true scale to zero / fast scale up /out systems like DynamoDB.

Will we see a "DynamoDB - Graph edition" at re:Invent sometime soon?