Introducing VeloxDB

This is a project I have been working on with some friends for some time. In March, we decided to leave our jobs and pursue it full-time, and now we have something to show.

We built a database called VeloxDB (Velox means speed in Latin). True to its name, VeloxDB is a fast database, with performance being its main focus. However, we have not sacrificed correctness for performance. Many high-performing databases make compromises in consistency and correctness, but VeloxDB is both fast and accurate. While we have made some other tradeoffs compared to traditional and NoSQL databases, correctness is not one of them.

What is VeloxDB

VeloxDB is a fast, object-oriented, open source in-memory database for C# with an emphasis on correctness.

Let me give a quick breakdown of all the terms involved:

• "Fast": The whole database is built with performance in mind. We achieve 2.5 million ACID write transactions with an AWS c6a.8xlarge compute instance.
• "Object-oriented": The DB offers an object-oriented interface, giving you an ORM-like experience. Your code is deployed within the database, reducing the overhead of communicating with the database to a minimum.
• "Open source": Source code for community version of VeloxDB is available under MIT License.
• "In-memory": The entire dataset has to fit in memory, so if you have a 30GB database, you need 30GB of RAM. The data is still persisted on disk, so there is no data loss if the server crashes.
• "For C#": The ORM API is primarily designed for C#, although it should work with any .NET language (F#/VB).
• "Emphasis on correctness": The database is fully ACID, with no eventual consistency.

What is it good for?

VeloxDB is suitable for a wide range of use cases that require high performance, low latency and a strong emphasis on consistency and correctness. Some of the most common use cases include:

• High-performance transactional applications: VeloxDB's in-memory architecture and strict serializability guarantees make it a great fit for applications that require high levels of read and write throughput.

• Real-time data processing: Its low latency and high performance make it an ideal choice for applications that require real-time data processing, such as financial trading systems, gaming servers, and IoT applications.

• Graph-like queries: Its object-oriented approach and specialized inverse reference index make it well-suited for applications that require complex, graph-like queries, such as social networks, recommendation systems, and fraud detection.

It's important to note that as an in-memory database, VeloxDB requires that the entire dataset fits into memory. While this allows for very fast access to data, it also means that the amount of available memory must be taken into consideration when designing and scaling a VeloxDB application.

The Code

Let's take a closer look at the code required to build something using VeloxDB. Keep in mind that this is just a brief overview of the process. The goal is to give you a better understanding of what working with VeloxDB entails. I will be omitting certain details, but if you are interested in trying to build something yourself, there is a comprehensive getting started guide available on our website.

To start, we will create a new C# class library and add the VeloxDB and VeloxDB.Protocol NuGet packages to it. Then, we will begin coding.

As an example, let's say we are building a simple blog application that includes a collection of posts. We will first define our model for these posts.

[DatabaseClass]
public abstract class Post : DatabaseObject
{
    [DatabaseProperty]
    public abstract string Title { get; set; }

    [DatabaseProperty]
    public abstract string Content { get; set; }
}

As you can see, models in VeloxDB are represented as classes, similar to how most ORMs function. To utilize the model, we create a DbAPI for it. DbAPI serves as a publicly accessible API through VeloxDB's protocol. Below is an example of a simple DbAPI that adds a new post to the database.

[DbAPI]
public class BlogApi
{
    [DbAPIOperation]
    public long CreatePost(ObjectModel om, PostDTO post)
    {
        Post newPost = om.CreateObject<Post>();
        newPost.Title = post.Title;
        newPost.Content = post.Content;
        return newPost.Id;
    }
}

Here we define a method, CreatePost, that can be called by a client. The method accepts PostDTO, which is a Data Transfer Object version of our Post class. There is nothing particularly noteworthy about it, so I have omitted it here. The ObjectModel is provided by VeloxDB and is used to access the database.

We now have a class library that defines a model and a simple operation to work with it. The easiest way to run it is by simply running the project. The VeloxDB NuGet package comes with a bundled VeloxDB server, which is used to run the project.

Behind the scenes

1. The VeloxDB server loads the compiled class library (dll)
2. The server scans the dll for any classes marked with the DbAPI and DatabaseClass attributes.
3. The server updates its internal data model based on the classes it found in the dll and makes the DbAPI methods publicly accessible through its protocol.
4. When a client calls a method in a DbAPI class, VeloxDB opens a new transaction, creates an ObjectModel and executes the DbAPI method.
5. The results of the method are then returned to the client.

Conclusion

The use of compiled dlls in VeloxDB allows for minimal overhead when executing DbAPI methods. This approach is similar to how stored procedures work in traditional SQL databases, but instead of using SQL procedures, VeloxDB utilizes .NET dlls.

Compared to traditional ORM-Relational database setups, VeloxDB offers a similar user experience with less overhead.

The object-oriented approach is also well-suited for graph-like queries. Algorithms such as graph searches, graph coloring, and shortest path queries are easier to implement with an OO database than with a traditional SQL database.

However, it is worth noting that SQL excels in certain use cases, such as ad-hoc queries, ease of use, and set operations. For this reason, we plan to add SQL support to VeloxDB in the future.

If this article has sparked your interest, please visit our GitHub page and our website for more information.

The database is still in beta, and any feedback is most welcome.