SirixDB - A versioned NoSQL data store for XML and JSON 🦁

Feel free to contribute on GitHub 💚

Hi all,

there's plenty of possibilties to work on a versioned document store, which preserves every revision of your data in a space-efficient manner. 🐬

SirixDB basically only stores changes between revisions, but any revison is indexed and can be reconstructed in O(n), whereas n is the number of nodes in the revision.

If you're eager to participate in #Hacktoberfest there are plenty of opportunities. For instance working on a just started frontend, the backend, a non-blocking REST API built with Vert.x, Kotlin and Keykloak, or a query optimizer (for instance to rewrite the AST based on cost-based rules). Or you might want to look into how to best horizontally scale SirixDB for instance via a distributed log (Apache BookKeeper?!) 😃

🐣 Frontend with Vue.js, TypeScript and D3.js

With the help of the community I just started a web frontend based on Vue.js, TypeScript and in the future D3.js a couple of days ago.

There are a few issues I created, but basically we start completely from scratch, so we can use the best technologies which fit our needs:

Issues for the frontend gathered in our Community Forum

In general I'd love to provide a similar space filling approach (modified Sunburst-View) as the one I once developed for my master's thesis:

🚀 Backend written in Java and a non-blocking REST-API built with Vert.x and Kotlin

If you're more into backend stuff, you can either work directly on issues regarding SirixDB or work on Kotlin based stuff with Vert.x.

For instance Web Socket support in the non-blocking HTTP-Server:

Websockets to stream resource changes into the client / browser #98

JohannesLichtenberger posted on Sep 28, 2019

I'd put this into the rest-api module and build it with Vert.x and Kotlin.

View on GitHub

OAuth2 authorization code flow:

OAuth2 auhtorization code flow for web applications #96

JohannesLichtenberger posted on Sep 27, 2019

Provide a configurable mechanism to switch between the authorization methods and implement the authorization code flow in SirixVerticle in the REST API.

View on GitHub

Try to create a native image with the GraalVM (however Java 11 or soon 13 might not be supported, though :()

Create native images with the GraalVM #77

JohannesLichtenberger posted on May 08, 2019

For a fast startup time in containers... peek performance however might not be as good.

View on GitHub

A Kotlin-based DSL for SirixDB:

Kotlin based wrapper APIs #63

JohannesLichtenberger posted on Dec 30, 2018

Therefore we also could create a nice DSL.

But in essence a basic version for the XdmNode-Transaction layer for instance would involve infix functions trx moveTo 3 and alike, removing builders, just use default parameters...

However, I'd love to discuss this with Kotlin experts :-)

View on GitHub

Maybe we can even convert the sirix-xquery bundle to Kotlin, as it provides a more high level API. That said, I'd love to change some things regarding the API:

Kotlin based wrapper APIs #63

JohannesLichtenberger posted on Dec 30, 2018

Therefore we also could create a nice DSL.

But in essence a basic version for the XdmNode-Transaction layer for instance would involve infix functions trx moveTo 3 and alike, removing builders, just use default parameters...

However, I'd love to discuss this with Kotlin experts :-)

View on GitHub

Query optimizations

Or if you would like to work on query optimizations, you can look into how to rewrite the AST produced by Brackit:

JSON with a database storage backend #5

JohannesLichtenberger posted on Jan 24, 2019

Hey Sebastian, sorry to bother you all the time, but maybe you can also give a good tip how you'd implement a binding for a native JSON storage backend. I'm currently finishing the work on a temporal JSON storage layer besides XML for my open source project (http://sirix.io) and would like to also use Brackit. Maybe I could also implement the interfaces you already provide (should be basically the same as for my XDM node transactions) and then implement some functions for opening a JSON resource...

Basically my JSON node layer looks like the one in Marklogic: https://docs.marklogic.com/guide/app-dev/json

View on GitHub

We have plently of optimization potential here, as we currently don't take any storage inherent optimizations into account 👍

This will increase the performance of certain queries considerably.

A query looks like this:

let $doc := jn:open('database','resource', xs:dateTime('2019-04-13T16:24:27Z'))
let $statuses := $doc=>statuses
let $foundStatus := for $status in bit:array-values($statuses)
  let $dateTimeCreated := xs:dateTime($status=>created_at)
  where $dateTimeCreated > xs:dateTime("2018-02-01T00:00:00")
        and not(exists(jn:previous($status)))
  order by $dateTimeCreated
  return $status
return {"revision": sdb:revision($foundStatus), $foundStatus{text}}

Have a look at Sebastian's thesis Separating Key Concernsin Query Processing

He worked under supervision of Theo Härder, who invented ACID transactions 👌

👾 Think about how to horizontally scale SirixDB

I've thought about it and would love to distribute the transaction log with Apache BookKeeper or Apache Pulsar (which uses Apache BookKeeper). It might have some advantages in comparison to Apache Kafka.

❤️ Happy coding

So, there's plenty of stuff to work on. It' a super diverse field. For sure beginners are invited to work on stuff as I'm myself a Frontend beginner for instance and I have not too much knowledge about query optimizers. However, I'm more than eager to learn... 👋

I hope the project is useful for some of you and I even got my first contributions after years of working alone, that's simply awesome. Thank you so much 😎

And if you like the project you might share it on twitter and so and and spread the word!? 🙈

Contribute on GitHub SirixDB and GitHub SirixDB Web Frontend 💚

Have a nice day and a great #Hacktoberfest
Johannes