🎉One must-have tool for anyone in Data field 🤯✨

#productivity #opensource #programming

TL;DR

SQL databases are great for handling structured data and complex queries, but they fall short when performing complex calculations and optimizing performance. 🐢

I am not sure if you are ready to read this. 🤐 I think most of you might be shocked by reading this.

✨ There is a faster and probably better alternative to SQL 😮

I recently came across this, and let me tell you one thing: 'It is really good!' Let me introduce you to esProc. 🏃‍♂️💨

esProc SPL is an open-source next-gen data processing language that can surpass the limitations of SQL and boost your data processing. 🤩

What even is SPL? 🤔

💡 SPL short for Structured Process Language is software for computing structured/semi-structured data.

I am sure many of you may not know about SPL. There is just a letter's difference between SQL and SPL, yet they vary in many ways. 😵‍💫

SPL uses a grid-based 📊 syntax instead of text, making it more visual and concise. As a data computing engine, SPL can achieve higher performance 💨 compared to traditional SQL.

SPL is specially dedicated to low code 🧑‍💻 and high performance. At its core, it is software comprising three major pieces:

Programming Language 🚀: SPL is a language that facilitates the process of structured data computing, and has excellent performance in handling most computing scenarios.

Data Computing Middleware 🤝: SPL provides independent computation ability independent of databases. It can perform various structured data computations on its own and is easily embeddable in applications. 📱

Big Data Computing Platform 📈: SPL has its cluster computing system for large data, providing a distributed environment for programmers to control task distribution and implement efficient algorithms. 🧠

👉 To learn more about SPL, follow this link.

How SPL can be an alternative to SQL? 🧐

Before talking about whether SPL can be an alternative to SQL, let's first take a look into some limitations of SQL.

SQL queries grow in complexity fairly quickly 🧵

SQL is considered a pretty easy language to write. It often reads like plain English but it is not always the case. 😓

When writing nested queries, it grows in complexity and even a senior developer 🧑‍💻 finds it hard to make sense of it.

Here's a simple SQL query to calculate the maximum number of consecutive days that a stock price keeps rising. See how quickly the SQL statement is harder to read. 😵‍💫

SELECT MAX(consecutive_day)
FROM (
    SELECT COUNT(*) as consecutive_day
    FROM (
        SELECT SUM(rise_mark) OVER (ORDER BY trade_date) AS days_no_gain
        FROM (
            SELECT trade_date,
                   CASE WHEN closing_price > LAG(closing_price) OVER (ORDER BY trade_date)
                        THEN 0 ELSE 1 END AS rise_mark
            FROM stock_price
        )
    )
    GROUP BY days_no_gain
);

SQL can be super slow sometimes 💩

If you have ever worked in a real-world situation, you likely had to perform queries on a database containing thousands or even millions of entries.

With such a vast amount of data, SQL often performs poorly in such situations.

Let's take an example of this below SQL statement 👇

SELECT TOP 10 COLUMN_NAME FROM TABLE_NAME ORDER BY COLUMN_NAME ASC;

This code in itself is not complex but SQL has to perform the following operations to return the final data.

First, sort the entire data.

From the entire sorted dataset, select the top 10.

Rewriting the previous SQL query in SPL will provide a clear idea of ease when using SPL. To calculate the maximum consecutive days that a stock keeps rising 👇

stock_price.sort(trade_date).group@i(closing_price<closing_price[-1]).max(~.len())

Although the logic is the same, it is much easier to read/write and is a lot more intuitive. 😄

Finally, what is esProc SPL? 🤩

💡 esProc is a scripting language for data processing with rich library functions and powerful syntax.

So now you have a basic understanding of SPL, let's discuss what esProc actually is.

Now that you're aware that SPL is an alternative to SQL, think of esProc as an even superior alternative to traditional SPL with added functionalities. 😲

esProc is a pure Java written that makes it easy to add powerful data processing capabilities to your Java 🍵 applications but non-Java applications can invoke esProc through RESTful APIs 🎯.

It provides structured data types and functions 👷 that let you perform operations like filtering, grouping, joining 🪢, and many other computations - similar to what you can do with SQL - but using simple Java code. ✨ esProc implements the basic class libraries entirely within itself.

🤔 What platforms can esProc run?

As it is built purely in Java, it can run smoothly in any OS equipped with JVM (Java Virtual Machine), cloud servers, or even containers. 😎

✨ In short, esProc is blazing fast, independent, and provides advanced scripting capabilities for data handling and analysis over traditional SPL.

Let's Wrap Up! ✨

By now, you have a generic understanding of what SPL is. What esProc is and how can it come in handy in situations, and whether or not you should replace SQL with SPL or esProc.

After reading this article, if you feel that esProc is a great project with huge potential, don't forget to give it a star!

🌟 esProc on GitHub