Imagine you have a big database/dataframe of tabular data. And you don't know enough SQL or doesn't know what the values are in the dataframe to make a good enough SQL query to do the task for you. This is where you wish you could ask something in natural language and hope AI got you covered to find the results for you. TableQA is one such product that gets you the result you want.
Suppose you are dealing with cancer death data. You have columns named as Year, Nationality, Gender, Age Group, Cancer site, Death Count. You could use TableQA to simply ask
"what is the maximum age of men having stomach cancer"
and get the result. As simple as that.
TableQA uses huggingface transformers under the hood and uses a combination of an AI-based entity extractor to generate key value pair mapping of column name and values, and custom trained classifiers to classify the aggregate type,i.e COUNT,MAX,MIN,SUM,AVG. From there on it builds the associated SQL query on its own, with a rule-based approach. The rules are pretty much of an advantage because unlike other AI products which rely heavily on the dataset, you can modify the blocks of rules so that you can get a more accurate result.
The use of such rules could be more explained by a schema.
Suppose you want to include custom keywords for the natural language query to detect the column to which it belongs, or you need keywords for any column values, you could add a schema which contains this info and make the performance better.
The output could be visualized as the result of a query, or a bar or pie chart based on the result, or you could get an SQL query so that you could use that on your own database.
The ultimate use-case and userbase are widespread. You could use this on many databases including MySQL, Postgres, SQLite, and the same even on Amazon RDS. You could even use this plugin under your chatbots, Heck yeah!
There are several features I would like to show you that tableQA is capable of supporting. But enough talk, let's get coding! Feel free to check out the source code and also there is a colab example for you to try out!
Feel free to mail me at abhijithneilabrahampk@gmail.com for any questions.
Top comments (0)