Sergey Nikolaev

Posted on Aug 7, 2023 • Edited on Nov 6 • Originally published at manticoresearch.com

Manticore Search 6.2.0

We're continuing the Manticore 6 series with a new major release - Manticore Search 6.2.0. Here's what's new:

Integrations

mysqldump for logical backups

In past versions, we created the manticore-backup tool and the BACKUP SQL command. These are great for physical backup, but logical backup is often just as important. mysqldump, a well-known and trusted tool for logical backups from the MySQL ecosystem, now works with Manticore. Since Manticore tightly integrates with MySQL, you can now use the mysql client to log in to Manticore, and mysqldump to backup Manticore tables.

Grafana and Apache Superset

Grafana and Apache Superset are two excellent platforms for visualizing data. Starting with this release, you can use them to visualize Manticore. Read more on our blog: Grafana, Apache Superset.

HeidiSQL and DBForge

If you prefer GUI over command-line tools for Manticore access, you can now use HeidiSQL and DBForge.

Fluentbit and Vector

You can now use not just Logstash and Filebeat to upload data to Manticore, but also Fluentbit and Vector by Datadog. Read more on our blog

Performance

COUNT(DISTINCT) is now much quicker thanks to the use of hash tables and HyperLogLog.
select count(*) should now be much faster in many cases due to closer integration with secondary indexes.
Queries with secondary indexes can now be run in parallel. The number of threads is limited to the number of physical CPU cores, which should greatly improve query speed.
Performance of pseudo_sharding which is ON by default has been significantly boosted in case of high concurrency workloads due to smarter thread utilization. According to our tests, the 99p response time under high concurrency is now a magnitude lower, e.g.: before, after.
Users can now choose the default attribute storage engine through configuration settings, for better customization to fit specific needs.
The logic of the cost-based query optimizer has been improved, and the default histogram resolution is now set to 8k for better accuracy with randomly distributed values. It also more precisely estimates the complexity of applying filters over string attributes.

/cli

The /cli endpoint has been updated, making it even easier to manage Manticore through a browser.

Buddy PHP plugins

You can now create your own Manticore Search SQL/JSON query as a plugin, publish it on packagist.org, and install it using the CREATE PLUGIN SQL command.

Packaging

Language packs for English, German, and Russian can now be easily installed on Linux with the command apt/yum install manticore-language-packs. On macOS, use brew install manticoresoftware/tap/manticore-language-packs.
New Homebrew formulas for quicker installation.

CI changes

We've begun using GitHub workflows, making it easier for contributors to use the same Continuous Integration (CI) process as the core team when preparing packages. All jobs can be run on GitHub-hosted runners, which allows for simple testing of changes in your Manticore Search fork.
We've started using CLT to test complex situations. For instance, we can now make sure that a package built after a commit can be installed across all supported Linux operating systems. The Command Line Tester (CLT) offers an easy way to record and replay tests.