DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’»

Cover image for How to setup Clouseau for  CouchDB search on Unix-like systems
Jordan Soo Yen Yih
Jordan Soo Yen Yih

Posted on

How to setup Clouseau for CouchDB search on Unix-like systems

Introduction

Clouseau is written in Scala code and provides access to the Lucene library. You can use Lucene Query Syntax to run search queries. In previous article I did talk about we can query in Apache CouchDB with Clouseau, today I would like to talk about how to setup Clouseau that link with CouchDB.


PrerequisitesπŸ“œ

Before you begin this guide, you should have a regular, non-root user with sudo privileges configured on your server. Additionally, you will need to have a up and running CouchDB version v3+ on your server. You can learn how to install Apache CouchDB on Unix-like systems by following the official documentation here. Besides that, since Clouseau is written in Scala, we also need to install Java to run Clouseau.

The search plugin is runtime-compatible with Java JDKs 6, 7 and 8. Building a release from source requires JDK 6. It will not work with any newer version of Java.

In this tutorial, I am using Ubuntu 20.04, JDK 8 and CouchDB v3.2.1 . CouchDB v3 and higher is a must as CouchDB v3 has extremely simplified the setup of Lucene search.
server
initial_couchdb_status

lets_begin

1. Download Clouseau

You can download the latest Clouseau release from here. Your Clouseau might be different depending on the time you download. My current latest version is Clouseau 2.17.0. I am using wget to download from my Ubuntu server.

sudo wget https://github.com/cloudant-labs/clouseau/releases/download/2.17.0/clouseau-2.17.0-dist.zip
Enter fullscreen mode Exit fullscreen mode

2. Extract Clouseau zip

We are using unzip to extract the Clouseau jar files.

Tips: If you don't have unzip, you can run the command sudo apt-get install unzip for Ubuntu.

Then, we can unzip it with the command below:

sudo unzip clouseau-2.17.0-dist.zip -d /opt/
Enter fullscreen mode Exit fullscreen mode

As you can see, we extract the Clouseau to the destination /opt/
. Now we can navigate to the extracted destination and view the extracted files.

cd /opt/clouseau-2.17.0/
Enter fullscreen mode Exit fullscreen mode

3. Create Configuration files

According to CouchDB official documentation, in order to setup the search plugin, there are 2 configuration files required called clouseau.ini and log4j.properties with the following content:

clouseau.ini

[clouseau]

; the name of the Erlang node created by the service, leave this unchanged
name=clouseau@127.0.0.1

; set this to the same distributed Erlang cookie used by the CouchDB nodes
cookie=monster

; the path where you would like to store the search index files
dir=/opt/couchdb/share/

; the number of search indexes that can be open simultaneously
max_indexes_open=500
Enter fullscreen mode Exit fullscreen mode

log4j.properties

log4j.rootLogger=debug, CONSOLE
log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender
log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout
log4j.appender.CONSOLE.layout.ConversionPattern=%d{ISO8601} %c [%p] %m%n
Enter fullscreen mode Exit fullscreen mode

4. Create a script to run Clouseau

Below is the script to run Clouseau. If you have a Java background, below syntax should be familiar for you. In this case, I will place the script file under /opt/clouseau-2.17.0/bin/ and the file name called clouseau, full path would be /opt/clouseau-2.17.0/bin/clouseau

#!/bin/sh
/usr/bin/java -server \
     -Xmx2G \
     -Dsun.net.inetaddr.ttl=30 \
     -Dsun.net.inetaddr.negative.ttl=30 \
     -Dlog4j.configuration=file:/opt/clouseau-2.17.0/log4j.properties \
     -XX:OnOutOfMemoryError="kill -9 %p" \
     -XX:+UseConcMarkSweepGC \
     -XX:+CMSParallelRemarkEnabled \
     -classpath '/opt/clouseau-2.17.0/*' \
     com.cloudant.clouseau.Main \
     /opt/clouseau-2.17.0/clouseau.ini
Enter fullscreen mode Exit fullscreen mode

Your path could be different, it depends on where you put the clouseau files, you could change the /opt/clouseau-2.17.0/ if needed, in this case, I'm putting all the clouseau files and configurations under /opt/clouseau-2.17.0 directories.

We are almost there. Now we want to tell Ubuntu to run the service when the server startup. To do that, we can navigate to /lib/systemd/system

cd /lib/systemd/system
Enter fullscreen mode Exit fullscreen mode

and create a file called clouseau.service with the content below:

[Unit]
Description=clouseau service

[Service]
WorkingDirectory=/opt/clouseau-2.17.0
ExecStart=/opt/clouseau-2.17.0/bin/clouseau
Restart=always
RestartSec=3
User=root

[Install]
WantedBy=multi-user.target
Enter fullscreen mode Exit fullscreen mode

Once created, then run the 3 commands below:

sudo systemctl daemon-reload
Enter fullscreen mode Exit fullscreen mode
sudo systemctl enable clouseau.service
Enter fullscreen mode Exit fullscreen mode
sudo systemctl start clouseau
Enter fullscreen mode Exit fullscreen mode

Then, Clouseau should already be up and running. Check with the systemd init system to make sure the service is running by typing

sudo systemctl status apache2
Enter fullscreen mode Exit fullscreen mode

Finally, we can check whether our Apache CouchDB and Clouseau is connected by using CouchDB root endpoint.

curl 127.0.0.1:5984
Enter fullscreen mode Exit fullscreen mode

final_couchdb_status
If Clouseau is connected to CouchDB successfully, You should see the search keyword display under features.

That's it, we did it~πŸŽ‰

hooray
Now we can query with Lucene Query Syntax in CouchDB.
Hope you find this guide useful.πŸ‘‹

Top comments (18)

Collapse
 
efinley1272 profile image
efinley1272

I followed this procedure (used version 2.21.1 instead of 2.17), but didn't get "search" in the "features" output of couchdb. clouseau is running as verified by systemctl status clouseau, but I don't get any logging from clouseau to troubleshoot because of:
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation

I'm flailing in the dark when it comes to Java. Any pointers would be appreciated.

Collapse
 
yenyih profile image
Jordan Soo Yen Yih • Edited on

Hi efinley, May I know what CouchDB version you are using? And, did you create any search index? If yes, you may try to query with the search index, because recently I have experienced the same issue on one of my CouchDB server and I need to query with the search index one time then only the "search" will show in the "features". Let me know whether it works for you.

Collapse
 
efinley1272 profile image
efinley1272

I'm using version 2.21.1. I have NOT tried creating a search index and searching with it. I will try that and report back.

Thread Thread
 
yenyih profile image
Jordan Soo Yen Yih

oh... you can't use CouchDB v2 and lower. You must use CouchDB v3 and higher as I said in the article above

Thread Thread
 
efinley1272 profile image
efinley1272

Sorry, I mean the version of Clouseau is 2.21.1 the version of CouchDB is 3.2.2.

Thread Thread
 
yenyih profile image
Jordan Soo Yen Yih

I see. Then you may try to create a search index and query it.

Thread Thread
 
efinley1272 profile image
efinley1272

Sorry for taking so long to reply. Life got crazy and I just got back to it today. I added a database 'test', then the 'search' feature showed up. I did NOT create a search index or even add any documents to the 'test' database. I did NOT check for the search feature today before adding the test database so I'm not sure whether it was adding the database or just waiting that made the feature show up.

Thread Thread
 
yenyih profile image
Jordan Soo Yen Yih

I see. Do you mind sharing the server spec you are using? This is because from my experience, low server spec is another possibility that will cause the search feature late to show up.

Thread Thread
 
efinley1272 profile image
efinley1272

This is running inside an LXD container, which is running in a Ubuntu virtual machine with 16GB RAM and 4 processors, which is running on bare metal hardware with 88 processors and 1.5TB RAM. It's the only VM on that hardware, so no contention for resources.

Interesting data point, I haven't done anything AT ALL with the database since I last posted that the 'search' feature showed up. When I queried it today, the 'search' feature is missing again. This is really depressing. Any pointers on getting logging working for Clouseau 2.21.1 would be greatly appreciated.

Thread Thread
 
efinley1272 profile image
efinley1272

I'm replying to my own comment because I have some relevant data points. I was able to get logging working by putting slf4j-simple.jar in the classpath as mentioned in some of the other comments. The search feature showed up and started working after creating a search document and adding a record. I appreciate all the help from this blog and from Jordan's comments. Thank you.

I'm very much looking forward to Nouveau. Anyone know the timeline on that?

Collapse
 
okeribok profile image
Mies Buis • Edited on

Thanks for the tut! Hoping someone could help me, because I get

{"error":"ou_est_clouseau","reason":"Could not connect to the Clouseau Java service at clouseau@127.0.0.1"}
Enter fullscreen mode Exit fullscreen mode

I am running couchdb 3.2.2 and clouseau 2.21.2. The systemctl status is "available" and I fixed a slf4j error by placing a slf4j-simple jar in the classpath. I built a simple index (as discussed here), rebooted a couple of times, but: nothing. The log in systemctl status gives me:

INFO scalang.node.ErlangHandler - channel disconnected org.jboss.blablabla
ERROR scalang.node.ServerHandshakeHandler - Channel closed during handshake
Enter fullscreen mode Exit fullscreen mode

Can anybody help me find clouseau?

Collapse
 
yenyih profile image
Jordan Soo Yen Yih

Hi... Seems like your CouchDB can't find Clouseau. When you curl 127.0.0.1:5984 , it didn't show the Search feature right?

Collapse
 
okeribok profile image
Mies Buis

Thanks. That is correct. I do not see the search feature when I curl CouchDB, but I was not worried because of what I read in these comments. I also managed to solve the fact that Clouseau moved to another logging solution, which is not in this tutorial. (I added an slf4j-simple.jar in my Clouseau dir). The log for Clouseau shows everything copacetic untill I try to use search. Then the connection drops and times out in the browser with the "ou est clouseau" error.
I am thinking of moving to a "real" Solr installation instead. This also gives me Lucene 9 whereas Clouseau is Lucene 6 I think. I have not looked into the differences yet.
Hoping some CouchDB devs get around to actually including FTS and all nice features such as true faceting in CouchDB 4.

Thread Thread
 
yenyih profile image
Jordan Soo Yen Yih

According to CouchDB devs latest update, there is no CouchDB 4 yet and one thing confirmed that there are no new features in CouchDB 4. But the good news is, they are currently working on some new features for CouchDB v3.3.0. One of the new features is the new Full-text search for CouchDB called Nouveau which supports Lucene 9 + DropWizard. github.com/rnewson/nouveau . If everything goes well according to the plan, then we can have the new full-text search in CouchDB v3.3.0😊

Thread Thread
 
okeribok profile image
Mies Buis

That is excellent news! When I looked further, there was talk about moving CouchDB on top of FoundationDB, which might have been the "4" I had seen "somewhere", but apparently IBM would have funded the move and pulled the plug this march.
I will look into Nouveau and hope an official release comes soonish. Some QOL improvements in Fauxton would be nice too.

Thread Thread
 
yenyih profile image
Jordan Soo Yen Yih

Yup, you are right, they did discuss about that before. But last week CouchDB Berlin Meetup, Janl just announced that no FoundationDB for CouchDB 4. If you ever heard about it before, then just pretend nothing happened or forget about it. 🀣

Collapse
 
cortescasado profile image
Alvaro Cortes • Edited on

Hi Jordan, thanks for your tutorial, is by far clearer than official documentation installing, however I don't get the search working.
The clouseau.service loads right but it fails when I try to start it without any log message. Any idea?

Collapse
 
yenyih profile image
Jordan Soo Yen Yih • Edited on

Hi Alvaro Cortes, did you try to use systemctl status clouseau.service to check the clouseau status? Btw, I ever experience that clouseau service slow to bootup due to server instance resources not enough, in this scenario, clouseau didn't throw any error message and when I checked on the CouchDB features list, there is no "search" feature show up. But after a time of period, It did appeared and was connected successful. I am not sure whether it is the same issue you have encountered, but you may try to use systemctl or check on CouchDB logs see any hints from there.

Create an Account! πŸ‘€ Just want to lurk?
Β 
You can still create an account and turn on features like 🌚 dark mode.