Dmytro Werner

Posted on Sep 28, 2022

Searching in JAVA: Elasticsearch vs MySQL vs Hibernate

#java #elasticsearch #mysql #hibernate

Hello folks :)

I was thinking lately: if I have logic for searching data in frontend and JAVA for backend, which technology should be used to find data? In this case some search engine would be a good idea. But is it easy to implement? I wanted to try a couple things and compare, which option is the best.
Learning by doing I understood how it works and I´d like to share this experience.

JAVA

First of all we need to create our JAVA application. I used the "Initializr" from Spring to get basic configuration for my project. Spring Initializr

Spoiler!:-D
My end configuration looks like this (Java version 18):

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.7.3</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.tutorial</groupId>
    <artifactId>search</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>search</name>
    <description>Tutorial project for search engine</description>
    <properties>
        <java.version>18</java.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-data-jpa</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.hibernate.search</groupId>
            <artifactId>hibernate-search-mapper-orm</artifactId>
            <version>6.1.1.Final</version>
        </dependency>
        <dependency>
            <groupId>org.hibernate.search</groupId>
            <artifactId>hibernate-search-backend-lucene</artifactId>
            <version>6.1.1.Final</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.data</groupId>
            <artifactId>spring-data-elasticsearch</artifactId>
            <version>4.0.0.RELEASE</version>
        </dependency>
        <dependency>
            <groupId>co.elastic.clients</groupId>
            <artifactId>elasticsearch-java</artifactId>
            <version>8.3.1</version>
        </dependency>

        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>2.12.3</version>
        </dependency>

        <dependency>
            <groupId>jakarta.json</groupId>
            <artifactId>jakarta.json-api</artifactId>
            <version>2.0.1</version>
        </dependency>

        <dependency>
            <groupId>com.h2database</groupId>
            <artifactId>h2</artifactId>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

Then just install all dependencies and we are good to move forward.

mvn install or mvn package

MySQL

Ok, now we need our database and some data. To fetch data we need to have/install mysql to local machine. To make it easier, we could install Workbench
to get also GUI. We need to put information about database(username, password, database url) in application.properties as well.

spring.datasource.url={url_path}
spring.datasource.username={database_username}
spring.datasource.password={database_password}
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver

Now just type Book model. I also added the annotations we need for our search engines.

Important

A table named book should exist in your database and should include some data for test.

@Entity
@Indexed
@Table(name = "book")
@Document(indexName = "books", type = "book")
public class Book {
    @Id
    private int id;

    @FullTextField
    @Field(type = FieldType.Text, name = "name")
    private String name;


    @FullTextField
    @Field(type = FieldType.Text, name = "isbn")
    private String isbn;

    public int getId() {
        return id;
    }

    public String getIsbn() {
        return isbn;
    }

    public String getName() {
        return name;
    }

    public void setIsbn(String isbn) {
        this.isbn = isbn;
    }

    public void setName(String name) {
        this.name = name;
    }
}

Now to get response from request we need Repository class.

public interface BookRepository extends PagingAndSortingRepository<Book, Integer>
{
    List<Book> findByName(String name);    
    List<Book> findByIsbn(String isbn);    
}

And now to get data from url we need a controller. I keep it very simple just to get response and check if data was loaded.

@RestController
@RequestMapping(value = "book")
public class BookController {    
    @Autowired
    BookRepository bookRepository;

    @GetMapping("/database")
    ResponseEntity<Iterable<Book>> getBooksFromDatabase(@RequestParam("query") String query)
    {
        Iterable<Book> booksFromDatabase = bookRepository.findByIsbn(query);

        return ResponseEntity.ok(booksFromDatabase);
    }
}

Ok, it seems to be done with database.

Was not so hard, but this is not most flexible option to interact with a searching flow.

Let´s take a look at the searching engines. We will start with very powerful technology in JAVA.

Hibernate

First we need to create configuration file.

@Component
public class BookIndexConfiguration implements CommandLineRunner
{
    @PersistenceContext
    EntityManager entityManager;

    @Override
    @Transactional(readOnly = true)
    public void run(String ...args) throws Exception
    {
        SearchSession searchSession = Search.session(entityManager);
        MassIndexer indexer = searchSession.massIndexer(Book.class);
        indexer.startAndWait();
    }
}

Over here the indexer from hibernate will create index with data from our model once the application was started. To explore books in index we also need service class. Ok, let's create one.

@Service
public class BookSearchService {
    @PersistenceContext
    EntityManager entityManager;

    @Transactional(readOnly = true)
    public Iterable<Book> search(Pageable pageable, String query)
    {
        SearchSession session = Search.session(entityManager);

        SearchResult<Book> result = session.search(Book.class)
            .where(
                    f -> f.match().
                    fields("name", "isbn").
                    matching(query)
                )
            .fetch((int) pageable.getOffset(), pageable.getPageSize())
        ;
        return result.hits();
    }
}

We have search function that loads a current session and checks in index, if the query matches to the name or isbn. If some objects were found, it will be returned as Iterable of Book objects

And thats it! Was it even easier that typical database call?

Now we need to configure a different search engine.

Elasticsearch

Ok, now we need a little bit more configuration ~~or maybe not~~. Just like in the case with MySQL, we should install Elasticsearch. I will use Docker image to get the client. Alternatively you can use one of the methods under the following link Elasticsearch install

version: '3.7'

services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.3.2
    container_name: elasticsearch
    environment:
      - xpack.security.enabled=false
      - discovery.type=single-node
    ports:
      - 9200:9200
      - 9300:9300

Important
For testing purposes I disabled the security mode. You should not do this if you deploy your application into the live system.

After Elasticsearch has been installed and started, we can create one more configuration file with Client.

public class BookElasticsearchClient {
    private ElasticsearchClient client;

    public ElasticsearchClient getClient()
    {
        if (client == null) {
            initClient();
        }

        return client;
    }

    private void initClient()
    {
        RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200)).build();

        ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());

        client = new ElasticsearchClient(transport);
    }
}

If you use authentication with username and password, you need to configure these as well. The information on how to do this, is under the following link: Authentication

Aaaand now service class for searching.


@Configuration
public class BookElasticsearchService {
    BookElasticsearchClient client = new BookElasticsearchClient();

    private static final String BOOK_INDEX = "books";

    public void createBooksIndexBulk(final List<Book> books)
    {
        BulkRequest.Builder builder = new BulkRequest.Builder();

        for (Book book : books)
        {
            builder.operations(op -> op
                .index(index -> index
                    .index(BOOK_INDEX)
                    .id(String.valueOf(book.getId()))
                    .document(book)
                )
            );

            try {
                client.getClient().bulk(builder.build());
            } catch (ElasticsearchException exception) {
                exception.printStackTrace();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }

    public List<Book> findBookByIsbn(String query)
    {
        List<Book> books = new ArrayList<>();

        try {
            SearchResponse<Book> search = client.getClient().search(s -> s
                .index(BOOK_INDEX)
                .query(q -> q
                    .match(t -> t
                        .field("isbn")
                        .query(query))),
                Book.class);

            List<Hit<Book>> hits = search.hits().hits();
            for (Hit<Book> hit: hits) {
                books.add(hit.source());
            }

            return books;
        } catch (ElasticsearchException exception) {
            exception.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }

        return null;
    }
}

These are two very important functions. One of them is to create an index and put data into a document, another one have been used to get information about book from isbn.

And last but not least, we need a repository class for Elasticsearch.

public interface BookElasticsearchRepository extends ElasticsearchRepository<Book, String> {}

In the end we need to complete our controller to send requests.

@RestController
@RequestMapping(value = "book")
public class BookController {    
    @Autowired
    BookSearchService searchService;

    @Autowired
    BookRepository bookRepository;

    @Autowired
    BookElasticsearchService bookElasticsearchService;

    @GetMapping("/database")
    ResponseEntity<Iterable<Book>> getBooksFromDatabase(@RequestParam("query") String query)
    {
        Iterable<Book> booksFromDatabase = bookRepository.findByIsbn(query);

        return ResponseEntity.ok(booksFromDatabase);
    }

    @GetMapping("/hibernate")
    ResponseEntity<Iterable<Book>> getBooksFromHibernate(Pageable pageable, @RequestParam("query") String query)
    {
        Iterable<Book> booksFromHibernate = searchService.search(pageable, query);

        return ResponseEntity.ok(booksFromHibernate);
    }

    @GetMapping("/elasticsearch")
    ResponseEntity<Iterable<Book>> getBooksFromElasticsearch(@RequestParam("query") String query)
    {
        Iterable<Book> booksFromElasticSearch = bookElasticsearchService.findBookByIsbn(query);

        return ResponseEntity.ok(booksFromElasticSearch);
    }
}

One more important thing before starting the application:
we need to add the annotations(see below) in the main ApplicationClass, to make it work with multiple search engine technologies in one application.

@EnableElasticsearchRepositories(basePackageClasses = BookElasticsearchRepository.class)
@EnableJpaRepositories(excludeFilters = @ComponentScan.Filter(
    type = FilterType.ASSIGNABLE_TYPE, value = BookElasticsearchRepository.class
))
@SpringBootApplication
public class SearchApplication {

    public static void main(String[] args) {
        SpringApplication.run(SearchApplication.class, args);
    }

}

Let´s test all the things together.

I also had to create an index in my very first time. I called the function createBooksIndexBulk from BookElasticsearchService. There is also an option to call index directly in Elasticsearch console. How to do this is described here: Bulk Api

I send couple requests (in my database i have an isbn=123test):

http://localhost:8080/book/hibernate?query=123test
http://localhost:8080/book/database?query=123test
http://localhost:8080/book/elasticsearch?query=123test

For each request I got a right response. Mission completed.