Java Snippet #1: In-memory ElasticSearch node

Welcome to my first post from Java Snippet series. In the firs post I will explain how to write your own in-memory ElasticSearch node. It can be quiet useful for testing purposes. I have wrote this to be able to test my REST client. If you will be satisfied with this post, I can describe in the second one how to write own REST client for ES.

I have chosen the newest version of ElasticSearch 5.4.3. Necessary maven dependencies are presented below. I have omitted there libraries for REST client.

    <dependency>
        <groupId>com.sun.jersey</groupId>
        <artifactId>jersey-client</artifactId>
        <version>1.19.3</version>
    </dependency>
    <dependency>
        <groupId>org.elasticsearch</groupId>
        <artifactId>elasticsearch</artifactId>
        <version>5.4.3</version>
    </dependency>
    <dependency>
        <groupId>org.elasticsearch.plugin</groupId>
        <artifactId>transport-netty4-client</artifactId>
        <version>5.4.3</version>
    </dependency>

At first, since ES 5 it is not possible to use builder for Node class. It forces us to write own extended with transport client version of Node. The most important method which can cause quiet much problems is prepareNodeSettings(). Properties for ES are changing in almost every minor version. In the listening I have added comments with short explanation of property's meaning.

Below you can see implementation of main class for ElastiscSearch node. ElasticSearchExample class has 5 attributes. One important remark for them is that HTTP ports can't be used by any other process in the system. Other thing about which we should remember during creation of own in-memory node is cleaning before we start our node again. It is ensured by removeOldDataDirectories(String dataDirectory) method. Here is necessary that name of basis folder is exactly the same like given in the path.data property. Actually, to make it is easier, all path properties can point to the same directory.

Method createIndex() is useful in case you set property action.auto_create_index to false. I prefer to have full control over the newly created index.

import java.io.File;
import java.util.Map;

import org.elasticsearch.common.io.FileSystemUtils;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.node.Node;
import org.elasticsearch.transport.Netty4Plugin;

import org.example.ExtendedNode;
import org.example.IndexRestClient;

public class ElasticSearchExample {
    private IndexRestClient restClient;
    private Node node;
    private String indexName;
    private final String HTTP_PORT = "9205";
    private final String HTTP_TRANSPORT_PORT = "9305"; 

    public void startElasticSearch() throws Exception {
        final String nodeName = "indexernode"; 
        testIndexName =  "testindex";
        restClient = initializeRestClient();
        Map settingsMap = prepareNodeSettings(nodeName); 
        removeOldDataDirectories("target/" + nodeName); 
        Settings settings = Settings.builder().put(settingsMap).build(); 
        node = new ExtendedNode(settings, asList(Netty4Plugin.class));
        node.start();
    } 

    private void removeOldDataDirectories(String dataDirectory)
              throws Exception {
        File dataDir = new File(dataDirectory);
        if (dataDir.exists()) {
            FileSystemUtils.deleteSubDirectories(dataDir.toPath()); 
        } 
    }

    public void stopElasticSearch() throws Exception {
        node.close(); 
    } 

    public void createIndex() throws Exception {
        restClient.createIndex(); 
    } 

    public void deleteIndex() throws Exception {
        restClient.deleteIndex();
    }

    private IndexRestClient initializeRestClient() {
        IndexRestClient restClient = new IndexRestClient();
        restClient.initiateClient();
        restClient.setIndex(indexName);
        restClient.setType("indexer");
        return restClient;
    }

    private Map prepareNodeSettings(String nodeName) {
        Map settingsMap = new HashMap();
        settingsMap.put("node.name", nodeName);
        // create all data directories under Maven build directory
        settingsMap.put("path.conf", "target");
        settingsMap.put("path.data", "target");
        settingsMap.put("path.logs", "target");
        settingsMap.put("path.home", "target");
        // set ports used by Elastic Search to something different than default
        settingsMap.put("http.type", "netty4");
        settingsMap.put("http.port", HTTP_PORT);
        settingsMap.put("transport.tcp.port", HTTP_TRANSPORT_PORT);
        settingsMap.put("transport.type", "netty4");
        // disable automatic index creation
        settingsMap.put("action.auto_create_index", "false");
        return settingsMap;
    }
}

ExtendendNode class for being able to build own node.

import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.node.Node;
import org.elasticsearch.node.InternalSettingsPreparer;
import org.elasticsearch.plugins.Plugin;

import java.util.Collection;

public class ExtendedNode extends Node {
    public ExtendedNode(Settings preparedSettings,
               Collection<Class<? extends Plugin>> classpathPlugins) {
        super(InternalSettingsPreparer.prepareEnvironment(preparedSettings, null),
               classpathPlugins);
    }
}

I hope you enjoyed my first post from Java Snippets series. Please upvote the post if you like it, follow for more, and leave a comment to let me know what you think!