Are there memory leaks in ElasticSearch's TransportClient 5.4.3 or is my code flawed?

Dan-K · July 5, 2017, 9:51pm

Are there memory leaks in ElasticSearch's TransportClient 5.4.3? Or am I not using and closing the objects correctly?

I'm using the client in a Jenkins (2.68) which is on JRE 1.8.0_66 and writing my code in Groovy. BulkRequestBuilder is being used to submit multiple records.

Throughout the code, these 4x 16 MB byte arrays appear in memory but never get closed even past the TransportClient.close() method:

If I step through the code more slowly, I can get up to 9x of these 16 MB bytes arrays, some of which look very similar:

I believes I'm following the official documentation fully: https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/transport-client.html

Here's a trimmed down version of the code I'm using:

@Grapes([
    @Grab(group = "org.apache.logging.log4j", module = "log4j-api", version = "2.8.2", initClass = true),
    @Grab(group = "org.apache.logging.log4j", module = "log4j-core", version = "2.8.2", initClass = true),
    @Grab(group = "org.elasticsearch.client", module = "transport", version = "5.4.3", initClass = true)
])
public class ElasticSearchReport implements Serializable {

    pubblic void execute() {
        // Data to report.
        List<Map<String, Object>> data = ...

        // ElasticSearch settings. Do not use sniffing because we want to upload data to the master node.
        Settings settings = Settings.builder()
            .put("cluster.name", "my-cluster")
            .put("client.transport.sniff", false)
            .build()

        PreBuiltTransportClient preBuiltTransportClient = new PreBuiltTransportClient(settings)
        TransportClient client = preBuiltTransportClient.addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("my-elastic-search-host"), 9300))
        List<String> errors = new LinkedList<String>()

        try {
            BulkRequestBuilder bulkRequest = client.prepareBulk()

            // Build the bulk query with each data entry.
            data.each({entry ->
                String serialisedEntry = SerializationUtils.toJson(entry)
                bulkRequest.add(client.prepareIndex("my-index", "my-type").setSource(serialisedEntry, XContentType.JSON))
            })

            // Process the response.
            bulkRequest.get().getItems().each({response ->
                if (response.failed) {
                    errors.add(response.failure.message)
                    return
                }

                final String statusName = response.status().name()

                // Check statusName ...
            })

            if (errors) throw new Exception(...)
        } finally {
            client.close()
            preBuiltTransportClient.close()
        }
    }
}

From the objects I'm using, only PreBuiltTransportClient and TransportClient implement Closeable so I'm trying to explicitly close them in my finally block.

Tried using .withCloseable() (Groovy's equivalent to try-with-resources) but it did not prevent the issue from happening.

I thought perhaps my bulkRequest.get().getItems().each({response -> line was doing something funny so I replaced it with the following but to no effect:

BulkResponse bulkResponse = bulkRequest.get()
BulkItemResponse[] bulkItemResponses = bulkResponse.getItems()
for (int responseIndex = 0; responseIndex < bulkItemResponses.length; responseIndex++) { ... }

I've also tried replacing BulkResponse bulkResponse = bulkRequest.get() with BulkResponse bulkResponse = bulkRequest.execute().actionGet() with no luck.

jprante · July 6, 2017, 10:03am

Instead of

try

BulkResponse bulkResponse = bulkRequest.execute().actionGet()

Dan-K · July 6, 2017, 6:04pm

Thanks for suggestion Jörg.
Tried it out but I still get the same result.

Dan-K · July 12, 2017, 5:03pm

The ElasticSearch Java REST Client doesn't appear to have these issues.
I've switched over to it and haven't experienced out-of-memory errors. All relatively large byte arrays are released once the client is closed and scoped out.

Thiago_Sayao · July 31, 2017, 7:48pm

I have a similar code and it is also leaking. I am using 5.5.1.

It seems to be leaking byte arrays:

The code looks like this:

try(PreBuiltTransportClient preBuiltTransportClient = new PreBuiltTransportClient(settings)) {
    try(TransportClient transportClient = preBuiltTransportClient.addTransportAddress(
            new InetSocketTransportAddress(InetAddress.getByName(endPoint), javaPort))) {
    
        populateIndex(transportClient, newIndexName, produtoService);
        reassignAlias(transportClient, newIndexName);
        cleanIndices(transportClient, newIndexName);
    }
    catch(IOException e) {
        logger.error(e.getMessage(), e);
    }
}