Elastic 8.1 RAM memory leak

Hello. I'm using Elasticsearch 8.1, dependency co.elastic.clients Elasticsearch-java version 8.1.2. When creating one scroll request for 15_000 records, a large memory consumption occurs. The request consumed 8 GB of memory and vened 8 GB after 20 - 25 minutes (1 GB freed up after about 6 minutes). Who's to say what's the matter? The code for the scroll request is given below:

RestClient httpClient = RestClient.builder(new HttpHost("localhost", 9200)).build();
ElasticsearchTransport transport = new RestClientTransport(httpClient, new JacksonJsonpMapper());
ElasticsearchClient client = new ElasticsearchClient(transport);

public Long createScrollRequest(String input) throws IOException {
        //List<News> list = new LinkedList<>();

        SearchRequest searchRequest = new SearchRequest.Builder()
                .scroll(new Time.Builder().time("20s").build())
                .query(qb -> qb.multiMatch(mmqb -> mmqb.query(input).fields("status")))
                .build();
        SearchResponse<News> searchResponse = client.search(searchRequest, News.class);
        String scrollId = searchResponse.scrollId();
        List<Hit<News>> searchHits = searchResponse.hits().hits();
        while (searchHits != null && searchHits.size() > 0) {
            ScrollRequest scrollRequest = new ScrollRequest.Builder()
                    .scroll(new Time.Builder().time("20s").build())
                    .scrollId(scrollId)
                    .build();
            searchResponse = client.scroll(scrollRequest, News.class);
            scrollId = searchResponse.scrollId();
            searchHits = searchResponse.hits().hits();
        }

        ClearScrollRequest clearScrollRequest = new ClearScrollRequest.Builder()
                .scrollId(scrollId)
                .build();
        client.clearScroll(clearScrollRequest);

        return searchResponse.hits().total().value();
    }

If we add property searchRequest.size (10000) it saves memory. Instead of Gigabytes, 100 - 300 Megabytes will be spent. Which is quite a lot (

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.