# Too many scroll contexts error while running Scroll request through RestHighLevelClient in ElasticSearch 7.3.2

**URL:** https://discuss.elastic.co/t/too-many-scroll-contexts-error-while-running-scroll-request-through-resthighlevelclient-in-elasticsearch-7-3-2/228560
**Category:** Elasticsearch
**Created:** [April 17, 2020, 7:22pm UTC](https://discuss.elastic.co/t/too-many-scroll-contexts-error-while-running-scroll-request-through-resthighlevelclient-in-elasticsearch-7-3-2/228560 "2020-04-17T19:22:45Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![Nazim\_Hussain](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nazim_hussain/32/66545_2.png) [@Nazim\_Hussain](https://discuss.elastic.co/u/Nazim_Hussain)
#### Post date: [April 17, 2020, 7:22pm UTC](https://discuss.elastic.co/t/too-many-scroll-contexts-error-while-running-scroll-request-through-resthighlevelclient-in-elasticsearch-7-3-2/228560/1 "2020-04-17T19:22:45Z")

</div>

Hi,

I am new to ES and we migrated from ES 6 to ES 7 recently, but while making a Scroll search call, we are getting following error:

```auto
org.elasticsearch.ElasticsearchStatusException: Elasticsearch exception [type=search_phase_execution_exception, reason=all shards failed]
	at org.elasticsearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:177)
	at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1727)
	at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1704)
	at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1467)
	at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1424)
	at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1394)
	at org.elasticsearch.client.RestHighLevelClient.search(RestHighLevelClient.java:930)
	at com.mycomp.www.dts.api.lookup.KeyLookup.search(KeyLookup.java:63)
	at com.mycomp.www.dts.api.lookup.ValueLookup.query(ValueLookup.java:72)
	at com.mycomp.www.dts.api.resources.ValueSearchResource.search(ValueSearchResource.java:60)
	at sun.reflect.GeneratedMethodAccessor55.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)
	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:143)
	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347)
	at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102)
	at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
	at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
	at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
	at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473)
	at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
	at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
	at io.dropwizard.jetty.NonblockingServletHolder.handle(NonblockingServletHolder.java:49)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1623)
	at io.dropwizard.servlets.ThreadNameFilter.doFilter(ThreadNameFilter.java:35)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at io.dropwizard.jersey.filter.AllowedMethodsFilter.handle(AllowedMethodsFilter.java:45)
	at io.dropwizard.jersey.filter.AllowedMethodsFilter.doFilter(AllowedMethodsFilter.java:39)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at org.eclipse.jetty.servlets.CrossOriginFilter.handle(CrossOriginFilter.java:311)
	at org.eclipse.jetty.servlets.CrossOriginFilter.doFilter(CrossOriginFilter.java:265)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:239)
	at io.dropwizard.jetty.RoutingHandler.handle(RoutingHandler.java:52)
	at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:703)
	at io.dropwizard.jetty.BiDiGzipHandler.handle(BiDiGzipHandler.java:67)
	at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:56)
	at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:174)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.Server.handle(Server.java:505)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804)
	at java.lang.Thread.run(Thread.java:748)
	Suppressed: org.elasticsearch.client.ResponseException: method [POST], host [http://1.1.1.111:9200], URI [/myapp-time_series/_search?typed_keys=true&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&ignore_throttled=true&scroll=1m&search_type=query_then_fetch&batched_reduce_size=512&ccs_minimize_roundtrips=true], status line [HTTP/1.1 500 Internal Server Error]
{"error":{"root_cause":[{"type":"exception","reason":"Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"myapp-time_series","node":"dVU8MrD4SQuLz7HFePrlyw","reason":{"type":"exception","reason":"Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting."}}]},"status":500}
		at org.elasticsearch.client.RestClient.convertResponse(RestClient.java:253)
		at org.elasticsearch.client.RestClient.performRequest(RestClient.java:231)
		at org.elasticsearch.client.RestClient.performRequest(RestClient.java:205)
		at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1454)

```

This is how the search method looks like:

```auto
        final List<KeySearchResponse> result = new ArrayList<>();
        try {
            final SearchSourceBuilder sourceBuilder = new SearchSourceBuilder()
                    .query(getKeyMapQuery(replaceDot(keys), keyNamesFilter))
                    .size(size > 3000 ? 3000 : size)
                    .from(0);
            final Scroll scroll = new Scroll(TimeValue.timeValueSeconds(60L);
            final SearchRequest searchRequest = getScrollSearchRequest(scroll, sourceBuilder);
            SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
            Timer.Context ctxt = esRequestTimer.time();
            String scrollId = searchResponse.getScrollId();
            SearchHits hits = searchResponse.getHits();
            addHitsToResultMap(result, hits);
            ctxt.close();
            SearchScrollRequest scrollRequest;
            while (hits != null && hits.getHits().length > 0 && result.size() < size) {
                scrollRequest = new SearchScrollRequest(scrollId);
                ctxt = esRequestTimer.time();
                scrollRequest.scroll(scroll);
                searchResponse = client.scroll(scrollRequest, RequestOptions.DEFAULT);
                hits = searchResponse.getHits();
                addHitsToResultMap(result, hits);
                scrollId = searchResponse.getScrollId();
                ctxt.close();
            }
            final ClearScrollRequest clearScrollRequest = new ClearScrollRequest();
            clearScrollRequest.addScrollId(scrollId);
            final ClearScrollResponse clearScrollResponse = client.clearScroll(clearScrollRequest, RequestOptions.DEFAULT);

        } catch (IllegalStateException ise){
            LOGGER.error("IllegalStateException, seems HTTP Rest Client is stale, recreating the client connection pool!");
            client = ESRestClientFactory.resetRestClient(this.esConf);
        }
        catch (Exception e) {
            LOGGER.warn("Exception in retrieving results from ES", e);

        }
        return result;
    }

private BoolQueryBuilder getKeyMapQuery(KeyMap keys, boolean keyNamesFilter) {
        BoolQueryBuilder queryBuilder = QueryBuilders.boolQuery();
        filtersForKeys(keys, keyNamesFilter).forEach(queryBuilder::must);
        return queryBuilder;
    }

```

```auto
dependencies:
                <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-client</artifactId>
            <version>7.3.2</version>
        </dependency>
        <dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>elasticsearch-rest-high-level-client</artifactId>
            <version>7.3.2</version>
        </dependency>

```

This was working fine with ES6.  
Any pointers on this would be greatly appreciated.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 15, 2020, 7:34pm UTC](https://discuss.elastic.co/t/too-many-scroll-contexts-error-while-running-scroll-request-through-resthighlevelclient-in-elasticsearch-7-3-2/228560/2 "2020-05-15T19:34:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.