Elasticsearch gives execption while report download["message":"Request cannot be executed; I/O reactor status: STOPPED","errorstacktrace":"java.lang.RuntimeException]

PRASHANT_MEHTA · January 10, 2023, 11:44am

Hello All,

I'm getting following exception while downloading report by query elastic.Unable to understand resolution for this.Tried increasing the jvm heap to 12 gb ,still same exception.

Exception:

{"statusCode":"REPORT_SERVICE_1009","timestamp":"2023-01-10T11:23:54.568+00:00","message":"Request cannot be executed; I/O reactor status: STOPPED","errorstacktrace":"java.lang.RuntimeException: Request cannot be executed; I/O reactor status: STOPPED\n\tat org.elasticsearch.client.RestClient.extractAndWrapCause(RestClient.java:889)\n\tat org.elasticsearch.client.RestClient.performRequest(RestClient.java:283)\n\tat org.elasticsearch.client.RestClient.performRequest(RestClient.java:270)\n\tat org.elasticsearch.client.RestHighLevelClient.performClientRequest(RestHighLevelClient.java:2082)\n\tat org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1732)\n\tat org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1702)\n\tat org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1672)\n\tat org.elasticsearch.client.RestHighLevelClient.search(RestHighLevelClient.java:1128)\n\tat new.connect.integration.services.reportservice.service.randReportService.getrandUniqueIds(randReportService.java:282)\n\tat new.connect.integration.services.reportservice.service.randReportService.retrieveImportJobsData(randReportService.java:128)\n\tat new.connect.integration.services.reportservice.service.randReportService.generaterandReport(randReportService.java:96)\n\tat new.connect.integration.services.reportservice.controller.ReportServiceController.downloadrandReport(ReportServiceController.java:1045)\n\tat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\tat java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.base/java.lang.reflect.Method.invoke(Method.java:566)\n\tat org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)\n\tat org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:150)\n\tat org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:117)\n\tat org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:895)\n\tat org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:808)\n\tat org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)\n\tat org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1067)\n\tat org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:963)\n\tat org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)\n\tat org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:655)\n\tat org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:764)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)\n\tat org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)\n\tat org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)\n\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)\n\tat org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)\n\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)\n\tat org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:96)\n\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)\n\tat org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)\n\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:117)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)\n\tat org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)\n\tat org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)\n\tat org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:360)\n\tat org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:399)\n\tat org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)\n\tat org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:889)\n\tat org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1743)\n\tat org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)\n\tat org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)\n\tat org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)\n\tat org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)\n\tat java.base/java.lang.Thread.run(Thread.java:829)\nCaused by: java.lang.IllegalStateException: Request cannot be executed; I/O reactor status: STOPPED\n\tat org.apache.http.util.Asserts.check(Asserts.java:46)\n\tat org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase.ensureRunning(CloseableHttpAsyncClientBase.java:90)\n\tat org.apache.http.impl.nio.client.InternalHttpAsyncClient.execute(InternalHttpAsyncClient.java:123)\n\tat org.elasticsearch.client.RestClient.performRequest(RestClient.java:279)\n\t... 64 more\n"}

Query:

GET /his-monitor-adapter-*/_search
{
  "query": {
 GET /tis-monitor-adapter-*/_search
{
  "size": 0,
  "query": {
    "bool": {
      "must": [
        {
          "exists": {
            "field": "adapter.scheduler"
          }
        }
      ]
    }
  }, 
  "aggs": {
    "uniqueIdList": {
      "terms": {
        "field": "adapter.uniqueId.keyword",
        "size": 10000
      }
    }
  }
}

TEST INSTANCE:

How many indices and shards does it run against?
1 Index,1 shard

How much data do those indices contain?
8840 documents,1.4mb

What is the specification of your Elasticsearch cluster?
Single Node elastic cluster,8 core CPU

JVM HEAP Allocated:
-Xms8g
-Xmx8g

Any suggestion would be helpful.

Thanx

Ayush_Mathur · January 10, 2023, 2:04pm

Have a look at: Low-level REST client status error:ava.lang.IllegalStateException: Request cannot be executed; I/O reactor status: STOPPED · Issue #42133 · elastic/elasticsearch (github.com)
and
Elasticsearch Request cannot be executed; I/O reactor status: STOPPED - Stack Overflow
It's worth checking on how many connections are being created from your client to ES, creating too many connections or not closing them properly can lead to this situation.

PRASHANT_MEHTA · January 11, 2023, 12:03pm

Hello @Ayush_Mathur ,

Thanx for your time for looking into this and for links mentioned.Will look into points you've mentioned.

Many Thanx

system · February 8, 2023, 12:03pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Exception occurred java.lang.RuntimeException: Request cannot be executed; I/O reactor status: STOPPED Elasticsearch language-clients	2	1535	September 17, 2020
Request cannot be executed i/o reactor status stopped Elasticsearch	6	29840	September 13, 2019
Exception in thread "main" java.net.ConnectException: Timeout connecting to [localhost:9200] Elasticsearch	8	5583	May 30, 2019
Java API ping operation hangs the application on Elasticsearch 7.11.0 Elasticsearch	21	1508	March 17, 2021
IO Reactor errors closing connection Elasticsearch	12	4127	August 17, 2021

Elasticsearch gives execption while report download["message":"Request cannot be executed; I/O reactor status: STOPPED","errorstacktrace":"java.lang.RuntimeException]

Related Topics