Failed to execute fetch phase SearchContextMissingException

(Meet Dave) #1


We are constantly seeing below errors in elasticsearch log file. We want to know what could be the root cause of this problem and how to prevent these type of errors?

[2018-06-12T19:03:21,239][DEBUG][o.e.a.s.TransportSearchAction] [master-ip-x-x-x-x] [21240589] Failed to execute fetch phase
org.elasticsearch.transport.RemoteTransportException: [data-ip-x-x-x-x][x-x-x-x:9300][indices:data/read/search[phase/fetch/id]]
Caused by: No search context found for id [21240589]
        at ~[elasticsearch-5.4.1.jar:5.4.1]
        at ~[elasticsearch-5.4.1.jar:5.4.1]
        at$12.messageReceived( ~[elasticsearch-5.4.1.jar:5.4.1]
        at$12.messageReceived( ~[elasticsearch-5.4.1.jar:5.4.1]
        at$ProfileSecuredRequestHandler$1.doRun( ~[?:?]
        at ~[elasticsearch-5.4.1.jar:5.4.1]
        at org.elasticsearch.common.util.concurrent.EsExecutors$1.execute( ~[elasticsearch-5.4.1.jar:5.4.1]
        at$ProfileSecuredRequestHandler.lambda$messageReceived$0( ~[?:?]
        at org.elasticsearch.action.ActionListener$1.onResponse( ~[elasticsearch-5.4.1.jar:5.4.1]
        at$NodeProfile.lambda$authorizeAsync$5( ~[?:?]
        at$AsyncAuthorizer.maybeRun( ~[?:?]
        at$AsyncAuthorizer.setRunAsRoles( ~[?:?]
        at$AsyncAuthorizer.authorize( ~[?:?]
        at$NodeProfile.authorizeAsync( ~[?:?]
        at$NodeProfile.lambda$inbound$2( ~[?:?]
        at org.elasticsearch.action.ActionListener$1.onResponse( ~[elasticsearch-5.4.1.jar:5.4.1]
        at$Authenticator.lambda$authenticateAsync$0( ~[?:?]
        at$Authenticator.lambda$lookForExistingAuthentication$2( ~[?:?]
        at$Authenticator.lookForExistingAuthentication( ~[?:?]
        at$Authenticator.authenticateAsync( ~[?:?]
        at$Authenticator.access$000( ~[?:?]
        at ~[?:?]
        at$NodeProfile.inbound( ~[?:?]
        at$ProfileSecuredRequestHandler.messageReceived( ~[?:?]
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived( ~[elasticsearch-5.4.1.jar:5.4.1]
        at org.elasticsearch.transport.TcpTransport$RequestHandler.doRun( ~[elasticsearch-5.4.1.jar:5.4.1]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun( ~[elasticsearch-5.4.1.jar:5.4.1]
        at ~[elasticsearch-5.4.1.jar:5.4.1]
        at java.util.concurrent.ThreadPoolExecutor.runWorker( ~[?:1.8.0_151]
        at java.util.concurrent.ThreadPoolExecutor$ ~[?:1.8.0_151]
        at [?:1.8.0_151]


(Igor Motov) #2

Are you using scroll API?

(Meet Dave) #3

@Igor_Motov No we are not using scroll API

(Igor Motov) #4

Long running query or non-trivial network equipment (like load balancer, "smart"switches, etc ) between nodes? 3rd party plugins?

(Meet Dave) #5

No third party plugins and also nodes are connected directly with each other. Not sure about long running searches but we see too many search rejection in thread_pool

(Igor Motov) #6

Yes, this is possible when the cluster is overloaded with search traffic. Perhaps, it's time to increase the cluster capacity?

(Meet Dave) #7

@Igor_Motov We are currently using hot warm architecture. We have 6 hot data nodes and 8 warm nodes. How can we overcome this problem without increasing cluster capacity? Can you provide us some guide how can we improve searching?

(Igor Motov) #8

Sorry, I cannot really make any performance tuning suggestions based on a single stacktrace and the number of nodes. That's just not enough information. We would need to look at your cluster stats, analyze your queries, etc...

(system) #9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.