Continuous transform failures

Hello,

What can cause this error when running a continuous transform ?

[s1-aggs-xxx] Search context missing, falling back to normal search; request [apply_results]

Caused by: org.Elasticsearch.search.SearchContextMissingException: No search context found for id [573107]

"org.Elasticsearch.action.search.SearchPhaseExecutionException: Partial shards failure

{"type": "server", "timestamp": "2022-04-10T12:03:16,294Z", "level": "WARN", "component": "o.e.x.t.t.ClientTransformIndexer", "cluster.name": "cluster_dc01", "node.name": "es_node02", "message": "[s1-aggs-xxx] Search context missing, falling back to normal search; request [apply_results]", "cluster.uuid": "1W-tM1tyRV6L_ciSZWfekg", "node.id": "XNrcJTuuSIO5cPPghL3O3w" ,
"stacktrace": ["org.elasticsearch.action.search.SearchPhaseExecutionException: Partial shards failure",
"at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:725) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:429) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:757) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:509) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.search.AbstractSearchAsyncAction.access$000(AbstractSearchAsyncAction.java:64) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.search.AbstractSearchAsyncAction$1.onFailure(AbstractSearchAsyncAction.java:343) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.ActionListener$Delegating.onFailure(ActionListener.java:66) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:48) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.search.SearchTransportService$ConnectionCountingHandler.handleException(SearchTransportService.java:651) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:883) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.transport.TransportService.sendChildRequest(TransportService.java:937) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.transport.TransportService.sendChildRequest(TransportService.java:925) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.search.SearchTransportService.sendExecuteQuery(SearchTransportService.java:250) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.search.SearchQueryThenFetchAsyncAction.executePhaseOnShard(SearchQueryThenFetchAsyncAction.java:104) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.search.AbstractSearchAsyncAction.lambda$performPhaseOnShard$3(AbstractSearchAsyncAction.java:328) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.search.AbstractSearchAsyncAction$PendingExecutions.tryRun(AbstractSearchAsyncAction.java:855) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.search.AbstractSearchAsyncAction.performPhaseOnShard(AbstractSearchAsyncAction.java:362) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.search.AbstractSearchAsyncAction.run(AbstractSearchAsyncAction.java:256) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.search.AbstractSearchAsyncAction.executePhase(AbstractSearchAsyncAction.java:466) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.search.AbstractSearchAsyncAction.start(AbstractSearchAsyncAction.java:211) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.search.TransportSearchAction.executeSearch(TransportSearchAction.java:1045) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.search.TransportSearchAction.executeLocalSearch(TransportSearchAction.java:761) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.search.TransportSearchAction.lambda$executeRequest$6(TransportSearchAction.java:397) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:136) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:112) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:77) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.search.TransportSearchAction.executeRequest(TransportSearchAction.java:485) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:283) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:99) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:179) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.support.ActionFilter$Simple.apply(ActionFilter.java:53) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:177) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$3(SecurityActionFilter.java:190) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.ActionListener$DelegatingFailureActionListener.onResponse(ActionListener.java:219) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:577) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:571) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.interceptor.BulkShardRequestInterceptor.intercept(BulkShardRequestInterceptor.java:87) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:575) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:571) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:85) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.interceptor.SearchRequestInterceptor.intercept(SearchRequestInterceptor.java:26) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:575) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:571) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.interceptor.ResizeRequestInterceptor.intercept(ResizeRequestInterceptor.java:103) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:575) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:571) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.interceptor.DlsFlsLicenseRequestInterceptor.intercept(DlsFlsLicenseRequestInterceptor.java:85) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:575) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:571) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:85) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.interceptor.UpdateRequestInterceptor.intercept(UpdateRequestInterceptor.java:27) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:575) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:571) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.interceptor.IndicesAliasesRequestInterceptor.intercept(IndicesAliasesRequestInterceptor.java:128) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:575) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:571) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:85) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.interceptor.ShardSearchRequestInterceptor.intercept(ShardSearchRequestInterceptor.java:26) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService.runRequestInterceptors(AuthorizationService.java:571) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService.handleIndexActionAuthorizationResult(AuthorizationService.java:556) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorizeAction$11(AuthorizationService.java:450) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService$AuthorizationResultListener.onResponse(AuthorizationService.java:967) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService$AuthorizationResultListener.onResponse(AuthorizationService.java:931) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:31) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.RBACEngine.lambda$authorizeIndexAction$3(RBACEngine.java:352) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:136) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.common.util.concurrent.ListenableFuture.notifyListenerDirectly(ListenableFuture.java:113) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.common.util.concurrent.ListenableFuture.addListener(ListenableFuture.java:55) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.common.util.concurrent.ListenableFuture.addListener(ListenableFuture.java:41) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService$CachingAsyncSupplier.getAsync(AuthorizationService.java:1015) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.RBACEngine.authorizeIndexAction(RBACEngine.java:343) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService.authorizeAction(AuthorizationService.java:443) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService.maybeAuthorizeRunAs(AuthorizationService.java:371) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorize$1(AuthorizationService.java:256) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:136) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:31) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.RBACEngine.lambda$resolveAuthorizationInfo$1(RBACEngine.java:138) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:136) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.getRoles(CompositeRolesStore.java:277) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.RBACEngine.getRoles(RBACEngine.java:144) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.RBACEngine.resolveAuthorizationInfo(RBACEngine.java:127) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authz.AuthorizationService.authorize(AuthorizationService.java:258) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$4(SecurityActionFilter.java:186) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:136) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.ActionListener$MappedActionListener.onResponse(ActionListener.java:101) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authc.AuthenticatorChain.authenticateAsync(AuthenticatorChain.java:102) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:171) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.applyInternal(SecurityActionFilter.java:182) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.apply(SecurityActionFilter.java:124) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:177) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:154) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:82) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:95) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:73) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:407) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.client.FilterClient.doExecute(FilterClient.java:57) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.client.ParentTaskAssigningClient.doExecute(ParentTaskAssigningClient.java:55) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:407) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.core.ClientHelper.executeWithHeadersAsync(ClientHelper.java:220) [x-pack-core-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.transform.transforms.ClientTransformIndexer.doSearch(ClientTransformIndexer.java:486) [transform-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.transform.transforms.ClientTransformIndexer.lambda$doNextSearch$0(ClientTransformIndexer.java:145) [transform-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:136) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.transform.transforms.ClientTransformIndexer.injectPointInTimeIfNeeded(ClientTransformIndexer.java:420) [transform-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.transform.transforms.ClientTransformIndexer.doNextSearch(ClientTransformIndexer.java:143) [transform-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.core.indexing.AsyncTwoPhaseIndexer.triggerNextSearch(AsyncTwoPhaseIndexer.java:607) [x-pack-core-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.core.indexing.AsyncTwoPhaseIndexer.nextSearch(AsyncTwoPhaseIndexer.java:594) [x-pack-core-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.core.indexing.AsyncTwoPhaseIndexer.onBulkResponse(AsyncTwoPhaseIndexer.java:565) [x-pack-core-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.core.indexing.AsyncTwoPhaseIndexer.lambda$onSearchResponse$11(AsyncTwoPhaseIndexer.java:531) [x-pack-core-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:136) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.transform.transforms.ClientTransformIndexer.handleBulkResponse(ClientTransformIndexer.java:173) [transform-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.transform.transforms.ClientTransformIndexer.lambda$doNextBulk$1(ClientTransformIndexer.java:162) [transform-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:136) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:31) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:88) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:82) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:31) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$2(SecurityActionFilter.java:192) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.ActionListener$DelegatingFailureActionListener.onResponse(ActionListener.java:219) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.ActionListener$RunBeforeActionListener.onResponse(ActionListener.java:389) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.ActionListener$MappedActionListener.onResponse(ActionListener.java:101) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.bulk.TransportBulkAction$BulkOperation$1.finishHim(TransportBulkAction.java:625) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.bulk.TransportBulkAction$BulkOperation$1.onResponse(TransportBulkAction.java:601) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.bulk.TransportBulkAction$BulkOperation$1.onResponse(TransportBulkAction.java:590) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:88) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:82) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:31) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$2(SecurityActionFilter.java:192) [x-pack-security-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.ActionListener$DelegatingFailureActionListener.onResponse(ActionListener.java:219) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.support.replication.TransportReplicationAction$ReroutePhase.finishOnSuccess(TransportReplicationAction.java:1070) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.support.replication.TransportReplicationAction$ReroutePhase$1.handleResponse(TransportReplicationAction.java:978) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.support.replication.TransportReplicationAction$ReroutePhase$1.handleResponse(TransportReplicationAction.java:969) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.transport.TransportService$4.handleResponse(TransportService.java:847) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleResponse(TransportService.java:1471) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.transport.InboundHandler.doHandleResponse(InboundHandler.java:340) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.transport.InboundHandler.handleResponse(InboundHandler.java:324) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:134) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:88) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:743) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:147) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:119) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:84) [elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:71) [transport-netty4-client-7.16.2.jar:7.16.2]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:280) [netty-handler-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) [netty-codec-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1374) [netty-handler-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1237) [netty-handler-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1286) [netty-handler-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507) [netty-codec-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446) [netty-codec-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) [netty-codec-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) [netty-transport-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719) [netty-transport-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:620) [netty-transport-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:583) [netty-transport-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) [netty-common-4.1.66.Final.jar:4.1.66.Final]",
"at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.66.Final.jar:4.1.66.Final]",
"at java.lang.Thread.run(Thread.java:833) [?:?]",
"Caused by: org.elasticsearch.search.SearchContextMissingException: No search context found for id [573234]",
"at org.elasticsearch.search.SearchService.findReaderContext(SearchService.java:824) ~[elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.search.SearchService.createOrGetReaderContext(SearchService.java:839) ~[elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:615) ~[elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.search.SearchService.lambda$executeQueryPhase$2(SearchService.java:483) ~[elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:47) ~[elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:62) ~[elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) ~[elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:33) ~[elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:777) ~[elasticsearch-7.16.2.jar:7.16.2]",
"at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) ~[elasticsearch-7.16.2.jar:7.16.2]",
"at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]",
"at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]",
"at java.lang.Thread.run(Thread.java:833) ~[?:?]"] }

Transform uses the point in time API internally to reduce overhead. The error message says that the point in time context could not be found. The error is benign, because transform falls back to an ordinary search request, no data is lost. However it means some performance overhead if this happens a lot as it has to go back and forth.

How often do you see this error?
At the time this happens, do you have some instabilities in the cluster?
Do you manage source indices of your transform via ILM?

Related issue: Point in time fails for deleted indices and fails differently in comparison to scroll · Issue #81256 · elastic/elasticsearch · GitHub

Transform issue: [Transform] Transform fails with task encountered irrecoverable failure: org.elasticsearch.index.IndexNotFoundException · Issue #81252 · elastic/elasticsearch · GitHub
(Note, the error does not happen in 7.16.2, however you get the log message about the fallback)

You can disable the use of point in time in transform by setting use_point_in_time to false in the settings section. (via _update)

2 Likes

Thanks @Hendrik_Muhs for your feedback
I have the issue only when the transform hit an index of 40M records (18GB size) with 3 shards (The cluster is 3 nodes)
I don't use ILM for now

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.