# Failed to send release search context - SourceLookup fetching cost (?)

**URL:** <https://discuss.elastic.co/t/failed-to-send-release-search-context-sourcelookup-fetching-cost/17753>\
**Category:** Elasticsearch\
**Created:** [May 27, 2014, 1:46pm UTC](https://discuss.elastic.co/t/failed-to-send-release-search-context-sourcelookup-fetching-cost/17753 "2014-05-27T13:46:18Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mateusz\_Kaczynski](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mateusz_kaczynski/32/1206_2.png) [@Mateusz\_Kaczynski](https://discuss.elastic.co/u/Mateusz_Kaczynski)\
**Post date:** [May 27, 2014, 1:46pm UTC](https://discuss.elastic.co/t/failed-to-send-release-search-context-sourcelookup-fetching-cost/17753/1 "2014-05-27T13:46:18Z")

</div>

We have recently changed some of our code to include additional call to  
SourceLookup.extractValue(path) in fetch stage. Soon after, we have started  
experiencing some issues with search stability (with some of our searches  
failing to finish, others taking very long).

I can see search lookup queue getting filled up (to 1000) and I can see the  
following spamming the logs:

es-cluster-3-3 elasticsearch[2014-05-26 18:59:28,584][WARN][search.action  
] [Maeby] Failed to send release search context  
es-cluster-3-3  
elasticsearchorg.elasticsearch.transport.SendRequestTransportException:  
[Oscar][inet[/10.0.0.1/ip-10-0-0-1.eu-west-1.compute.internal:9300]][search/freeContext]  
es-cluster-3-3 elasticsearch at  
org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:202)  
es-cluster-3-3 elasticsearch at  
org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:173)  
es-cluster-3-3 elasticsearch at  
org.elasticsearch.search.action.SearchServiceTransportAction.sendFreeContext(SearchServiceTransportAction.java:103)  
es-cluster-3-3 elasticsearch at  
org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.releaseIrrelevantSearchContexts(TransportSearchTypeAction.java:392)  
es-cluster-3-3 elasticsearch at  
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.finishHim(TransportSearchQueryThenFetchAction.java:191)  
es-cluster-3-3 elasticsearch at  
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.onFetchFailure(TransportSearchQueryThenFetchAction.java:177)  
es-cluster-3-3 elasticsearch at  
org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction$3.onFailure(TransportSearchQueryThenFetchAction.java:165)  
es-cluster-3-3 elasticsearch at  
org.elasticsearch.search.action.SearchServiceTransportAction$10.handleException(SearchServiceTransportAction.java:426)  
es-cluster-3-3 elasticsearch at  
org.elasticsearch.transport.TransportService$Adapter$2$1.run(TransportService.java:316)  
es-cluster-3-3 elasticsearch at  
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
es-cluster-3-3 elasticsearch at  
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
es-cluster-3-3 elasticsearch at java.lang.Thread.run(Thread.java:745)  
es-cluster-3-3 elasticsearchCaused by:  
org.elasticsearch.transport.NodeNotConnectedException:  
[Oscar][inet[/10.0.0.1/ip-10-0-0-1.eu-west-1.compute.internal:9300]] Node  
not connected  
es-cluster-3-3 elasticsearch at  
org.elasticsearch.transport.netty.NettyTransport.nodeChannel(NettyTransport.java:859)  
es-cluster-3-3 elasticsearch at  
org.elasticsearch.transport.netty.NettyTransport.sendRequest(NettyTransport.java:540)  
es-cluster-3-3 elasticsearch at  
org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:189)  
es-cluster-3-3 elasticsearch ... 11 more

There is also a long GC running at the same time, not sure if it might be  
the cause or the effect.  
Is it at all possible that this might have been caused by a call to  
SourceLookup.extractValue() assuming the field that is extracted was not  
specified in the query?

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/0b05891b-b578-42fa-8eed-5e5704c4e686%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/0b05891b-b578-42fa-8eed-5e5704c4e686%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Mateusz\_Kaczynski](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mateusz_kaczynski/32/1206_2.png) [@Mateusz\_Kaczynski](https://discuss.elastic.co/u/Mateusz_Kaczynski)\
**Post date:** [May 29, 2014, 9:56am UTC](https://discuss.elastic.co/t/failed-to-send-release-search-context-sourcelookup-fetching-cost/17753/2 "2014-05-29T09:56:13Z")

</div>

Anyone encountered something similar / related?

On Tuesday, 27 May 2014 13:46:19 UTC, Mateusz Kaczynski wrote:

> We have recently changed some of our code to include additional call to  
> SourceLookup.extractValue(path) in fetch stage. Soon after, we have  
> started experiencing some issues with search stability (with some of our  
> searches failing to finish, others taking very long).
> 
> I can see search lookup queue getting filled up (to 1000) and I can see  
> the following spamming the logs:
> 
> es-cluster-3-3 elasticsearch[2014-05-26 18:59:28,584][WARN][search.action  
> ] [Maeby] Failed to send release search context  
> es-cluster-3-3  
> elasticsearchorg.elasticsearch.transport.SendRequestTransportException:  
> [Oscar][inet[/  
> 10.0.0.1/ip-10-0-0-1.eu-west-1.compute.internal:9300]][search/freeContext[http://10.0.0.1/ip-10-0-0-1.eu-west-1.compute.internal:9300]][search/freeContext](http://10.0.0.1/ip-10-0-0-1.eu-west-1.compute.internal:9300%5D%5D%5Bsearch/freeContext)  
> ]  
> es-cluster-3-3 elasticsearch at  
> org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:202)  
> es-cluster-3-3 elasticsearch at  
> org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:173)  
> es-cluster-3-3 elasticsearch at  
> org.elasticsearch.search.action.SearchServiceTransportAction.sendFreeContext(SearchServiceTransportAction.java:103)  
> es-cluster-3-3 elasticsearch at  
> org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.releaseIrrelevantSearchContexts(TransportSearchTypeAction.java:392)  
> es-cluster-3-3 elasticsearch at  
> org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.finishHim(TransportSearchQueryThenFetchAction.java:191)  
> es-cluster-3-3 elasticsearch at  
> org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.onFetchFailure(TransportSearchQueryThenFetchAction.java:177)  
> es-cluster-3-3 elasticsearch at  
> org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction$3.onFailure(TransportSearchQueryThenFetchAction.java:165)  
> es-cluster-3-3 elasticsearch at  
> org.elasticsearch.search.action.SearchServiceTransportAction$10.handleException(SearchServiceTransportAction.java:426)  
> es-cluster-3-3 elasticsearch at  
> org.elasticsearch.transport.TransportService$Adapter$2$1.run(TransportService.java:316)  
> es-cluster-3-3 elasticsearch at  
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)  
> es-cluster-3-3 elasticsearch at  
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)  
> es-cluster-3-3 elasticsearch at java.lang.Thread.run(Thread.java:745)  
> es-cluster-3-3 elasticsearchCaused by:  
> org.elasticsearch.transport.NodeNotConnectedException: [Oscar][inet[/  
> 10.0.0.1/ip-10-0-0-1.eu-west-1.compute.internal:9300]] Node not connected  
> es-cluster-3-3 elasticsearch at  
> org.elasticsearch.transport.netty.NettyTransport.nodeChannel(NettyTransport.java:859)  
> es-cluster-3-3 elasticsearch at  
> org.elasticsearch.transport.netty.NettyTransport.sendRequest(NettyTransport.java:540)  
> es-cluster-3-3 elasticsearch at  
> org.elasticsearch.transport.TransportService.sendRequest(TransportService.java:189)  
> es-cluster-3-3 elasticsearch ... 11 more
> 
> There is also a long GC running at the same time, not sure if it might be  
> the cause or the effect.  
> Is it at all possible that this might have been caused by a call to  
> SourceLookup.extractValue() assuming the field that is extracted was not  
> specified in the query?

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/8f226f6e-e453-4bb9-b204-2c61a9f30662%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/8f226f6e-e453-4bb9-b204-2c61a9f30662%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![Mateusz\_Kaczynski](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mateusz_kaczynski/32/1206_2.png) [@Mateusz\_Kaczynski](https://discuss.elastic.co/u/Mateusz_Kaczynski)\
**Post date:** [May 29, 2014, 9:57am UTC](https://discuss.elastic.co/t/failed-to-send-release-search-context-sourcelookup-fetching-cost/17753/3 "2014-05-29T09:57:00Z")

</div>

Anyone encoutered something similar / related?

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/498619f8-f05b-44ca-8846-3e15a15946c6%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/498619f8-f05b-44ca-8846-3e15a15946c6%40googlegroups.com).  
For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 1:26am UTC](https://discuss.elastic.co/t/failed-to-send-release-search-context-sourcelookup-fetching-cost/17753/4 "2017-07-06T01:26:10Z")

</div>


