# ES Cluster via proxy IP, but listener timeout after waiting for \[30000\] ms

**URL:** <https://discuss.elastic.co/t/es-cluster-via-proxy-ip-but-listener-timeout-after-waiting-for-30000-ms/116192>\
**Category:** Elasticsearch\
**Created:** [January 19, 2018, 9:01am UTC](https://discuss.elastic.co/t/es-cluster-via-proxy-ip-but-listener-timeout-after-waiting-for-30000-ms/116192 "2018-01-19T09:01:28Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![lauea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lauea/32/26743_2.png) [@lauea](https://discuss.elastic.co/u/lauea)\
**Post date:** [January 19, 2018, 9:01am UTC](https://discuss.elastic.co/t/es-cluster-via-proxy-ip-but-listener-timeout-after-waiting-for-30000-ms/116192/1 "2018-01-19T09:01:29Z")

</div>

Hi Everyone,

I ran into below issue.  
My ES cluster has 3 nodes, and I setup one proxy IP, you can regard it as Loadbalance IP, which points to 3 nodes with ip\_hash.

**ES Version: 6.0.0**

But when I call search API via es rest client, it throws below exception.

> Something wrong with middleware: java.io.IOException: listener timeout after waiting for [30000] ms at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:661) at org.elasticsearch.client.RestClient.performRequest(RestClient.java:220) at org.elasticsearch.client.RestClient.performRequest(RestClient.java:192) at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:428) at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:414) at org.elasticsearch.client.RestHighLevelClient.search(RestHighLevelClient.java:355) at com.cargosmart.b2b.fwk2.process.bkt.dao.impl.BizkeyDaoESImpl.retrieveTransactionId(BizkeyDaoESImpl.java:152) at com.cargosmart.b2b.fwk2.process.bkt.controller.BizkeyTrackMiddleware.process(BizkeyTrackMiddleware.java:62) at org.apache.camel.processor.DelegateSyncProcessor.process(DelegateSyncProcessor.java:63) at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77) at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:541) at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:198) at org.apache.camel.processor.Pipeline.process(Pipeline.java:120) at org.apache.camel.processor.Pipeline.process(Pipeline.java:83) at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:198) at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:97) at org.apache.camel.component.jms.EndpointMessageListener.onMessage(EndpointMessageListener.java:112)

I also tried it in server side.  
For bigIP via telnet, it works:

```bash
[user@elk ~]$ curl http://bigIP:9200                                                  
 {
  "name" : "node1",
  "cluster_name" : "mycluster",
  "cluster_uuid" : "xt66V3oDTP2FsXj66eIWIQ",
  "version" : {
    "number" : "6.0.0",
    "build_hash" : "8f0685b",
    "build_date" : "2017-11-10T18:41:22.859Z",
    "build_snapshot" : false,
    "lucene_version" : "7.0.1",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

```

But stuck in below search API and without response:

```bash
[user@elk ~]$ curl -XGET 'http://bigIP:9200/_search?pretty' -H 'Content-Type: application/json' -d'{"query": {"bool": {"must": [{ "match": { "title":"Search"}}, { "match": { "content": "Elasticsearch" }}],"filter": [{"term": { "status": "published" }}, { "range": { "publish_date": { "gte": "2015-01-01" }}}]}}}'          

```

Per trying in one es server node by curl, all nodes are running normally

```bash
[user@elk ~]$ curl -XGET 'http://node1:9200/_search?pretty' -H 'Content-Type: application/json' -d'{"query": {"bool": {"must": [{ "match": { "title":"Search"}}, { "match": { "content": "Elasticsearch" }}],"filter": [{"term": { "status": "published" }}, { "range": { "publish_date": { "gte": "2015-01-01" }}}]}}}'          
 {
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 6,
    "successful" : 6,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 0,
    "max_score" : null,
    "hits" : []
  }
}

```

**Very very strange** , I can search full record via bigip:9200 in command line based:

```bash
[user@elk ~]$ curl -XGET http://bigIP:9200/_search?pretty

```

However not for below one:

```bash
[user@elk ~]$ curl -XGET 'http://bigIP:9200/_search?pretty' -H 'Content-Type: application/json' -d'{"query": {"bool": {"must": [{ "match": { "title":"Search"}}, { "match": { "content": "Elasticsearch" }}],"filter": [{"term": { "status": "published" }}, { "range": { "publish_date": { "gte": "2015-01-01" }}}]}}}'    

```

Anyone has the same issue? Please help me jump out of this damn thing. Thanks advanced.

Something I can confirm is as below:

- Not rest API version compatability issue.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 19, 2018, 9:25am UTC](https://discuss.elastic.co/t/es-cluster-via-proxy-ip-but-listener-timeout-after-waiting-for-30000-ms/116192/2 "2018-01-19T09:25:34Z")

</div>

This might be a limitation of your proxy. It might not support `GET` with body.  
Try to run the same request with `POST` instead.

---

<div class="post-metadata">

**Author:** ![lauea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lauea/32/26743_2.png) [@lauea](https://discuss.elastic.co/u/lauea)\
**Post date:** [January 19, 2018, 9:43am UTC](https://discuss.elastic.co/t/es-cluster-via-proxy-ip-but-listener-timeout-after-waiting-for-30000-ms/116192/3 "2018-01-19T09:43:15Z")

</div>

@dadoonet, sounds a good advise. Let's jump into the proxy server setting to dig it out.

---

<div class="post-metadata">

**Author:** ![lauea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lauea/32/26743_2.png) [@lauea](https://discuss.elastic.co/u/lauea)\
**Post date:** [January 22, 2018, 12:57am UTC](https://discuss.elastic.co/t/es-cluster-via-proxy-ip-but-listener-timeout-after-waiting-for-30000-ms/116192/4 "2018-01-22T00:57:36Z")

</div>

Hi @dadoonet , may I know, at es rest client side, if call es with **curl -XGET** backend?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 22, 2018, 4:18am UTC](https://discuss.elastic.co/t/es-cluster-via-proxy-ip-but-listener-timeout-after-waiting-for-30000-ms/116192/5 "2018-01-22T04:18:48Z")

</div>

I don't understand your question. Could you rephrase it?

---

<div class="post-metadata">

**Author:** ![lauea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lauea/32/26743_2.png) [@lauea](https://discuss.elastic.co/u/lauea)\
**Post date:** [January 22, 2018, 5:51am UTC](https://discuss.elastic.co/t/es-cluster-via-proxy-ip-but-listener-timeout-after-waiting-for-30000-ms/116192/6 "2018-01-22T05:51:05Z")

</div>

Means that if we use Java rest client to search es result, like:

```auto
SearchRequest searchRequest = new SearchRequest(); 
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); 
searchSourceBuilder.query(QueryBuilders.matchAllQuery());

```

If request with **curl -XGET** which java rest client used? Thanks.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 22, 2018, 7:58am UTC](https://discuss.elastic.co/t/es-cluster-via-proxy-ip-but-listener-timeout-after-waiting-for-30000-ms/116192/7 "2018-01-22T07:58:38Z")

</div>

We are using `GET` indeed. I just checked the code.

I opened this

> <https://github.com/elastic/elasticsearch/issues/28326>

For discussion with the team.

---

<div class="post-metadata">

**Author:** ![lauea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lauea/32/26743_2.png) [@lauea](https://discuss.elastic.co/u/lauea)\
**Post date:** [January 22, 2018, 9:09am UTC](https://discuss.elastic.co/t/es-cluster-via-proxy-ip-but-listener-timeout-after-waiting-for-30000-ms/116192/8 "2018-01-22T09:09:13Z")

</div>

Thanks.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 22, 2018, 9:21am UTC](https://discuss.elastic.co/t/es-cluster-via-proxy-ip-but-listener-timeout-after-waiting-for-30000-ms/116192/9 "2018-01-22T09:21:02Z")

</div>

Could you confirm that using curl with `GET` does not work but curl with `POST` works?

And also, what is the proxy you are using?

---

<div class="post-metadata">

**Author:** ![lauea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lauea/32/26743_2.png) [@lauea](https://discuss.elastic.co/u/lauea)\
**Post date:** [January 22, 2018, 9:37am UTC](https://discuss.elastic.co/t/es-cluster-via-proxy-ip-but-listener-timeout-after-waiting-for-30000-ms/116192/10 "2018-01-22T09:37:19Z")

</div>

```bash
# hung without response
[user@elk~]$ curl -XGET 'http://bigIP:9200/_search?pretty' -H 'Content-Type: application/json' -d'
> {
> "query": {
> "bool": {
> "must": [
> { "match": { "title": "Search" }},
> { "match": { "content": "Elasticsearch" }}
> ],
> "filter": [
> { "term": { "status": "published" }},
> { "range": { "publish_date": { "gte": "2015-01-01" }}}
> ]
> }
> }
> }
> '

^C

# response with result
[user@elk ~]$ curl -XPOST 'http://bigIP:9200/_search?pretty' -H 'Content-Type: application/json' -d'
{
  "query": {
    "bool": {
      "must": [
        { "match": { "title": "Search" }},
        { "match": { "content": "Elasticsearch" }}
      ],
      "filter": [
        { "term": { "status": "published" }},
        { "range": { "publish_date": { "gte": "2015-01-01" }}}
      ]
    }
  }
}
'
{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 6,
    "successful" : 6,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 0,
    "max_score" : null,
    "hits" : []
  }
}
[user@elk ~]$

```

I use F5 BIG IP.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [January 22, 2018, 9:49am UTC](https://discuss.elastic.co/t/es-cluster-via-proxy-ip-but-listener-timeout-after-waiting-for-30000-ms/116192/11 "2018-01-22T09:49:53Z")

</div>

Thanks! We will fix this unless you are a Java dev and you want to propose a PR. 😛

---

<div class="post-metadata">

**Author:** ![lauea](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lauea/32/26743_2.png) [@lauea](https://discuss.elastic.co/u/lauea)\
**Post date:** [January 22, 2018, 9:52am UTC](https://discuss.elastic.co/t/es-cluster-via-proxy-ip-but-listener-timeout-after-waiting-for-30000-ms/116192/12 "2018-01-22T09:52:22Z")

</div>

You expert dev are preferred. 😀

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 19, 2018, 9:52am UTC](https://discuss.elastic.co/t/es-cluster-via-proxy-ip-but-listener-timeout-after-waiting-for-30000-ms/116192/13 "2018-02-19T09:52:41Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.