# Unable to resolve Gateway Timeout errors **URL:** \ **Category:** Kibana\ **Created:** [July 14, 2015, 11:12pm UTC](https://discuss.elastic.co/t/unable-to-resolve-gateway-timeout-errors/25560 "2015-07-14T23:12:40Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![chromeriver\_tk](https://avatars.discourse-cdn.com/v4/letter/c/91b2a8/32.png) [@chromeriver\_tk](https://discuss.elastic.co/u/chromeriver_tk)\ **Post date:** [July 14, 2015, 11:12pm UTC](https://discuss.elastic.co/t/unable-to-resolve-gateway-timeout-errors/25560/1 "2015-07-14T23:12:40Z")
At our organization, we have 7 primary shard Elastic Search servers serving a single Kibana frontend. I have set `request_timeout` in the Kibana settings to `600000`, and yet I still get these Kibana "Gateway Timeout" exceptions. The stack trace I see in my browser for Kibana is: ```auto Error: Gateway Timeout at respond (https://hostname.com/index.js?_b=7489:85288:15) at checkRespForFailure (https://hostname.com/index.js?_b=7489:85256:7) at https://hostname.com/index.js?_b=7489:83894:7 at wrappedErrback (https://hostname.com/index.js?_b=7489:20902:78) at wrappedErrback (https://hostname.com/index.js?_b=7489:20902:78) at wrappedErrback (https://hostname.com/index.js?_b=7489:20902:78) at https://hostname.com/index.js?_b=7489:21035:76 at Scope.$eval (https://hostname.com/index.js?_b=7489:22022:28) at Scope.$digest (https://hostname.com/index.js?_b=7489:21834:31) at Scope.$apply (https://hostname.com/index.js?_b=7489:22126:24) ``` We can't seem to get rid of this issue. We are running all of these instances in EC2 in a VPC on Amazon Web Services. All 7 Elastic Search servers are behind an Elastic Load Balancer which forwards traffic to Elastic Search over port 9200. If I repeatedly `curl` the ELB, I can see that I'm taken round-robin through the list of hosts. All of the Elastic Search servers respond to requests, so I don't think that one server failing is causing Kibana to fail hard. Why am I getting this Gateway Timeout exception and what can I do to fix the problem? ---
**Author:** ![tbragin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tbragin/32/45166_2.png) [@tbragin](https://discuss.elastic.co/u/tbragin)\ **Post date:** [July 16, 2015, 5:18am UTC](https://discuss.elastic.co/t/unable-to-resolve-gateway-timeout-errors/25560/2 "2015-07-16T05:18:41Z")
What versions of Kibana and Elasticsearch are you running? ---
**Author:** ![chromeriver\_tk](https://avatars.discourse-cdn.com/v4/letter/c/91b2a8/32.png) [@chromeriver\_tk](https://discuss.elastic.co/u/chromeriver_tk)\ **Post date:** [July 16, 2015, 8:25pm UTC](https://discuss.elastic.co/t/unable-to-resolve-gateway-timeout-errors/25560/3 "2015-07-16T20:25:10Z")
Kibana is the latest, 4.1.1. Elastic Search is 1.5.0, we haven't upgraded to 1.6.0. Internally, NGINX throws a 499, which means that: > **499 Client Closed Request (Nginx)** > Used in Nginx logs to indicate when the connection has been closed by client while the server is still processing its request, making server unable to send a status code back. The client sees a 504, though this is internally a 499 in NGINX. I'm not sure who the client is, in this case. Is NGINX closing the connection between itself and the local Kibana? Is the browser terminating its connection to NGINX? Is Kibana's connection to the ELB being terminated? Kibana's own logs are pretty unhelpful here, nothing is logged. We set the connection idle timeout and the connection draining policy to 300 seconds on the ELB, which is 5 minutes, and the request fails well before 5 minutes. ---
**Author:** ![spalger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spalger/32/14092_2.png) [@spalger](https://discuss.elastic.co/u/spalger)\ **Post date:** [July 16, 2015, 10:09pm UTC](https://discuss.elastic.co/t/unable-to-resolve-gateway-timeout-errors/25560/4 "2015-07-16T22:09:52Z")
The client that is logging this error is the elasticsearch.js client. It is receiving a 504 status code from the browser, so I'm really don't think there is anything you can do in Kibana to make this work. What browser are you using? Have you checked the network traffic reported by the browser (Chrome exposes a [network debugging panel](https://developer.chrome.com/devtools/docs/network#network-panel-overview) for instance)? Sometimes the text of the response can provide a clue about what part of the stack terminated the response. When the kibana server (the one shipping with 4.1.1 at least) receives an error from elasticsearch it may [respond with a 502](https://github.com/elastic/kibana/blob/bd4fcb654a2fa062c9a99853ac5f776965ab7637/src/server/routes/proxy.js#L117-L130) but since you're getting a 504 it must be coming from AWS, or something sitting between the kibana server and AWS. ---
**Author:** ![kitty779](https://avatars.discourse-cdn.com/v4/letter/k/439d5e/32.png) [@kitty779](https://discuss.elastic.co/u/kitty779)\ **Post date:** [February 15, 2016, 4:47pm UTC](https://discuss.elastic.co/t/unable-to-resolve-gateway-timeout-errors/25560/5 "2016-02-15T16:47:40Z")
Hi, We are currently using Amazon elasticsearch service and Kibana 4.1.0 on ec2 instance. I set the request\_timeout in kibana config settings to a large number 41100000 and I am still getting the gateway timeout exception as below: Error: Gateway Timeout at respond ([http://hostname:5601/index.js?\_b=7616:86367:15](http://hostname:5601/index.js?_b=7616:86367:15)) at checkRespForFailure ([http://hostname:5601/index.js?\_b=7616:86335:7](http://hostname:5601/index.js?_b=7616:86335:7)) at [http://hostname:5601/index.js?\_b=7616:84973:7](http://hostname:5601/index.js?_b=7616:84973:7) at wrappedErrback ([http://hostname:5601/index.js?\_b=7616:20902:78](http://hostname:5601/index.js?_b=7616:20902:78)) at wrappedErrback ([http://hostname:5601/index.js?\_b=7616:20902:78](http://hostname:5601/index.js?_b=7616:20902:78)) at wrappedErrback ([http://hostname:5601/index.js?\_b=7616:20902:78](http://hostname:5601/index.js?_b=7616:20902:78)) at [http://hostname:5601/index.js?\_b=7616:21035:76](http://hostname:5601/index.js?_b=7616:21035:76) at Scope.$eval ([http://hostname:5601/index.js?\_b=7616:22022:28](http://hostname:5601/index.js?_b=7616:22022:28)) at Scope.$digest ([http://hostname:5601/index.js?\_b=7616:21834:31](http://hostname:5601/index.js?_b=7616:21834:31)) at Scope.$apply ([http://hostname:5601/index.js?\_b=7616:22126:24](http://hostname:5601/index.js?_b=7616:22126:24)) We noticed that this is error is occurring if we have too many aggregations on the dashboards across large documents. For our current requirement, we want to get the results even if it takes more time. Could you please tell us why we have this issue and how to resolve it? ---
**Author:** ![gh0stid](https://avatars.discourse-cdn.com/v4/letter/g/f9ae1b/32.png) [@gh0stid](https://discuss.elastic.co/u/gh0stid)\ **Post date:** [July 26, 2016, 3:54pm UTC](https://discuss.elastic.co/t/unable-to-resolve-gateway-timeout-errors/25560/6 "2016-07-26T15:54:37Z")
> [@kitty779](#): > > We noticed that this is error is occurring if we have too many aggregations on the dashboards across large documents. For our current requirement, we want to get the results even if it takes more time. Could you please tell us why we have this issue and how to resolve it? did you get an answer to your situation ? it seem I have the same problem aswell Thank you ---
**Author:** ![kitty779](https://avatars.discourse-cdn.com/v4/letter/k/439d5e/32.png) [@kitty779](https://discuss.elastic.co/u/kitty779)\ **Post date:** [July 27, 2016, 3:18pm UTC](https://discuss.elastic.co/t/unable-to-resolve-gateway-timeout-errors/25560/7 "2016-07-27T15:18:51Z")
Hi gh0stid, Yes. Elasticsearch needs lot of memory to perform the aggregations so even if the cluster health is green, we happened to have the same shard issues at our end on Kibana. We solved for it by upgrading our ec2 instance. But, this is only a temporary solution as we cannot upgrade the cluster forever(the cost would be high too). I then took the snapshot of our cluster, stored the backup on S3 and purged the data older than 2 months. This helped us to resolve for the shard failures and also save our data. Whenever we want the data from S3, we could always restore this data in a new cluster and perform the aggregations as usual. ---
**Author:** ![RomanMauricio](https://avatars.discourse-cdn.com/v4/letter/r/57b2e6/32.png) [@RomanMauricio](https://discuss.elastic.co/u/RomanMauricio)\ **Post date:** [March 3, 2017, 8:19pm UTC](https://discuss.elastic.co/t/unable-to-resolve-gateway-timeout-errors/25560/8 "2017-03-03T20:19:33Z")
I had the same issue. The solution is to increase the ELB idle timeout to something above 60s, which is the default. The instructions on how to do that are here: [https://aws.amazon.com/blogs/aws/elb-idle-timeout-control/](https://aws.amazon.com/blogs/aws/elb-idle-timeout-control/) When the ELB times out, upstream from the ELB you will see a 499 error (Client Closed Connection) and downstream, you will see a 504 error (Gateway Timeout) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:32pm UTC](https://discuss.elastic.co/t/unable-to-resolve-gateway-timeout-errors/25560/9 "2017-07-06T13:32:52Z")