# Frustrated with REST API **URL:** \ **Category:** Elasticsearch\ **Created:** [August 2, 2014, 2:46pm UTC](https://discuss.elastic.co/t/frustrated-with-rest-api/19045 "2014-08-02T14:46:04Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![crimondi](https://avatars.discourse-cdn.com/v4/letter/c/f05b48/32.png) [@crimondi](https://discuss.elastic.co/u/crimondi)\ **Post date:** [August 2, 2014, 2:46pm UTC](https://discuss.elastic.co/t/frustrated-with-rest-api/19045/1 "2014-08-02T14:46:04Z")
This is going to sound like a bit of a gripe session so I apologize in advance. There seems to be a lot of instability and ineffectiveness around using the REST API to make configuration changes. I realize there have been some issues related to the NPE returns on certain calls. In addition to those problems (which I believe have been addressed in ES 1.2.x and 1.3.x) I have found that if the cluster is in anything but a pristine state the calls simply do not return or error out with a 503 response. Activities such as changing the number of replicas on certain indices or modifying throttle settings almost always return a 503 on a cluster that is yellow. It is when the cluster is in a degraded state that we need these calls the most! Also, simple information calls such as /\_cat/nodes will many times not return when the cluster is yellow. Sometimes it appears that an API call is hanging only to find out that the setting really did take. We maintain multiple ES clusters internally and all the tooling we have built around supporting them simply assumes the acknowledgement returned from the API calls is unreliable. Can we expect better reliability with the Java APIs? Is there plans to make the RESTful calls more robust? Thanks, Chris -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6c44ad78-d699-4ec0-937c-15322914f924%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6c44ad78-d699-4ec0-937c-15322914f924%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [August 2, 2014, 2:54pm UTC](https://discuss.elastic.co/t/frustrated-with-rest-api/19045/2 "2014-08-02T14:54:33Z")
This looks a a network configuration challenge. You should review your whole network setup. Look at network device config, host names, gateway setup, DNS name resolution, routers/switches (if your ES clusters spans over subnetworks). If your network setup is not 100% solid, requests over the wire will timeout, fail, hang, etc. ES (not only ES) can not remedy such situations from the inside of an application. You should not blame REST API, with Java API, it would be the same situation. Jörg On Sat, Aug 2, 2014 at 4:46 PM, Chris Rimondi [chris.rimondi@gmail.com](mailto:chris.rimondi@gmail.com) wrote: > This is going to sound like a bit of a gripe session so I apologize in > advance. There seems to be a lot of instability and ineffectiveness around > using the REST API to make configuration changes. I realize there have been > some issues related to the NPE returns on certain calls. In addition to > those problems (which I believe have been addressed in ES 1.2.x and 1.3.x) > I have found that if the cluster is in anything but a pristine state the > calls simply do not return or error out with a 503 response. > > Activities such as changing the number of replicas on certain indices or > modifying throttle settings almost always return a 503 on a cluster that is > yellow. It is when the cluster is in a degraded state that we need these > calls the most! Also, simple information calls such as /\_cat/nodes will > many times not return when the cluster is yellow. Sometimes it appears that > an API call is hanging only to find out that the setting really did take. > > We maintain multiple ES clusters internally and all the tooling we have > built around supporting them simply assumes the acknowledgement returned > from the API calls is unreliable. Can we expect better reliability with the > Java APIs? Is there plans to make the RESTful calls more robust? > > Thanks, > > Chris > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/6c44ad78-d699-4ec0-937c-15322914f924%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6c44ad78-d699-4ec0-937c-15322914f924%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/6c44ad78-d699-4ec0-937c-15322914f924%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/6c44ad78-d699-4ec0-937c-15322914f924%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEU%2BtvKwO4yxq4050Ne7pYntDjPLRFU%3D6fbO5%2BuANcrVA%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEU%2BtvKwO4yxq4050Ne7pYntDjPLRFU%3D6fbO5%2BuANcrVA%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![crimondi](https://avatars.discourse-cdn.com/v4/letter/c/f05b48/32.png) [@crimondi](https://discuss.elastic.co/u/crimondi)\ **Post date:** [August 2, 2014, 3:10pm UTC](https://discuss.elastic.co/t/frustrated-with-rest-api/19045/3 "2014-08-02T15:10:15Z")
I have considered network before but I don't understand a few things: 1. Why would these API calls work consistently if the cluster was green versus yellow or red? 2. Before I make these calls I can check the health of the cluster and 100% of the nodes are "checked in". Therefore it is not like network connectivity is preventing my data nodes from communicating to the masters. 3. We run multiple clusters. Some are split across subnets and some are not. However, this API badness takes place in both scenarios. In short I have seen 0 evidence where network latency or flakiness has caused any cluster issues. I am open to any type of testing that might disprove that, but I really don't think this is network connectivity problems. On Saturday, August 2, 2014 10:54:41 AM UTC-4, Jörg Prante wrote: > This looks a a network configuration challenge. You should review your > whole network setup. Look at network device config, host names, gateway > setup, DNS name resolution, routers/switches (if your ES clusters spans > over subnetworks). > > If your network setup is not 100% solid, requests over the wire will > timeout, fail, hang, etc. > > ES (not only ES) can not remedy such situations from the inside of an > application. You should not blame REST API, with Java API, it would be the > same situation. > > Jörg > > On Sat, Aug 2, 2014 at 4:46 PM, Chris Rimondi \<[chris....@gmail.com](mailto:chris....@gmail.com) > \\> wrote: > > > This is going to sound like a bit of a gripe session so I apologize in > > advance. There seems to be a lot of instability and ineffectiveness around > > using the REST API to make configuration changes. I realize there have been > > some issues related to the NPE returns on certain calls. In addition to > > those problems (which I believe have been addressed in ES 1.2.x and 1.3.x) > > I have found that if the cluster is in anything but a pristine state the > > calls simply do not return or error out with a 503 response. > > > > Activities such as changing the number of replicas on certain indices or > > modifying throttle settings almost always return a 503 on a cluster that is > > yellow. It is when the cluster is in a degraded state that we need these > > calls the most! Also, simple information calls such as /\_cat/nodes will > > many times not return when the cluster is yellow. Sometimes it appears that > > an API call is hanging only to find out that the setting really did take. > > > > We maintain multiple ES clusters internally and all the tooling we have > > built around supporting them simply assumes the acknowledgement returned > > from the API calls is unreliable. Can we expect better reliability with the > > Java APIs? Is there plans to make the RESTful calls more robust? > > > > Thanks, > > > > Chris > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/6c44ad78-d699-4ec0-937c-15322914f924%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6c44ad78-d699-4ec0-937c-15322914f924%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/6c44ad78-d699-4ec0-937c-15322914f924%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/6c44ad78-d699-4ec0-937c-15322914f924%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/29b5bf60-e08a-4885-a796-c0590f338da7%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/29b5bf60-e08a-4885-a796-c0590f338da7%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [August 2, 2014, 6:28pm UTC](https://discuss.elastic.co/t/frustrated-with-rest-api/19045/4 "2014-08-02T18:28:11Z")
Because I don't know your system setup, I have to do wild guessing. If it is not the network, can you observe frequent index recovery or shard reallocations? 503 (service unavailable) is a code that is sent when shards are not available. Have you set replica level by any chance or are indices replicas disabled? Jörg On Sat, Aug 2, 2014 at 5:10 PM, Chris Rimondi [chris.rimondi@gmail.com](mailto:chris.rimondi@gmail.com) wrote: > I have considered network before but I don't understand a few things: > > 1. Why would these API calls work consistently if the cluster was green > versus yellow or red? > 2. Before I make these calls I can check the health of the cluster and > 100% of the nodes are "checked in". Therefore it is not like network > connectivity is preventing my data nodes from communicating to the masters. > 3. We run multiple clusters. Some are split across subnets and some are > not. However, this API badness takes place in both scenarios. > > In short I have seen 0 evidence where network latency or flakiness has > caused any cluster issues. I am open to any type of testing that might > disprove that, but I really don't think this is network connectivity > problems. > > On Saturday, August 2, 2014 10:54:41 AM UTC-4, Jörg Prante wrote: > > > This looks a a network configuration challenge. You should review your > > whole network setup. Look at network device config, host names, gateway > > setup, DNS name resolution, routers/switches (if your ES clusters spans > > over subnetworks). > > > > If your network setup is not 100% solid, requests over the wire will > > timeout, fail, hang, etc. > > > > ES (not only ES) can not remedy such situations from the inside of an > > application. You should not blame REST API, with Java API, it would be the > > same situation. > > > > Jörg > > > > On Sat, Aug 2, 2014 at 4:46 PM, Chris Rimondi [chris....@gmail.com](mailto:chris....@gmail.com) > > wrote: > > > > > This is going to sound like a bit of a gripe session so I apologize in > > > advance. There seems to be a lot of instability and ineffectiveness around > > > using the REST API to make configuration changes. I realize there have been > > > some issues related to the NPE returns on certain calls. In addition to > > > those problems (which I believe have been addressed in ES 1.2.x and 1.3.x) > > > I have found that if the cluster is in anything but a pristine state the > > > calls simply do not return or error out with a 503 response. > > > > > > Activities such as changing the number of replicas on certain indices or > > > modifying throttle settings almost always return a 503 on a cluster that is > > > yellow. It is when the cluster is in a degraded state that we need these > > > calls the most! Also, simple information calls such as /\_cat/nodes will > > > many times not return when the cluster is yellow. Sometimes it appears that > > > an API call is hanging only to find out that the setting really did take. > > > > > > We maintain multiple ES clusters internally and all the tooling we have > > > built around supporting them simply assumes the acknowledgement returned > > > from the API calls is unreliable. Can we expect better reliability with the > > > Java APIs? Is there plans to make the RESTful calls more robust? > > > > > > Thanks, > > > > > > Chris > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > msgid/elasticsearch/6c44ad78-d699-4ec0-937c-15322914f924% > > > [40googlegroups.com](http://40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/6c44ad78-d699-4ec0-937c-15322914f924%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/6c44ad78-d699-4ec0-937c-15322914f924%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/29b5bf60-e08a-4885-a796-c0590f338da7%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/29b5bf60-e08a-4885-a796-c0590f338da7%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/29b5bf60-e08a-4885-a796-c0590f338da7%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/29b5bf60-e08a-4885-a796-c0590f338da7%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH5-agRncSA2dFioAKAsKOph7GPpp7%2BCRYZnUYk75OOAQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH5-agRncSA2dFioAKAsKOph7GPpp7%2BCRYZnUYk75OOAQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:11am UTC](https://discuss.elastic.co/t/frustrated-with-rest-api/19045/5 "2017-07-06T01:11:24Z")