# Pushback to hadoop **URL:** \ **Category:** Elasticsearch\ **Created:** [May 28, 2015, 11:00pm UTC](https://discuss.elastic.co/t/pushback-to-hadoop/1524 "2015-05-28T23:00:06Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![koert](https://avatars.discourse-cdn.com/v4/letter/k/a587f6/32.png) [@koert](https://discuss.elastic.co/u/koert)\ **Post date:** [May 28, 2015, 11:00pm UTC](https://discuss.elastic.co/t/pushback-to-hadoop/1524/1 "2015-05-28T23:00:06Z")
When we load data from hadoop into elasticsearch, we keep seeing errors in the tasks like this: org.elasticsearch.hadoop.EsHadoopException: Could not write all entries [99/347072] (maybe ES was overloaded?). Bailing out... Since our hadoop cluster can load/read data at an enormous rate i am not surprised our (much smaller) elasticsearch cluster can not keep up. Fair enough. So this question is not about optimizing elasticsearch for faster indexing. My question is: why can elasticsearch not do some kind of pushback to slow down the hadoop job to a speed that is acceptable for elasticsearch? It seems elasticsearch will happily keep on ingesting data at a rate it simply cannot sustain... ---
**Author:** ![koert](https://avatars.discourse-cdn.com/v4/letter/k/a587f6/32.png) [@koert](https://discuss.elastic.co/u/koert)\ **Post date:** [May 29, 2015, 3:30am UTC](https://discuss.elastic.co/t/pushback-to-hadoop/1524/2 "2015-05-29T03:30:09Z")
> [@koert](#): > > When we load data from hadoop into elasticsearch, we keep seeing errors in the tasks like this:org.elasticsearch.hadoop.EsHadoopException: Could not write all entries [99/347072] (maybe ES was overloaded?). Bailing out... > > Since our hadoop cluster can load/read data at an enormous rate i am not surprised our (much smaller) elasticsearch cluster can not keep up. Fair enough. So this question is not about optimizing elasticsearch for faster indexing. > > My question is: why can elasticsearch not do some kind of pushback to slow down the hadoop job to a speed that is acceptable for elasticsearch? It seems elasticsearch will happily keep on ingesting data at a rate it simply cannot sustain... oh i just reaized there is a forum for hadoop related stuff. moving this over to that one... ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:10am UTC](https://discuss.elastic.co/t/pushback-to-hadoop/1524/3 "2017-07-06T00:10:59Z")