# "Failed to flush outgoing items" Error with logstash-output-webhdfs plugin

**URL:** https://discuss.elastic.co/t/failed-to-flush-outgoing-items-error-with-logstash-output-webhdfs-plugin/80601
**Category:** Logstash
**Created:** [March 30, 2017, 7:01am UTC](https://discuss.elastic.co/t/failed-to-flush-outgoing-items-error-with-logstash-output-webhdfs-plugin/80601 "2017-03-30T07:01:19Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![jackhe](https://avatars.discourse-cdn.com/v4/letter/j/6f9a4e/32.png) [@jackhe](https://discuss.elastic.co/u/jackhe)
#### Post date: [March 30, 2017, 7:01am UTC](https://discuss.elastic.co/t/failed-to-flush-outgoing-items-error-with-logstash-output-webhdfs-plugin/80601/1 "2017-03-30T07:01:19Z")

</div>

Dear all,

When I tried use logstash-output-webhdfs plugin to archive logs into hdfs, I met below error occasionly(not always). Does anyone know what happened and how to fix this ?

Failed to flush outgoing items {:outgoing\_count=\>168, :exception=\>"WebHDFS::IOError", :backtrace=\>["/home/cloudera/elasticsearch/logstash/logstash/vendor/bundle/jruby/1.9/gems/webhdfs-0.8.0/lib/webhdfs/client\_v1.rb:401:in `request'", "/home/cloudera/elasticsearch/logstash/logstash/vendor/bundle/jruby/1.9/gems/webhdfs-0.8.0/lib/webhdfs/client_v1.rb:270:in`operate\_requests'", "/home/cloudera/elasticsearch/logstash/logstash/vendor/bundle/jruby/1.9/gems/webhdfs-0.8.0/lib/webhdfs/client\_v1.rb:73:in `create'", "/home/cloudera/elasticsearch/logstash/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-webhdfs-3.0.2/lib/logstash/outputs/webhdfs.rb:210:in`write\_data'", "/home/cloudera/elasticsearch/logstash/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-webhdfs-3.0.2/lib/logstash/outputs/webhdfs.rb:205:in `write_data'", "/home/cloudera/elasticsearch/logstash/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-webhdfs-3.0.2/lib/logstash/outputs/webhdfs.rb:195:in`flush'", "org/jruby/RubyHash.java:1342:in `each'", "/home/cloudera/elasticsearch/logstash/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-webhdfs-3.0.2/lib/logstash/outputs/webhdfs.rb:183:in`flush'", "/home/cloudera/elasticsearch/logstash/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/buffer.rb:219:in `buffer_flush'", "org/jruby/RubyHash.java:1342:in`each'", "/home/cloudera/elasticsearch/logstash/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/buffer.rb:216:in `buffer_flush'", "/home/cloudera/elasticsearch/logstash/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/buffer.rb:193:in`buffer\_flush'", "/home/cloudera/elasticsearch/logstash/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/buffer.rb:112:in `buffer_initialize'", "org/jruby/RubyKernel.java:1479:in`loop'", "/home/cloudera/elasticsearch/logstash/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/buffer.rb:110:in `buffer\_initialize'"], :level=\>:warn}

my configuration as follows:  
webhdfs {  
codec =\> json  
host =\> "master"  
port =\> 50070  
path =\> "/user/logstash/access\_error/dt=%{log\_date}/logstash-%{+HH}.log"  
user =\> "hdfs"  
}

---

<div class="post-metadata">

### Author: ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)
#### Post date: [March 31, 2017, 4:44pm UTC](https://discuss.elastic.co/t/failed-to-flush-outgoing-items-error-with-logstash-output-webhdfs-plugin/80601/2 "2017-03-31T16:44:17Z")

</div>

Hello @jackhe,

Is there any mention of retries in your log before the error you shared with us? From the source code, the plugin should [retry theses kinds of errors](https://github.com/logstash-plugins/logstash-output-webhdfs/blob/master/lib/logstash/outputs/webhdfs.rb#L230).

---

<div class="post-metadata">

### Author: ![jackhe](https://avatars.discourse-cdn.com/v4/letter/j/6f9a4e/32.png) [@jackhe](https://discuss.elastic.co/u/jackhe)
#### Post date: [April 1, 2017, 3:34am UTC](https://discuss.elastic.co/t/failed-to-flush-outgoing-items-error-with-logstash-output-webhdfs-plugin/80601/3 "2017-04-01T03:34:19Z")

</div>

Hi @pierhugues,

Thanks a lot for your response.

No, the error I shared is the whole thing. And when it happens, It keeps coming with every event.

I didn't modify retries\_times in my conf file.

Now I am trying "retry\_times =\> 100", so far so good.

---

<div class="post-metadata">

### Author: ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)
#### Post date: [April 7, 2017, 1:23pm UTC](https://discuss.elastic.co/t/failed-to-flush-outgoing-items-error-with-logstash-output-webhdfs-plugin/80601/4 "2017-04-07T13:23:47Z")

</div>


