LS Stops writing to ES every couple hours


(Tim Dunphy) #1

Hello,

Since I've been able to get LS 1.5.1 working with ES 1.6.0 and kibana 4 it ran for a few weeks without complaint.

But today I noticed that when I do a search in Kibana, there are no results.. Unless I use the time picker to select a time period from yesterday.

If I restart logstash it'll run for about an hour and a half or so before it stops returning results to the Kibana web interface. And when I take a look at the LS logs, this is what I'm seeing that probably relates to the problem:

{:timestamp=>"2015-07-07T12:41:52.498000-0400", :message=>"Failed to flush outgoing items", :outgoing_count=>199, :exception=>#<NameError: no method 'type' for arguments (org.jruby.RubyArray) on Java::OrgElasticsearchActionIndex::IndexRequest>, :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-0.2.8-java/lib/logstash/outputs/elasticsearch/protocol.rb:261:in `build_request'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-0.2.8-java/lib/logstash/outputs/elasticsearch/protocol.rb:222:in `bulk'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-0.2.8-java/lib/logstash/outputs/elasticsearch/protocol.rb:221:in `bulk'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-0.2.8-java/lib/logstash/outputs/elasticsearch.rb:466:in `submit'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-0.2.8-java/lib/logstash/outputs/elasticsearch.rb:465:in `submit'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-0.2.8-java/lib/logstash/outputs/elasticsearch.rb:490:in `flush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-0.2.8-java/lib/logstash/outputs/elasticsearch.rb:489:in `flush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.19/lib/stud/buffer.rb:219:in `buffer_flush'", "org/jruby/RubyHash.java:1341:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.19/lib/stud/buffer.rb:216:in `buffer_flush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.19/lib/stud/buffer.rb:193:in `buffer_flush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.19/lib/stud/buffer.rb:159:in `buffer_receive'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-0.2.8-java/lib/logstash/outputs/elasticsearch.rb:455:in `receive'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.1-java/lib/logstash/outputs/base.rb:88:in `handle'", "(eval):402:in `output_func'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.1-java/lib/logstash/pipeline.rb:243:in `outputworker'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.1-java/lib/logstash/pipeline.rb:165:in `start_outputs'"], :level=>:warn}



{:timestamp=>"2015-07-07T12:41:52.498000-0400", :message=>"Got error to send bulk of actions: no method 'type' for arguments (org.jruby.RubyArray) on Java::OrgElasticsearchActionIndex::IndexRequest", :level=>:error
}

What could the problem really be? And how do we resolve it?

Thanks


(system) #2