I am using Logstash version 1.5.1 to parse a log file and create or update documents on Elasticsearch. In one of the cases, I am trying to delete an existing document from elasticsearch using logstash configuration file. But I get the following exception infinitely:
Got error to send bulk of actions: [500] {"error":"IllegalArgumentException[Malformed action/metadata line [2], expected START_OBJECT or END_OBJECT but found [VALUE_STRING]]","status":500} {:level=>:error}
Failed to flush outgoing items {:outgoing_count=>9, :exception=>#<Elasticsearch::Transport::Transport::Errors::InternalServerError: [500] {"error":"IllegalArgumentException[Malformed action/metadata line [2], expected START_OBJECT or END_OBJECT but found [VALUE_STRING]]","status":500}>, :backtrace=>["/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/base.rb:135:in `__raise_transport_error'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/base.rb:227:in `perform_request'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/http/manticore.rb:54:in `perform_request'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/client.rb:119:in `perform_request'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-api-1.0.12/lib/elasticsearch/api/actions/bulk.rb:80:in `bulk'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-0.2.8-java/lib/logstash/outputs/elasticsearch/protocol.rb:103:in `bulk'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-0.2.8-java/lib/logstash/outputs/elasticsearch.rb:466:in `submit'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-0.2.8-java/lib/logstash/outputs/elasticsearch.rb:490:in `flush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.19/lib/stud/buffer.rb:219:in `buffer_flush'", "org/jruby/RubyHash.java:1341:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.19/lib/stud/buffer.rb:216:in `buffer_flush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.19/lib/stud/buffer.rb:193:in `buffer_flush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.19/lib/stud/buffer.rb:112:in `buffer_initialize'", "org/jruby/RubyKernel.java:1511:in `loop'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.19/lib/stud/buffer.rb:110:in `buffer_initialize'"], :level=>:warn}
^CSIGINT received. Terminating immediately.. {:level=>:fatal}
Here is my logstash output configuration:
output {
if !("_grokparsefailure" in [tags]) {
if [type] == "DGL" {
if [facility_id] != "MDN" {
elasticsearch {
document_id => "%{messageID}"
template_overwrite => true
template_name => syndromic
template => "/opt/logstash/template.json"
protocol => "http"
codec => "plain"
manage_template => true
host => "orion-mao-devehsreporting1v"
index => "syndromic-%{+YYYY.MM}"
}
} else {
elasticsearch {
document_id => "%{messageID}"
protocol => "http"
host => "orion-mao-devehsreporting1v"
index => "syndromic-%{+YYYY.MM}"
action => "delete"
}
}
}
if [type] == "CAL" {
elasticsearch {
template_overwrite => true
template_name => syndromic
template => "/opt/logstash/template.json"
protocol => "http"
codec => "plain"
manage_template => true
host => "orion-mao-devehsreporting1v"
index => "syndromic-%{+YYYY.MM}"
}
}
}
else {
stdout { codec => rubydebug }
}
}
The part of code that is not working is where action => "delete"
Please help me out here.
Thanks.