# Logstash to delete document from Elasticsearch **URL:** \ **Category:** Logstash\ **Created:** [August 10, 2015, 5:14pm UTC](https://discuss.elastic.co/t/logstash-to-delete-document-from-elasticsearch/27142 "2015-08-10T17:14:16Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![umangjain](https://avatars.discourse-cdn.com/v4/letter/u/8e8cbc/32.png) [@umangjain](https://discuss.elastic.co/u/umangjain)\ **Post date:** [August 10, 2015, 5:14pm UTC](https://discuss.elastic.co/t/logstash-to-delete-document-from-elasticsearch/27142/1 "2015-08-10T17:14:16Z")
I am using Logstash version 1.5.1 to parse a log file and create or update documents on Elasticsearch. In one of the cases, I am trying to delete an existing document from elasticsearch using logstash configuration file. But I get the following exception infinitely: > _Got error to send bulk of actions: [500] {"error":"IllegalArgumentException[Malformed action/metadata line [2], expected START\_OBJECT or END\_OBJECT but found [VALUE\_STRING]]","status":500} {:level=\>:error}_ > _Failed to flush outgoing items {:outgoing\_count=\>9, :exception=\>#\, :backtrace=\>["/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/base.rb:135:in `__raise_transport_error'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/base.rb:227:in `perform\_request'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/http/manticore.rb:54:in `perform_request'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/client.rb:119:in `perform\_request'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-api-1.0.12/lib/elasticsearch/api/actions/bulk.rb:80:in `bulk'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-0.2.8-java/lib/logstash/outputs/elasticsearch/protocol.rb:103:in `bulk'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-0.2.8-java/lib/logstash/outputs/elasticsearch.rb:466:in `submit'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-0.2.8-java/lib/logstash/outputs/elasticsearch.rb:490:in `flush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.19/lib/stud/buffer.rb:219:in `buffer_flush'", "org/jruby/RubyHash.java:1341:in `each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.19/lib/stud/buffer.rb:216:in `buffer_flush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.19/lib/stud/buffer.rb:193:in `buffer\_flush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.19/lib/stud/buffer.rb:112:in `buffer_initialize'", "org/jruby/RubyKernel.java:1511:in `loop'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.19/lib/stud/buffer.rb:110:in `buffer\_initialize'"], :level=\>:warn}_ > _^CSIGINT received. Terminating immediately.. {:level=\>:fatal}_ Here is my logstash output configuration: ``` output { if !("_grokparsefailure" in [tags]) { if [type] == "DGL" { if [facility_id] != "MDN" { elasticsearch { document_id => "%{messageID}" template_overwrite => true template_name => syndromic template => "/opt/logstash/template.json" protocol => "http" codec => "plain" manage_template => true host => "orion-mao-devehsreporting1v" index => "syndromic-%{+YYYY.MM}" } } else { elasticsearch { document_id => "%{messageID}" protocol => "http" host => "orion-mao-devehsreporting1v" index => "syndromic-%{+YYYY.MM}" action => "delete" } } } if [type] == "CAL" { elasticsearch { template_overwrite => true template_name => syndromic template => "/opt/logstash/template.json" protocol => "http" codec => "plain" manage_template => true host => "orion-mao-devehsreporting1v" index => "syndromic-%{+YYYY.MM}" } } } else { stdout { codec => rubydebug } } } ``` The part of code that is not working is where **action =\> "delete"** Please help me out here. Thanks. ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [August 11, 2015, 10:42am UTC](https://discuss.elastic.co/t/logstash-to-delete-document-from-elasticsearch/27142/2 "2015-08-11T10:42:00Z")
Is there anything corresponding in the ES logs? ---
**Author:** ![umangjain](https://avatars.discourse-cdn.com/v4/letter/u/8e8cbc/32.png) [@umangjain](https://discuss.elastic.co/u/umangjain)\ **Post date:** [August 12, 2015, 2:18pm UTC](https://discuss.elastic.co/t/logstash-to-delete-document-from-elasticsearch/27142/3 "2015-08-12T14:18:20Z")
No, there is no activity on the ES logs.. I constantly see this error when I run logstash again after deleting the .since\_db files.. and running logstash again to parse the files.. Got error to send bulk of actions: [500] {"error":"IllegalArgumentException[Malformed action/metadata line [2], expected START\_OBJECT or END\_OBJECT but found [VALUE\_STRING]]","status":500} {:level=\>:error} Failed to flush outgoing items {:outgoing\_count=\>81, :exception=\>"Elasticsearch::Transport::Transport::Errors::InternalServerError", :backtrace=\>["/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/base.rb:135:in `__raise_transport_error'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/base.rb:227:in`perform\_request'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/transport/http/manticore.rb:54:in `perform_request'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-transport-1.0.12/lib/elasticsearch/transport/client.rb:119:in`perform\_request'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/elasticsearch-api-1.0.12/lib/elasticsearch/api/actions/bulk.rb:80:in `bulk'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-1.0.7-java/lib/logstash/outputs/elasticsearch/protocol.rb:104:in`bulk'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-1.0.7-java/lib/logstash/outputs/elasticsearch.rb:542:in `submit'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-elasticsearch-1.0.7-java/lib/logstash/outputs/elasticsearch.rb:566:in`flush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.21/lib/stud/buffer.rb:219:in `buffer_flush'", "org/jruby/RubyHash.java:1341:in`each'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.21/lib/stud/buffer.rb:216:in `buffer_flush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.21/lib/stud/buffer.rb:193:in`buffer\_flush'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.21/lib/stud/buffer.rb:112:in `buffer_initialize'", "org/jruby/RubyKernel.java:1511:in`loop'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.21/lib/stud/buffer.rb:110:in `buffer\_initialize'"], :level=\>:warn} This started happening when I added "delete" in my configuration. Please let me know the best way to delete a record from ES using logstash config. Thanks. ---
**Author:** ![Radim\_Novotny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/radim_novotny/32/943_2.png) [@Radim\_Novotny](https://discuss.elastic.co/u/Radim_Novotny)\ **Post date:** [August 12, 2015, 6:02pm UTC](https://discuss.elastic.co/t/logstash-to-delete-document-from-elasticsearch/27142/4 "2015-08-12T18:02:13Z")
Please read this - seems to be related [CouchDB plugin set dynamic type for Elasticsearch](https://discuss.elastic.co/t/couchdb-plugin-set-dynamic-type-for-elasticsearch/2293/2) ---
**Author:** ![umangjain](https://avatars.discourse-cdn.com/v4/letter/u/8e8cbc/32.png) [@umangjain](https://discuss.elastic.co/u/umangjain)\ **Post date:** [August 13, 2015, 3:04pm UTC](https://discuss.elastic.co/t/logstash-to-delete-document-from-elasticsearch/27142/5 "2015-08-13T15:04:01Z")
Thank you for your reply. I am using a similar configuration and it still throws this exception constantly until I kill logstash. Is this a bug in Logstash? Notice this: expected START\_OBJECT or END\_OBJECT But, I am not having any object and there is no option in Elasticsearch output to send source as null. Please help me here. Thanks, Umang ---
**Author:** ![Radim\_Novotny](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/radim_novotny/32/943_2.png) [@Radim\_Novotny](https://discuss.elastic.co/u/Radim_Novotny)\ **Post date:** [August 13, 2015, 3:20pm UTC](https://discuss.elastic.co/t/logstash-to-delete-document-from-elasticsearch/27142/6 "2015-08-13T15:20:03Z")
The problem is actually caused by logstash-output-elasticsearch plugin. In 'http' mode it does send object together with 'delete' command which ES does not expect. You will have to wait until [https://github.com/logstash-plugins/logstash-output-elasticsearch/issues/195](https://github.com/logstash-plugins/logstash-output-elasticsearch/issues/195) is resolved or patch the protocol.rb file yourself as written in [https://github.com/logstash-plugins/logstash-output-elasticsearch/issues/195#issuecomment-119745469](https://github.com/logstash-plugins/logstash-output-elasticsearch/issues/195#issuecomment-119745469) ---
**Author:** ![umangjain](https://avatars.discourse-cdn.com/v4/letter/u/8e8cbc/32.png) [@umangjain](https://discuss.elastic.co/u/umangjain)\ **Post date:** [August 19, 2015, 2:36pm UTC](https://discuss.elastic.co/t/logstash-to-delete-document-from-elasticsearch/27142/7 "2015-08-19T14:36:18Z")
Ok... great.. looks promising. I'll try it and will post here the results.. Thanks again. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 5:31am UTC](https://discuss.elastic.co/t/logstash-to-delete-document-from-elasticsearch/27142/8 "2017-07-06T05:31:33Z")