# Delete by query with plugin not work for me **URL:** \ **Category:** Elasticsearch\ **Created:** [April 7, 2016, 7:15pm UTC](https://discuss.elastic.co/t/delete-by-query-with-plugin-not-work-for-me/46732 "2016-04-07T19:15:34Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![Juan\_Andres\_Ramirez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juan_andres_ramirez/32/9467_2.png) [@Juan\_Andres\_Ramirez](https://discuss.elastic.co/u/Juan_Andres_Ramirez)\ **Post date:** [April 7, 2016, 7:15pm UTC](https://discuss.elastic.co/t/delete-by-query-with-plugin-not-work-for-me/46732/1 "2016-04-07T19:15:34Z")
Hello Guys, I have the last ELK version, and I installed the plugin delete-by-query and my forwarder is Logstash. I had read the documentation 2.3 in ELK site. **First** TO check if my struct is the same as example Twitter, I found one server by ID: `curl -XGET 'http://localhost:9200/logstash-2016.02.23/Nxlogs/AVMPTRrp28MHpUptVm7C?pretty'` This respond (short version): ``` { "_index" : "logstash-2016.02.23", "_type" : "Nxlogs", "_id" : "AVMPTRrp28MHpUptVm7C", "_version" : 1, "found" : true, "_source" : { "EventReceivedTime" : "2016-02-23 13:03:42", "SourceModuleName" : "iis", "SourceModuleType" : "im_file", "date" : "2016-02-23", "time" : "17:30:49", "hostname" : "server1.xxx.com" } ``` **Second:** Now, when I tried delete by hostname: `curl -XDELETE 'http://localhost:9200/logstash-2016.02.23/Nxlogs/_query?q=hostname:server1.xxx.com` Respond: `{"found":false,"_index":"logstash-2016.02.23","_type":"Nxlogs","_id":"_query","_version":1,"_shards":{"total":2,"successful":2,"failed":0}}` **Found : false** I tried with other some ways with the same result: `curl -XDELETE 'http://localhost:9200/logstash-2016.02.23/Nxlogs/_query' -d '{ "term" : { "hostname" : "server1.xxx.com" }}` `curl -XDELETE 'http://localhost:9200/logstash-2016.02.23/Nxlogs/_query' -d '{ "query": { "term" : { "hostname" : "server1.xxx.com" } }}'` `curl -XDELETE 'http://localhost:9200/logstash-2016.02.23/Nxlogs/_query?pretty' -d '{ "query": { "query_string" : { "default_field": "hostname","query": "server1.xxx.com" }}}}'` Someone has ideas?. Thank you. ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [April 12, 2016, 9:40am UTC](https://discuss.elastic.co/t/delete-by-query-with-plugin-not-work-for-me/46732/2 "2016-04-12T09:40:46Z")
Hi Juan, that's interesting. I'd first look at the mapping for this index (for starters: I'd expect "hostname" to be a not\_analyzed field). I don't think this is a problem with the delete by query plugin. I'd guess that you also cannot find the document with a plain term query. Daniel ---
**Author:** ![Juan\_Andres\_Ramirez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juan_andres_ramirez/32/9467_2.png) [@Juan\_Andres\_Ramirez](https://discuss.elastic.co/u/Juan_Andres_Ramirez)\ **Post date:** [April 12, 2016, 7:48pm UTC](https://discuss.elastic.co/t/delete-by-query-with-plugin-not-work-for-me/46732/3 "2016-04-12T19:48:14Z")
Hello Daniel Thank you for you answer. I looked the index config in Kibana and the field is analized ![](https://us1.discourse-cdn.com/elastic/original/2X/b/ba110d20ec56da8e5eb82f1dd2a04abaf8c42a38.jpg) Can I check my index with other tool?. Or any other plugin to check this?. Thank you. ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [April 13, 2016, 6:46am UTC](https://discuss.elastic.co/t/delete-by-query-with-plugin-not-work-for-me/46732/4 "2016-04-13T06:46:55Z")
Hi Juan, sure, the easiest way is to use the [Mapping API](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-get-mapping.html) of Elasticsearch. This should return the mapping for your type in this specific index: ```auto curl http://localhost:9200/logstash-2016.02.23/_mapping/Nxlogs?pretty ``` I understand why the term query does not return a result as it only really works as intended for `not_analyzed` fields. However, the query string query should match. Can you try just to issue the query? ```auto curl 'http://localhost:9200/logstash-2016.02.23/Nxlogs/_search?pretty' -d '{ "query": { "query_string": { "default_field": "hostname", "query": "server1.xxx.com" } } }' ``` This should actually return some results. ### Changing the mapping I suggest also you change your mapping so all string fields that you want store exactly as they are `not_analyzed` (see [mapping docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/string.html)). As you use logstash, you have to change your mapping template and the mapping of all existing indices. To apply the mapping to all existing indices use the [multi-index syntax](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-put-mapping.html#_multi_index) so you just have issue a single request and not one for each index. After all mappings have been corrected you need to reindex your existing data (as the mapping affects only newly indexed data). As you are on Elasticsearch 2.3 you can use the [reindex API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html) for that. As all these operations write to your index I suggest you try this first on a test / playground system and if you want to be sure also create a backup of your data. Finally, you should be able to find the server in question using a term query: ```auto curl 'http://localhost:9200/logstash-2016.02.23/Nxlogs/_search' -d '{ "query": { "term": { "hostname": { "value": "server1.xxx.com" } } } }' ``` If you find the server using this query, you should also be able to delete the data with the delete by query plugin. Daniel ---
**Author:** ![Juan\_Andres\_Ramirez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juan_andres_ramirez/32/9467_2.png) [@Juan\_Andres\_Ramirez](https://discuss.elastic.co/u/Juan_Andres_Ramirez)\ **Post date:** [April 13, 2016, 7:21pm UTC](https://discuss.elastic.co/t/delete-by-query-with-plugin-not-work-for-me/46732/5 "2016-04-13T19:21:39Z")
Hello Daniel, Thank you for answer me again. The first curl command returned the following (only show you the hostname): ``` "hostname" : { "type" : "string", "norms" : { "enabled" : false }, "fielddata" : { "format" : "disabled" }, "fields" : { "raw" : { "type" : "string", "index" : "not_analyzed", "ignore_above" : 256 } } }, ``` The second curl command search, return results about the host, for example: ``` "_index" : "logstash-2016.02.23", "_type" : "Nxlogs", "_id" : "AVNEFBk3zTqQ6j3gBtmd", "_score" : 3.0796967, "_source" : { "EventReceivedTime" : "2016-02-23 19:01:21", "SourceModuleName" : "iis", "SourceModuleType" : "im_file", "date" : "2016-02-23", "time" : "00:01:19", "EventTime" : "2016-02-23T00:01:19Z", "SourceName" : "IIS", "hostname" : "server1.xxx.com", "type" : "Nxlogs", "tags" : ["windows", "logs"], } } ] ``` I'm going to change the field index with not analyzed by analized, but I have the same problem in the new index. I need fix it forever, how can I change this field forever? in logstash? or by my client NXlog?. Thank you. ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [April 14, 2016, 4:13am UTC](https://discuss.elastic.co/t/delete-by-query-with-plugin-not-work-for-me/46732/6 "2016-04-14T04:13:25Z")
Hi Juan, that mapping is good news. The "hostname" field has a "raw" subfield that is not analyzed. This means you don't have to change anything in your mapping. You should be able to delete the relevant entries with the following statement: ```auto curl -XDELETE 'http://localhost:9200/logstash-2016.02.23/Nxlogs/_query?pretty' -d '{ "query": { "term": { "hostname.raw": { "value": "server1.xxx.com" } } } }' ``` I've create a minimal example that you can try out in Sense and that works for me. ```auto PUT /logs { "mappings": { "nxlog": { "properties": { "hostname": { "type": "string", "fields": { "raw": { "type": "string", "index": "not_analyzed", "ignore_above": 256 } } } } } } } POST /logs/nxlog/1 { "hostname": "server1.xxx.com" } POST /logs/nxlog/2 { "hostname": "server2.xxx.com" } DELETE /logs/nxlog/_query { "query": { "term": { "hostname.raw": { "value": "server1.xxx.com" } } } } ``` The last request produces: ```auto { "took": 0, "timed_out": false, "_indices": { "_all": { "found": 1, "deleted": 1, "missing": 0, "failed": 0 }, "logs": { "found": 1, "deleted": 1, "missing": 0, "failed": 0 } }, "failures": [] } ``` And indeed the data for [server1.xxx.com](http://server1.xxx.com) are gone. If we run a match\_all query, we get: ```auto { "took": 2, "timed_out": false, "_shards": { "total": 5, "successful": 5, "failed": 0 }, "hits": { "total": 1, "max_score": 1, "hits": [ { "_index": "logs", "_type": "nxlog", "_id": "2", "_score": 1, "_source": { "hostname": "server2.xxx.com" } } ] } } ``` Daniel ---
**Author:** ![Juan\_Andres\_Ramirez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juan_andres_ramirez/32/9467_2.png) [@Juan\_Andres\_Ramirez](https://discuss.elastic.co/u/Juan_Andres_Ramirez)\ **Post date:** [April 18, 2016, 12:43pm UTC](https://discuss.elastic.co/t/delete-by-query-with-plugin-not-work-for-me/46732/7 "2016-04-18T12:43:26Z")
Hello Daniel, It didn't work. I created a new index with your example: The mapping for the new index: ``` { "logs": { "mappings": { "nxlog": { "properties": { "hostname": { "type": "string", "fields": { "raw": { "type": "string", "index": "not_analyzed", "ignore_above": 256 } } } } } } } } ``` I found 1 server: ``` GET logs/_validate/query { "query": { "term": { "hostname.raw": { "value": "server1.xxx.com" } } } } ``` The answer: ``` { "valid": true, "_shards": { "total": 1, "successful": 1, "failed": 0 } } ``` I tried delete the server: ``` DELETE /logs/nxlog/_query { "query": { "term": { "hostname.raw": { "value": "server1.xxx.com" } } } } ``` and the answer: ``` { "found": false, "_index": "logs", "_type": "nxlog", "_id": "_query", "_version": 1, "_shards": { "total": 2, "successful": 2, "failed": 0 } } ``` So I dont know what is the problem. Maybe some issues in the config?. Thank you. ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [April 19, 2016, 2:13pm UTC](https://discuss.elastic.co/t/delete-by-query-with-plugin-not-work-for-me/46732/8 "2016-04-19T14:13:02Z")
Hi Juan, can you please verify that you have the `delete-by-query` plugin installed? You can run: ```auto GET /_cluster/stats ``` in Sense. Right at the bottom of the response you should see the installed plugins. If the `delete-by-query` plugin is not listed, it is not installed. Then you have to install it first via: ```auto sudo bin/plugin install delete-by-query ``` Don't forget that you need to install the plugin on every node in the cluster and restart the nodes as the new plugin is only picked up after a restart. Daniel ---
**Author:** ![Juan\_Andres\_Ramirez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/juan_andres_ramirez/32/9467_2.png) [@Juan\_Andres\_Ramirez](https://discuss.elastic.co/u/Juan_Andres_Ramirez)\ **Post date:** [April 19, 2016, 3:23pm UTC](https://discuss.elastic.co/t/delete-by-query-with-plugin-not-work-for-me/46732/9 "2016-04-19T15:23:58Z")
Hello Daniel Yes I have installed as well the plugin, I tried installed again: `ERROR: plugin directory /usr/share/elasticsearch/plugins/delete-by-query already exists. To update the plugin, uninstall it first using 'remove delete-by-query' command` I updated the ELK version: `Exception in thread "main" java.lang.IllegalArgumentException: Plugin [delete-by-query] is incompatible with Elasticsearch [2.3.1]. Was designed for version [2.2.0]` So I installed the new version 2.3.1. I uninstalled the plugin and I installed again, now everyting works as well. I didn't know what happend but the plugin working as well, now I can delete by hostname.raw. Thank you very much for you patience. ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [April 19, 2016, 3:51pm UTC](https://discuss.elastic.co/t/delete-by-query-with-plugin-not-work-for-me/46732/10 "2016-04-19T15:51:01Z")
Hi Juan, You're right: The plugin version always has to match your Elasticsearch version. But I'm glad I could help and great that it finally worked out for you! 🙂 Daniel ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 10:57pm UTC](https://discuss.elastic.co/t/delete-by-query-with-plugin-not-work-for-me/46732/11 "2017-07-05T22:57:59Z")