Delete by query creating a new document in the index (ES 5.4)

Elastic Stack Elasticsearch
1

I am testing out ElasticSearch 5.4 before upgrading our existing code base and getting strange behaviour when running a delete by query on a couple of indices - in every case it looks like a new document with id '_delete_by_query' is being created:

09:30:13.822 [Test worker] DEBUG org.apache.http.headers - http-outgoing-0 >> POST /user-complaints-all/user-complaint/_delete_by_query HTTP/1.1
09:30:13.822 [Test worker] DEBUG org.apache.http.headers - http-outgoing-0 >> Content-Length: 123
09:30:13.822 [Test worker] DEBUG org.apache.http.headers - http-outgoing-0 >> Content-Type: application/json; charset=UTF-8
09:30:13.822 [Test worker] DEBUG org.apache.http.headers - http-outgoing-0 >> Host: localhost:9605
09:30:13.822 [Test worker] DEBUG org.apache.http.headers - http-outgoing-0 >> Connection: Keep-Alive
09:30:13.822 [Test worker] DEBUG org.apache.http.headers - http-outgoing-0 >> User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_91)
09:30:13.822 [Test worker] DEBUG org.apache.http.headers - http-outgoing-0 >> Accept-Encoding: gzip,deflate
09:30:13.823 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 >> "POST /user-complaints-all/user-complaint/_delete_by_query HTTP/1.1[\r][\n]"
09:30:13.823 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 >> "Content-Length: 123[\r][\n]"
09:30:13.823 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 >> "Content-Type: application/json; charset=UTF-8[\r][\n]"
09:30:13.823 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 >> "Host: localhost:9605[\r][\n]"
09:30:13.823 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 >> "Connection: Keep-Alive[\r][\n]"
09:30:13.823 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 >> "User-Agent: Apache-HttpClient/4.5.3 (Java/1.8.0_91)[\r][\n]"
09:30:13.823 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 >> "Accept-Encoding: gzip,deflate[\r][\n]"
09:30:13.823 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 >> "[\r][\n]"
09:30:13.823 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 >> "{[\n]"
09:30:13.823 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 >> "  "query" : {[\n]"
09:30:13.823 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 >> "    "term" : {[\n]"
09:30:13.823 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 >> "      "Date" : {[\n]"
09:30:13.823 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 >> "        "value" : "2017-02-27",[\n]"
09:30:13.823 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 >> "        "boost" : 1.0[\n]"
09:30:13.823 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 >> "      }[\n]"
09:30:13.824 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 >> "    }[\n]"
09:30:13.824 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 >> "  }[\n]"
09:30:13.824 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 >> "}"
09:30:13.954 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 << "HTTP/1.1 201 Created[\r][\n]"
09:30:13.955 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 << "Location: /user-complaints-all/user-complaint/_delete_by_query[\r][\n]"
09:30:13.955 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 << "Content-Type: application/json; charset=UTF-8[\r][\n]"
09:30:13.955 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 << "Content-Encoding: gzip[\r][\n]"
09:30:13.955 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 << "Content-Length: 147[\r][\n]"
09:30:13.955 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 << "[\r][\n]"
09:30:13.955 [Test worker] DEBUG org.apache.http.wire - http-outgoing-0 << "[0x1f][0x8b][0x8][0x0][0x0][0x0][0x0][0x0][0x0][0x0]d[0x8b]A[0xe][0x2]![0x10][0x4][0xff]2g6Q[0x8f]|[0x86] [0xf4]F[0x92][0x11][0xd6][0x99][0xc1][0xb8][0xd9][0xec][0xdf][0x85][0x8b][0x17][0x8f][0xdd]UuP(5[0xe3]C[0x9e][0xba]B[0x96][0xd4][0x9e][0x1b][0xc7]RM[0x97][0xc8]L[0x8e][0x82][0xed][0x1b][0xfe][0xe8][0x4]%[0x8f];d0[0xc][0xe1][0xbe][0x87]W[0x87][0xec][0x13][0xbc]!ZZ%u$[0xd0][0xce]6[0xc4]$[0x88][0x86]<[0xb9]>[0xa2]d%[0x90]5[0x8b]L[0xfe][0xe6]H{JP][0xfb][0x98]#[c[0xe1]![0xfb][0xcb][0xe9]~[0xa5]7[0xe9]8[0xbf][0x0][0x0][0x0][0xff][0xff][0x3][0x0][0x82][0x92][0x99][0xb4][0xb1][0x0][0x0][0x0]"
09:30:13.956 [Test worker] DEBUG org.apache.http.headers - http-outgoing-0 << HTTP/1.1 201 Created
09:30:13.956 [Test worker] DEBUG org.apache.http.headers - http-outgoing-0 << Location: /user-complaints-all/user-complaint/_delete_by_query
09:30:13.956 [Test worker] DEBUG org.apache.http.headers - http-outgoing-0 << Content-Type: application/json; charset=UTF-8
09:30:13.956 [Test worker] DEBUG org.apache.http.headers - http-outgoing-0 << Content-Encoding: gzip
09:30:13.956 [Test worker] DEBUG org.apache.http.headers - http-outgoing-0 << Content-Length: 147

I'm using the latest JEST client to interact with ElasticSearch and the HTTP request looks correct to me but instead of deleting the target records, I get a new record in the index.

Any help would be greatly appreciated.

Thanks!

2

Providing the request you are creating would be helpful.

Also take a look at the java rest client - https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/index.html

3

I realised that the issue was that I was not loading the reindex plugin (I am using a manually created Node to do some testing against which I know is no longer supported). Creating the node with the plugin solved the problem.

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.