Duplicate value in a fileds

Elastic Stack Elasticsearch
1

due to a bad log treatment, the value of some fields is duplicated like :

fields_duration : 373, 373.

How can i use query (update_by_query) to keep just the first value ?

Thank you

2

Something like the following should work:

POST my_index/_update_by_query
{
  "script": {
    "source": "if (ctx._source['fields_duration'].size() > 1) { ctx._source['fields_duration'] = ctx._source['fields_duration'][0] }"
  }
}
3

{
"error": {
"root_cause": [
{
"type": "script_exception",
"reason": "runtime error",
"script_stack": [
"if (ctx._source['fields_duration'].size() > 1) { ",
" ^---- HERE"
],
"script": "if (ctx._source['fields_duration'].size() > 1) { ctx._source['fields_duration'] = ctx._source['fields_duration'][0] }",
"lang": "painless"
}
],
"type": "script_exception",
"reason": "runtime error",
"script_stack": [
"if (ctx._source['fields_duration'].size() > 1) { ",
" ^---- HERE"
],
"script": "if (ctx._source['fields_duration'].size() > 1) { ctx._source['fields_duration'] = ctx._source['fields_duration'][0] }",
"lang": "painless",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "dynamic method [java.lang.String, size/0] not found"
}
},
"status": 500
}

I'm sorry but i think that their is a problem whith size function, but i didn't find how to use it correctly.

Best regards

5

What version of Elasticsearch are you using?

6

i use version 6.8.2

7

The script should check whether a field indeed contains multiple values. Try this:

POST my_index/_update_by_query
{
  "script": {
    "source": "if (ctx._source['fields_duration'] instanceof List && ctx._source['fields_duration'].size() > 1) { ctx._source['fields_duration'] = ctx._source['fields_duration'][0] }"
  }
}
8

hi when i try this script, i have :

{
"statusCode": 504,
"error": "Gateway Time-out",
"message": "Client request timeout"
}

my index is may be too big ...

9

This is expected with a large dataset. No worries, the Update by Query will continue to run in the background until all data has been processed.