Duplicate value in a fileds

Elastic Stack Elasticsearch
1

due to a bad log treatment, the value of some fields is duplicated like :

fields_duration : 373, 373.

How can i use query (update_by_query) to keep just the first value ?

Thank you

2

Something like the following should work:

POST my_index/_update_by_query
{
  "script": {
    "source": "if (ctx._source['fields_duration'].size() > 1) { ctx._source['fields_duration'] = ctx._source['fields_duration'][0] }"
  }
}
3

{
"error": {
"root_cause": [
{
"type": "script_exception",
"reason": "runtime error",
"script_stack": [
"if (ctx._source['fields_duration'].size() > 1) { ",
" ^---- HERE"
],
"script": "if (ctx._source['fields_duration'].size() > 1) { ctx._source['fields_duration'] = ctx._source['fields_duration'][0] }",
"lang": "painless"
}
],
"type": "script_exception",
"reason": "runtime error",
"script_stack": [
"if (ctx._source['fields_duration'].size() > 1) { ",
" ^---- HERE"
],
"script": "if (ctx._source['fields_duration'].size() > 1) { ctx._source['fields_duration'] = ctx._source['fields_duration'][0] }",
"lang": "painless",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "dynamic method [java.lang.String, size/0] not found"
}
},
"status": 500
}

I'm sorry but i think that their is a problem whith size function, but i didn't find how to use it correctly.

Best regards

5

What version of Elasticsearch are you using?

6

i use version 6.8.2

7

The script should check whether a field indeed contains multiple values. Try this:

POST my_index/_update_by_query
{
  "script": {
    "source": "if (ctx._source['fields_duration'] instanceof List && ctx._source['fields_duration'].size() > 1) { ctx._source['fields_duration'] = ctx._source['fields_duration'][0] }"
  }
}
8

hi when i try this script, i have :

{
"statusCode": 504,
"error": "Gateway Time-out",
"message": "Client request timeout"
}

my index is may be too big ...

9

This is expected with a large dataset. No worries, the Update by Query will continue to run in the background until all data has been processed.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.