# Duplicate value in a fileds

**URL:** <https://discuss.elastic.co/t/duplicate-value-in-a-fileds/198936>\
**Category:** Elasticsearch\
**Created:** [September 10, 2019, 4:44pm UTC](https://discuss.elastic.co/t/duplicate-value-in-a-fileds/198936 "2019-09-10T16:44:29Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Reb](https://avatars.discourse-cdn.com/v4/letter/r/51bf81/32.png) [@Reb](https://discuss.elastic.co/u/Reb)\
**Post date:** [September 10, 2019, 4:44pm UTC](https://discuss.elastic.co/t/duplicate-value-in-a-fileds/198936/1 "2019-09-10T16:44:29Z")

</div>

due to a bad log treatment, the value of some fields is duplicated like :

fields\_duration : 373, 373.

How can i use query (update\_by\_query) to keep just the first value ?

Thank you

---

<div class="post-metadata">

**Author:** ![abdon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abdon/32/9195_2.png) [@abdon](https://discuss.elastic.co/u/abdon)\
**Post date:** [September 11, 2019, 2:08pm UTC](https://discuss.elastic.co/t/duplicate-value-in-a-fileds/198936/2 "2019-09-11T14:08:54Z")

</div>

Something like the following should work:

```auto
POST my_index/_update_by_query
{
  "script": {
    "source": "if (ctx._source['fields_duration'].size() > 1) { ctx._source['fields_duration'] = ctx._source['fields_duration'][0] }"
  }
}

```

---

<div class="post-metadata">

**Author:** ![Reb](https://avatars.discourse-cdn.com/v4/letter/r/51bf81/32.png) [@Reb](https://discuss.elastic.co/u/Reb)\
**Post date:** [September 12, 2019, 4:15pm UTC](https://discuss.elastic.co/t/duplicate-value-in-a-fileds/198936/3 "2019-09-12T16:15:14Z")

</div>

> {  
> "error": {  
> "root\_cause": [  
> {  
> "type": "script\_exception",  
> "reason": "runtime error",  
> "script\_stack": [  
> "if (ctx.\_source['fields\_duration'].size() \> 1) { ",  
> " ^---- HERE"  
> ],  
> "script": "if (ctx.\_source['fields\_duration'].size() \> 1) { ctx.\_source['fields\_duration'] = ctx.\_source['fields\_duration'][0] }",  
> "lang": "painless"  
> }  
> ],  
> "type": "script\_exception",  
> "reason": "runtime error",  
> "script\_stack": [  
> "if (ctx.\_source['fields\_duration'].size() \> 1) { ",  
> " ^---- HERE"  
> ],  
> "script": "if (ctx.\_source['fields\_duration'].size() \> 1) { ctx.\_source['fields\_duration'] = ctx.\_source['fields\_duration'][0] }",  
> "lang": "painless",  
> "caused\_by": {  
> "type": "illegal\_argument\_exception",  
> "reason": "dynamic method [java.lang.String, size/0] not found"  
> }  
> },  
> "status": 500  
> }

I'm sorry but i think that their is a problem whith size function, but i didn't find how to use it correctly.

Best regards

---

<div class="post-metadata">

**Author:** ![abdon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abdon/32/9195_2.png) [@abdon](https://discuss.elastic.co/u/abdon)\
**Post date:** [September 12, 2019, 7:00pm UTC](https://discuss.elastic.co/t/duplicate-value-in-a-fileds/198936/5 "2019-09-12T19:00:50Z")

</div>

What version of Elasticsearch are you using?

---

<div class="post-metadata">

**Author:** ![Reb](https://avatars.discourse-cdn.com/v4/letter/r/51bf81/32.png) [@Reb](https://discuss.elastic.co/u/Reb)\
**Post date:** [September 13, 2019, 8:03am UTC](https://discuss.elastic.co/t/duplicate-value-in-a-fileds/198936/6 "2019-09-13T08:03:06Z")

</div>

i use version 6.8.2

---

<div class="post-metadata">

**Author:** ![abdon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abdon/32/9195_2.png) [@abdon](https://discuss.elastic.co/u/abdon)\
**Post date:** [September 13, 2019, 8:15am UTC](https://discuss.elastic.co/t/duplicate-value-in-a-fileds/198936/7 "2019-09-13T08:15:47Z")

</div>

The script should check whether a field indeed contains multiple values. Try this:

```auto
POST my_index/_update_by_query
{
  "script": {
    "source": "if (ctx._source['fields_duration'] instanceof List && ctx._source['fields_duration'].size() > 1) { ctx._source['fields_duration'] = ctx._source['fields_duration'][0] }"
  }
}

```

---

<div class="post-metadata">

**Author:** ![Reb](https://avatars.discourse-cdn.com/v4/letter/r/51bf81/32.png) [@Reb](https://discuss.elastic.co/u/Reb)\
**Post date:** [September 16, 2019, 10:12am UTC](https://discuss.elastic.co/t/duplicate-value-in-a-fileds/198936/8 "2019-09-16T10:12:25Z")

</div>

hi when i try this script, i have :

{  
"statusCode": 504,  
"error": "Gateway Time-out",  
"message": "Client request timeout"  
}

my index is may be too big ...

---

<div class="post-metadata">

**Author:** ![abdon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abdon/32/9195_2.png) [@abdon](https://discuss.elastic.co/u/abdon)\
**Post date:** [September 16, 2019, 10:30am UTC](https://discuss.elastic.co/t/duplicate-value-in-a-fileds/198936/9 "2019-09-16T10:30:09Z")

</div>

This is expected with a large dataset. No worries, the Update by Query will continue to run in the background until all data has been processed.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 14, 2019, 10:30am UTC](https://discuss.elastic.co/t/duplicate-value-in-a-fileds/198936/10 "2019-10-14T10:30:12Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.