# Remove duplicates from multi-valued field

**URL:** <https://discuss.elastic.co/t/remove-duplicates-from-multi-valued-field/166232>\
**Category:** Elasticsearch\
**Created:** [January 29, 2019, 8:43pm UTC](https://discuss.elastic.co/t/remove-duplicates-from-multi-valued-field/166232 "2019-01-29T20:43:25Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dipak](https://avatars.discourse-cdn.com/v4/letter/d/90ced4/32.png) [@Dipak](https://discuss.elastic.co/u/Dipak)\
**Post date:** [January 29, 2019, 8:43pm UTC](https://discuss.elastic.co/t/remove-duplicates-from-multi-valued-field/166232/1 "2019-01-29T20:43:26Z")

</div>

Hello,

I have multi valued field which contains different values. How can I remove duplicate tokens from multi valued field? For example: I am applying unique analyzer for categories field. But it doesn't seem to remove duplicates.

POST \_analyze  
{  
"tokenizer": "standard",  
"filter": ["lowercase", "unique"],  
"text": ["cheese", "cheese"]  
}

{  
"tokens": [  
{  
"token": "cheese",  
"start\_offset": 0,  
"end\_offset": 6,  
"type": "",  
"position": 0  
},  
{  
"token": "cheese",  
"start\_offset": 7,  
"end\_offset": 13,  
"type": "",  
"position": 1  
}  
]  
}

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 26, 2019, 8:53pm UTC](https://discuss.elastic.co/t/remove-duplicates-from-multi-valued-field/166232/2 "2019-02-26T20:53:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.