Hello,
I have multi valued field which contains different values. How can I remove duplicate tokens from multi valued field? For example: I am applying unique analyzer for categories field. But it doesn't seem to remove duplicates.
POST _analyze
{
"tokenizer": "standard",
"filter": [ "lowercase", "unique" ],
"text": ["cheese", "cheese"]
}
{
"tokens": [
{
"token": "cheese",
"start_offset": 0,
"end_offset": 6,
"type": "",
"position": 0
},
{
"token": "cheese",
"start_offset": 7,
"end_offset": 13,
"type": "",
"position": 1
}
]
}