# How to keep only longest token occupying the same positions

**URL:** https://discuss.elastic.co/t/how-to-keep-only-longest-token-occupying-the-same-positions/325849
**Category:** Elasticsearch
**Created:** [February 17, 2023, 2:04pm UTC](https://discuss.elastic.co/t/how-to-keep-only-longest-token-occupying-the-same-positions/325849 "2023-02-17T14:04:09Z")
**Posts on this page:** 2
**Page:** 1

<div class="post-metadata">

### Author: ![Valentin\_Pletzer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/valentin_pletzer/32/105548_2.png) [@Valentin\_Pletzer](https://discuss.elastic.co/u/Valentin_Pletzer)
#### Post date: [February 17, 2023, 2:04pm UTC](https://discuss.elastic.co/t/how-to-keep-only-longest-token-occupying-the-same-positions/325849/1 "2023-02-17T14:04:09Z")

</div>

Is there a way too keep only the longest token if two or more tokens occupy the same positions?

e.g. if I define "fox" and "quick fox" as keep words obviously both would be return when analyzing the sentence "the quick fox jumped of the lazy dog." But since "quick fox" is more specific I would like to keep only "quick fox" and discard "fox".

```auto
curl -X PUT "localhost:9200/my-index-000001?pretty" -H 'Content-Type: application/json' -d'
{
  "settings": {
    "analysis": {
      "analyzer": {
        "std_folded": { 
          "type": "custom",
          "tokenizer": "standard",
          "filter": [
            "lowercase",
            "custom_shingle",
            "keepMe"
          ]
        }
      },
      "filter": {
        "keepMe": {
          "type": "keep",
          "keep_words": ["quick fox", "fox"]
        },
        "custom_shingle": {
            "type": "shingle",
            "min_shingle_size": "2",
            "max_shingle_size": "3",
            "output_unigrams_if_no_shingles": true
        }
      }
    }
  },
  "mappings": {
    "properties": {
      "my_text": {
        "type": "text",
        "analyzer": "std_folded" 
      }
    }
  }
}
'

```

```auto
curl -X POST "localhost:9200/my-index-000001/_analyze?pretty" -H 'Content-Type: application/json' -d'
{
  "analyzer": "std_folded",
  "text": "The quick fox jumped over the lazy dog."
}
'

```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 17, 2023, 2:05pm UTC](https://discuss.elastic.co/t/how-to-keep-only-longest-token-occupying-the-same-positions/325849/2 "2023-03-17T14:05:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.