# Cannot get the Length Token Filter to work in a custom anaylzer

**URL:** https://discuss.elastic.co/t/cannot-get-the-length-token-filter-to-work-in-a-custom-anaylzer/212288
**Category:** Elasticsearch
**Created:** [December 18, 2019, 9:10am UTC](https://discuss.elastic.co/t/cannot-get-the-length-token-filter-to-work-in-a-custom-anaylzer/212288 "2019-12-18T09:10:47Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![Anthony\_Chiu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anthony_chiu/32/59618_2.png) [@Anthony\_Chiu](https://discuss.elastic.co/u/Anthony_Chiu)
#### Post date: [December 18, 2019, 9:10am UTC](https://discuss.elastic.co/t/cannot-get-the-length-token-filter-to-work-in-a-custom-anaylzer/212288/1 "2019-12-18T09:10:47Z")

</div>

I am setting up a filter within my index:

```
  "min-length-token":{
                "type": "length",
                "min": 2,
   },

```

then, I add this filter to my anayzer like this:

```
"extend-standard" : {
              "filter" : [
                "lowercase", "min-length-token"
              ],
              "tokenizer" : "standard"
            },

```

But when I call the analyzer api with

```
{
  "analyzer": "extend-standard",
  "text": "a&b"
}

```

It still returns "a" and "b", which have length 1 only.

---

<div class="post-metadata">

### Author: ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)
#### Post date: [December 18, 2019, 12:24pm UTC](https://discuss.elastic.co/t/cannot-get-the-length-token-filter-to-work-in-a-custom-anaylzer/212288/2 "2019-12-18T12:24:44Z")

</div>

can you share a fully reproducible example? I tried this in the analyze API and it works.

```auto
GET _analyze
{
  "tokenizer": "standard",
  "filter": [
    {
      "type": "length",
      "min": 2
    }
  ],
  "text": "a&b"
}

```

also, please share your Elasticsearch version.

Thanks!

---

<div class="post-metadata">

### Author: ![Anthony\_Chiu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anthony_chiu/32/59618_2.png) [@Anthony\_Chiu](https://discuss.elastic.co/u/Anthony_Chiu)
#### Post date: [December 19, 2019, 1:27am UTC](https://discuss.elastic.co/t/cannot-get-the-length-token-filter-to-work-in-a-custom-anaylzer/212288/3 "2019-12-19T01:27:18Z")

</div>

Thanks for your reply, here is a full sample:

```
POST t_index/_close

PUT t_index/_settings
{
  "analysis":{
    "filter":{
      "min-length-token":{
        "type": "length",
        "min": 2,
        "max": 512
      }
    },
    "analyzer":{
      "t-standard":{
        "filter":["min-length-token"],
        "type":"standard"
      }
    }
  }
}

POST t_index/_open

GET t_index/_analyze
{
  "analyzer": "t-standard",
  "text": "a&b"
}

```

The response:

```
{
  "tokens" : [
    {
      "token" : "a",
      "start_offset" : 0,
      "end_offset" : 1,
      "type" : "<ALPHANUM>",
      "position" : 0
    },
    {
      "token" : "b",
      "start_offset" : 2,
      "end_offset" : 3,
      "type" : "<ALPHANUM>",
      "position" : 1
    }
  ]
}
```

---

<div class="post-metadata">

### Author: ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)
#### Post date: [December 19, 2019, 9:56am UTC](https://discuss.elastic.co/t/cannot-get-the-length-token-filter-to-work-in-a-custom-anaylzer/212288/4 "2019-12-19T09:56:10Z")

</div>

try this

```auto
PUT t_index
{
  "settings": {
    "analysis": {
      "filter": {
        "min-length-token": {
          "type": "length",
          "min": 2,
          "max": 512
        }
      },
      "analyzer": {
        "t-standard": {
          "filter": [
            "min-length-token"
          ],
          "tokenizer" : "standard",
          "type": "custom"
        }
      }
    }
  }
}

GET t_index/_analyze
{
  "analyzer": "t-standard",
  "text": "a&b"
}

```

using `standard` as the analyzer seems to ignore all the other settings. I will take a deeper look and potentially open an issue.

Thanks!

---

<div class="post-metadata">

### Author: ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)
#### Post date: [December 19, 2019, 10:13am UTC](https://discuss.elastic.co/t/cannot-get-the-length-token-filter-to-work-in-a-custom-anaylzer/212288/5 "2019-12-19T10:13:47Z")

</div>

I opened [https://github.com/elastic/elasticsearch/issues/50356](https://github.com/elastic/elasticsearch/issues/50356) for further discussion if you are interested.

---

<div class="post-metadata">

### Author: ![Anthony\_Chiu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/anthony_chiu/32/59618_2.png) [@Anthony\_Chiu](https://discuss.elastic.co/u/Anthony_Chiu)
#### Post date: [December 19, 2019, 10:32am UTC](https://discuss.elastic.co/t/cannot-get-the-length-token-filter-to-work-in-a-custom-anaylzer/212288/6 "2019-12-19T10:32:10Z")

</div>

Thanks for the work around, I also agree it is an issue. I got it fixed with standard tokeniser, but I can't get the english work to work.

```
"t-standard":{
        "filter":["min-length-token"],
        "type":"english",
        "tokenizer": "standard"
      }
```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 16, 2020, 10:32am UTC](https://discuss.elastic.co/t/cannot-get-the-length-token-filter-to-work-in-a-custom-anaylzer/212288/7 "2020-01-16T10:32:11Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.