# Edgengram doesn't give tokens after spaces

**URL:** <https://discuss.elastic.co/t/edgengram-doesnt-give-tokens-after-spaces/131747>\
**Category:** Elasticsearch\
**Created:** [May 14, 2018, 1:17pm UTC](https://discuss.elastic.co/t/edgengram-doesnt-give-tokens-after-spaces/131747 "2018-05-14T13:17:44Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![satyarajp](https://avatars.discourse-cdn.com/v4/letter/s/90db22/32.png) [@satyarajp](https://discuss.elastic.co/u/satyarajp)\
**Post date:** [May 14, 2018, 1:17pm UTC](https://discuss.elastic.co/t/edgengram-doesnt-give-tokens-after-spaces/131747/1 "2018-05-14T13:17:44Z")

</div>

Hi Team,

I'm using edgenGram analyzer using settings like below

```
{
  "content" : {
    "settings" : {
      "index" : {
        "number_of_shards" : "5",
        "provided_name" : "content",
        "creation_date" : "1526295076710",
        "analysis" : {
          "analyzer" : {
            "autocomplete_search" : {
              "tokenizer" : "lowercase"
            },
            "edgeNgram_analyzer" : {
              "filter" : [
                "lowercase"
              ],
              "type" : "custom",
              "tokenizer" : "autocomplete_tokenizer"
            }
          },
          "tokenizer" : {
            "autocomplete_tokenizer" : {
              "type" : "edge_ngram",
              "min_gram" : "2",
              "max_gram" : "50"
            }
          }
        },
        "number_of_replicas" : "1",
        "uuid" : "pJ_A1EfOTFyfXFcoPRCemQ",
        "version" : {
          "created" : "5060099"
        }
      }
    }
  }
}

```

But when I tried to see tokens it is giving only starting first character. I need tokens from spaces too. Like in example from documentation.

```
localhost:9200/content/_analyze
{
"analyzer" : "edgeNgram_analyzer",
"text" : "Quick Brown Fox"
}

{"tokens":[{"token":"qu","start_offset":0,"end_offset":2,"type":"word","position":0},{"token":"qui","start_offset":0,"end_offset":3,"type":"word","position":1},{"token":"quic","start_offset":0,"end_offset":4,"type":"word","position":2},{"token":"quick","start_offset":0,"end_offset":5,"type":"word","position":3},{"token":"quick ","start_offset":0,"end_offset":6,"type":"word","position":4},{"token":"quick b","start_offset":0,"end_offset":7,"type":"word","position":5},{"token":"quick br","start_offset":0,"end_offset":8,"type":"word","position":6},{"token":"quick bro","start_offset":0,"end_offset":9,"type":"word","position":7},{"token":"quick brow","start_offset":0,"end_offset":10,"type":"word","position":8},{"token":"quick brown","start_offset":0,"end_offset":11,"type":"word","position":9},{"token":"quick brown ","start_offset":0,"end_offset":12,"type":"word","position":10},{"token":"quick brown f","start_offset":0,"end_offset":13,"type":"word","position":11},{"token":"quick brown fo","start_offset":0,"end_offset":14,"type":"word","position":12},{"token":"quick brown fox","start_offset":0,"end_offset":15,"type":"word","position":13}]}

```

But in example documentation, tokens should look similar to below

{qu,qui,quick,br,brown}

Please help

Regards,  
SatyaRaj

---

<div class="post-metadata">

**Author:** ![mayya](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mayya/32/83147_2.png) [@mayya](https://discuss.elastic.co/u/mayya)\
**Post date:** [May 18, 2018, 7:07pm UTC](https://discuss.elastic.co/t/edgengram-doesnt-give-tokens-after-spaces/131747/2 "2018-05-18T19:07:51Z")

</div>

As you can see in the [documentation example](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-edgengram-tokenizer.html), we used `token_chars` configuration parameter .

```
"token_chars": [
        "letter",
        "digit"
      ]

```

This means that text will be split on all non-letter, and non-digit symbols.  
If you want your text to be split on whitespaces you can also use this configuration option.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 15, 2018, 7:07pm UTC](https://discuss.elastic.co/t/edgengram-doesnt-give-tokens-after-spaces/131747/3 "2018-06-15T19:07:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.