# Search query does'nt find the token

**URL:** <https://discuss.elastic.co/t/search-query-doesnt-find-the-token/208543>\
**Category:** Elasticsearch\
**Created:** [November 19, 2019, 4:07pm UTC](https://discuss.elastic.co/t/search-query-doesnt-find-the-token/208543 "2019-11-19T16:07:49Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Giannis](https://avatars.discourse-cdn.com/v4/letter/g/aca169/32.png) [@Giannis](https://discuss.elastic.co/u/Giannis)\
**Post date:** [November 19, 2019, 4:07pm UTC](https://discuss.elastic.co/t/search-query-doesnt-find-the-token/208543/1 "2019-11-19T16:07:49Z")

</div>

I have the sentence " **Is with any job to take courses**" in title field. If i query with token " **cours**" doesnt match anything. I must query with token "courses" and then is ok! I use the english analyser [https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-lang-analyzer.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-lang-analyzer.html)

Take a look.  
GET enarticles/\_analyze  
{  
"analyzer" : "englishtoken",  
"text" : "courses"  
}  
Output:  
{  
"tokens" : [  
{  
"token" : "cours",  
"start\_offset" : 0,  
"end\_offset" : 7,  
"type" : "",  
"position" : 0  
}  
]  
}

GET enarticles/\_search  
{  
"query": {  
"match" : {  
"title" : {  
"query" : "cours"  
}  
}  
}  
}  
Output:  
{  
"took" : 0,  
"timed\_out" : false,  
"\_shards" : {  
"total" : 3,  
"successful" : 3,  
"skipped" : 0,  
"failed" : 0  
},  
"hits" : {  
"total" : {  
"value" : 0,  
"relation" : "eq"  
},  
"max\_score" : null,  
"hits" :   
}  
}  
Nothing is matched!

I will use the token analyzer output for searching the database if a token exists. This must be 100% working. Any ideas?

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [November 21, 2019, 12:29pm UTC](https://discuss.elastic.co/t/search-query-doesnt-find-the-token/208543/2 "2019-11-21T12:29:00Z")

</div>

please share your full setup, as it's not clear here what the `englishtoken` analyzer consists off in terms of tokenizer and token filters. This will prevent guessing on our side 🙂

---

<div class="post-metadata">

**Author:** ![Giannis](https://avatars.discourse-cdn.com/v4/letter/g/aca169/32.png) [@Giannis](https://discuss.elastic.co/u/Giannis)\
**Post date:** [November 21, 2019, 3:07pm UTC](https://discuss.elastic.co/t/search-query-doesnt-find-the-token/208543/3 "2019-11-21T15:07:09Z")

</div>

This is the full analyzer for enarticles!  
$params = [  
'index' =\> 'enarticles',  
'body' =\> [  
'settings' =\> [  
'number\_of\_shards' =\> 3,  
'number\_of\_replicas' =\> 2,

```
        "analysis"=>[
            "filter"=> [
                "english_stop"=> [
                    "type"=> "stop",
                    "stopwords"=> "_english_"
                ],
                 
                
                "english_keywords"=> [
                    "type"=> "keyword_marker",
                    "keywords"=> ["example"]
                ],
                "english_stemmer"=> [
                    "type"=> "stemmer",
                    "language"=> "english"
                ],
                "english_possessive_stemmer"=> [ 
                "type"=> "stemmer",
                "language"=> "possessive_english" ],
                
                 
                
                
                
                
            ],
            "analyzer"=> [
                "englishtoken"=> [
                    "tokenizer"=> "standard",
                    "filter"=>[
                         
                        "english_stop",
                        "english_keywords",
                        "english_stemmer",
                        "english_possessive_stemmer"
                    ] 
                 ] 
                    
                
            ]
        ]
        
    ],
    'mappings' => [
         
            'properties' => [
                'etext' => [
                    'type' => 'text',
                    'analyzer' => 'englishtoken'
                ],
                'gtext' => [
                    'type' => 'text',
                    'analyzer' => 'englishtoken'
                ],
                'html' => [
                    'type' => 'text',
                    'analyzer' => 'standard'
                ],
                
                'title' => [
                    'type' => 'text',
                    'analyzer' => 'englishtoken'
                ],
                'keywords' => [
                    'type' => 'text',
                    'analyzer' => 'standard'
                ],
                'url' => [
                    'type' => 'text',
                    'analyzer' => 'standard'
                ],
                "date"=> [
                    "type"=> "date"
                ],
                'category' => [
                    'type' => 'text',
                    'analyzer' => 'englishtoken'
                ],
                "author"=> [
                    'type' => 'text',
                    'analyzer' => 'englishtoken'
                ] ,
                "sentiment"=> [
                    'type' => 'float',
                ] ,
                "ranking"=> [
                    'type' => 'float',
                ]
            ]
        ]
    ]

```

];

---

<div class="post-metadata">

**Author:** ![Glen\_Smith](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/glen_smith/32/111656_2.png) [@Glen\_Smith](https://discuss.elastic.co/u/Glen_Smith)\
**Post date:** [November 21, 2019, 8:39pm UTC](https://discuss.elastic.co/t/search-query-doesnt-find-the-token/208543/4 "2019-11-21T20:39:12Z")

</div>

Just quickly chiming in that what you're running into is the challenge of how stemming treats plurals. The term `courses`, as you've demonstrated, emits the token `cours`.

But then, the token that's emitted for the term `cours` is actually `cour`, as the stemmer sees the term as a plural rather than as a the stem of `course`.

There's a [github issue open with great discussion](https://github.com/elastic/elasticsearch/issues/42892) and analysis of the topic.

I'm not certain what the prescribed approach is to work around this; search fuzzification occurs to me, but applied systematically to your queries, that could increase the number of false positive results you retrieve.

---

<div class="post-metadata">

**Author:** ![Giannis](https://avatars.discourse-cdn.com/v4/letter/g/aca169/32.png) [@Giannis](https://discuss.elastic.co/u/Giannis)\
**Post date:** [November 23, 2019, 9:54pm UTC](https://discuss.elastic.co/t/search-query-doesnt-find-the-token/208543/5 "2019-11-23T21:54:03Z")

</div>

Solved. Thank you so much!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 21, 2019, 9:54pm UTC](https://discuss.elastic.co/t/search-query-doesnt-find-the-token/208543/6 "2019-12-21T21:54:20Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.