# Possible bug with 6.3 -- Analyzer not working correctly

**URL:** <https://discuss.elastic.co/t/possible-bug-with-6-3-analyzer-not-working-correctly/137225>\
**Category:** Elasticsearch\
**Created:** [June 25, 2018, 9:46am UTC](https://discuss.elastic.co/t/possible-bug-with-6-3-analyzer-not-working-correctly/137225 "2018-06-25T09:46:38Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jason\_Baumgartner](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jason_baumgartner/32/92716_2.png) [@Jason\_Baumgartner](https://discuss.elastic.co/u/Jason_Baumgartner)\
**Post date:** [June 25, 2018, 9:46am UTC](https://discuss.elastic.co/t/possible-bug-with-6-3-analyzer-not-working-correctly/137225/1 "2018-06-25T09:46:38Z")

</div>

I have the following mapping:

```
{
    "settings": {
        "index.codec": "best_compression",
        "index.shard.check_on_startup": "checksum",
        "index": {
            "number_of_shards": 10,
            "number_of_replicas": 0,
            "sort.field" : "created_at", 
            "sort.order" : "desc"
        },
        "analysis": {
            "analyzer": {
                "custom": {
                    "filter": ["lowercase"],
                    "tokenizer": "whitespace"
                }
            },
            "normalizer": {
                "my_normalizer": {
                    "type": "custom",
                    "char_filter": [],
                    "filter": ["lowercase", "asciifolding"]
                }
            }
        }
    },
    "mappings": {
        "user": {
            "_routing": {
                "required": false
            },
            "_all": {
                "enabled": false
            },
            "_source": {
                "enabled": true,
                "excludes": ["id"]
            },
            "dynamic": "false",
            "properties": {
                "description": {
                    "type": "text",
                    "analyzer": "custom"
                },
                "location": {
                    "type": "text",
                    "analyzer": "custom",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 128
                        }
                    }
                },
                "name": {
                    "type": "text",
                    "analyzer": "custom",
                    "fields": {
                        "keyword": {
                            "type": "keyword",
                            "ignore_above": 128
                        }
                    }
                },
                "followers_count": {
                    "type": "integer"
                },
                "friends_count": {
                    "type": "integer"
                },
                "listed_count": {
                    "type": "integer"
                },
                "favourites_count": {
                    "type": "integer"
                },
                "statuses_count": {
                    "type": "integer"
                },
                "lang": {
                    "type": "keyword",
                    "normalizer": "my_normalizer"
                },
                "screen_name": {
                    "type": "keyword",
                    "normalizer": "my_normalizer"
                },
                "profile_image_url_https": {
                    "type": "keyword",
                    "normalizer": "my_normalizer"
                },
                "profile_background_image_url_https": {
                    "type": "keyword",
                    "normalizer": "my_normalizer"
                },
                "profile_use_background_image": {
                    "type": "boolean"
                },
                "verified": {
                    "type": "boolean"
                },
                "protected": {
                    "type": "boolean"
                },
                "default_profile": {
                    "type": "boolean"
                },
                "created_at": {
                    "type": "date",
                    "format": "epoch_second"
                },
                "retrieved_on": {
                    "type": "date",
                    "format": "epoch_second"
                },
                "last_updated": {
                    "type": "date",
                    "format": "epoch_second"
                },
                "id": {
                    "type": "long"
                }
            }
        }
    }
}

```

I tested that the analyzer works correctly to get hashtags:

`curl -H "Content-Type: application/json" -XPOST localhost:9200/twitter/_analyze -d '{"text":"This is a test to see how a #hashtag is tokenized","analyzer":"custom"}'`

> {  
> "tokens" : [  
> {  
> "token" : "this",  
> "start\_offset" : 0,  
> "end\_offset" : 4,  
> "type" : "word",  
> "position" : 0  
> },  
> {  
> "token" : "is",  
> "start\_offset" : 5,  
> "end\_offset" : 7,  
> "type" : "word",  
> "position" : 1  
> },  
> {  
> "token" : "a",  
> "start\_offset" : 8,  
> "end\_offset" : 9,  
> "type" : "word",  
> "position" : 2  
> },  
> {  
> "token" : "test",  
> "start\_offset" : 10,  
> "end\_offset" : 14,  
> "type" : "word",  
> "position" : 3  
> },  
> {  
> "token" : "to",  
> "start\_offset" : 15,  
> "end\_offset" : 17,  
> "type" : "word",  
> "position" : 4  
> },  
> {  
> "token" : "see",  
> "start\_offset" : 18,  
> "end\_offset" : 21,  
> "type" : "word",  
> "position" : 5  
> },  
> {  
> "token" : "how",  
> "start\_offset" : 22,  
> "end\_offset" : 25,  
> "type" : "word",  
> "position" : 6  
> },  
> {  
> "token" : "a",  
> "start\_offset" : 26,  
> "end\_offset" : 27,  
> "type" : "word",  
> "position" : 7  
> },  
> {  
> "token" : "#hashtag",  
> "start\_offset" : 28,  
> "end\_offset" : 36,  
> "type" : "word",  
> "position" : 8  
> },  
> {  
> "token" : "is",  
> "start\_offset" : 37,  
> "end\_offset" : 39,  
> "type" : "word",  
> "position" : 9  
> },  
> {  
> "token" : "tokenized",  
> "start\_offset" : 40,  
> "end\_offset" : 49,  
> "type" : "word",  
> "position" : 10  
> }  
> ]  
> }

However, when I index and try to search on hashtags, I get no results:

`curl -H "Content-Type: application/json" -XGET localhost:9200/twitter/user/_search?pretty=true -d '{"query":{"match":{"description":{"query":"#privacy"}}}}'`

```
{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 10,
    "successful" : 10,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 0,
    "max_score" : null,
    "hits" : []
  }
}

```

Is this something that broke recently?

Bug: Elasticsearch is not using the same search analyzer as that defined for indexing.  
Edit: Honestly, I don't know what's going on here. It's still not working as documented.

---

<div class="post-metadata">

**Author:** ![polyfractal](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/polyfractal/32/48162_2.png) [@polyfractal](https://discuss.elastic.co/u/polyfractal)\
**Post date:** [June 26, 2018, 7:07pm UTC](https://discuss.elastic.co/t/possible-bug-with-6-3-analyzer-not-working-correctly/137225/2 "2018-06-26T19:07:00Z")

</div>

Can you show some of the documents that should be returned but aren't? I can try to reproduce it locally.

I've not heard anyone else encountering something like this yet, would be good to hunt down the problem.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 24, 2018, 7:07pm UTC](https://discuss.elastic.co/t/possible-bug-with-6-3-analyzer-not-working-correctly/137225/3 "2018-07-24T19:07:00Z")

</div>

Closed on July 24, 2018, 7:07pm UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.