# Lowercase searches not working

**URL:** <https://discuss.elastic.co/t/lowercase-searches-not-working/12087>\
**Category:** Elasticsearch\
**Created:** [May 23, 2013, 7:20am UTC](https://discuss.elastic.co/t/lowercase-searches-not-working/12087 "2013-05-23T07:20:36Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Chris\_Kinsalb](https://avatars.discourse-cdn.com/v4/letter/c/e99b99/32.png) [@Chris\_Kinsalb](https://discuss.elastic.co/u/Chris_Kinsalb)\
**Post date:** [May 23, 2013, 7:20am UTC](https://discuss.elastic.co/t/lowercase-searches-not-working/12087/1 "2013-05-23T07:20:36Z")

</div>

Hello group,

Here's what I am trying to achieve:

For testing purposes I created a default template which is inserted using  
the \_template  
endpoint into elastic search. The template should be applied to all indexes  
starting with the string "docs" (for brevity I omitted some of the other  
fields)

{  
"template": "docs\*",  
"settings": {  
"query.default\_field": "@message",  
"action.auto\_create\_index": "+docs\*",  
"number\_of\_shards": 1,  
"number\_of\_replicas": 1,  
"routing.allocation.total\_shards\_per\_node": 1,  
"auto\_expand\_replicas": false,  
"index": {  
"analysis": {  
"analyzer": {  
"my\_doc\_analyser": {  
"type": "custom",  
"tokenizer": "whitespace",  
"filter": ["standard", "lowercase"]  
}  
}  
}  
}  
},  
"mappings": {  
"_default_": {  
"\_all": {  
"enabled": true  
},  
"\_source": {  
"compress": false  
},  
"properties": {  
"@time": {  
"type": "date",  
"index": "not\_analyzed"  
},  
"@message": {  
"type": "string",  
"index": "analyzed",  
"analyzer": "my\_doc\_analyser"  
}  
}  
}  
}  
}

Now, I insert a document into my elasticsearch instance, for instance  
having the  
text: (index is automatically created)

"Hello, how are you doing Donny"

I then issue a query\_string search to this endpoint:

[http://127.0.0.1:9200/docs-testicus/\_search](http://127.0.0.1:9200/docs-testicus/_search)

... using this query in the body:

{  
"query": {  
"query\_string": {  
"query": "Donny"  
}  
}  
}

.. I then get the (1) result.

Now, I EXPECTED to also be able to issue searches with lowercased inputs,  
such as:

{  
"query": {  
"query\_string": {  
"query": "donny"  
}  
}  
}

However, this gives me 0 hits.

Can anyone explain WHY I cannot do lower-case searches when a lowercase  
filter is part  
of the analyzer?

When I tried testing my custom analyzer using the \_analyze endpoint, I can  
see  
that the input tokens are indeed turned into lowercase.

Cheers,

- Chris

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Franck\_GIRODON](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/franck_girodon/32/2318_2.png) [@Franck\_GIRODON](https://discuss.elastic.co/u/Franck_GIRODON)\
**Post date:** [May 23, 2013, 7:45am UTC](https://discuss.elastic.co/t/lowercase-searches-not-working/12087/2 "2013-05-23T07:45:04Z")

</div>

Hello, it's strange ... I have created the same index and the same seach  
and I have a result for "Donny" or "donny" ...

Le jeudi 23 mai 2013 09:20:36 UTC+2, Chris Kinsalb a écrit :

> Hello group,
> 
> Here's what I am trying to achieve:
> 
> For testing purposes I created a default template which is inserted using  
> the \_template  
> endpoint into Elasticsearch. The template should be applied to all indexes  
> starting with the string "docs" (for brevity I omitted some of the other  
> fields)
> 
> {  
> "template": "docs\*",  
> "settings": {  
> "query.default\_field": "@message",  
> "action.auto\_create\_index": "+docs\*",  
> "number\_of\_shards": 1,  
> "number\_of\_replicas": 1,  
> "routing.allocation.total\_shards\_per\_node": 1,  
> "auto\_expand\_replicas": false,  
> "index": {  
> "analysis": {  
> "analyzer": {  
> "my\_doc\_analyser": {  
> "type": "custom",  
> "tokenizer": "whitespace",  
> "filter": ["standard", "lowercase"]  
> }  
> }  
> }  
> }  
> },  
> "mappings": {  
> "_default_": {  
> "\_all": {  
> "enabled": true  
> },  
> "\_source": {  
> "compress": false  
> },  
> "properties": {  
> "@time": {  
> "type": "date",  
> "index": "not\_analyzed"  
> },  
> "@message": {  
> "type": "string",  
> "index": "analyzed",  
> "analyzer": "my\_doc\_analyser"  
> }  
> }  
> }  
> }  
> }
> 
> Now, I insert a document into my elasticsearch instance, for instance  
> having the  
> text: (index is automatically created)
> 
> "Hello, how are you doing Donny"
> 
> I then issue a query\_string search to this endpoint:
> 
> [http://127.0.0.1:9200/docs-testicus/\_search](http://127.0.0.1:9200/docs-testicus/_search)
> 
> ... using this query in the body:
> 
> {  
> "query": {  
> "query\_string": {  
> "query": "Donny"  
> }  
> }  
> }
> 
> .. I then get the (1) result.
> 
> Now, I EXPECTED to also be able to issue searches with lowercased inputs,  
> such as:
> 
> {  
> "query": {  
> "query\_string": {  
> "query": "donny"  
> }  
> }  
> }
> 
> However, this gives me 0 hits.
> 
> Can anyone explain WHY I cannot do lower-case searches when a lowercase  
> filter is part  
> of the analyzer?
> 
> When I tried testing my custom analyzer using the \_analyze endpoint, I can  
> see  
> that the input tokens are indeed turned into lowercase.
> 
> Cheers,
> 
> - Chris

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Chris\_Kinsalb](https://avatars.discourse-cdn.com/v4/letter/c/e99b99/32.png) [@Chris\_Kinsalb](https://discuss.elastic.co/u/Chris_Kinsalb)\
**Post date:** [May 23, 2013, 8:09am UTC](https://discuss.elastic.co/t/lowercase-searches-not-working/12087/3 "2013-05-23T08:09:07Z")

</div>

Hello,

I just upgraded my test instance to 0.90.0 (used 0.20.6 previously) and am  
still getting the same effect.

Did you do anything differently?  
E.g. create the document type mapping for the index separately (i.e. by  
explicitly creating the index?).

Cheerio,

- Chris

On Thursday, May 23, 2013 9:45:04 AM UTC+2, Franck GIRODON wrote:

> Hello, it's strange ... I have created the same index and the same seach  
> and I have a result for "Donny" or "donny" ...
> 
> Le jeudi 23 mai 2013 09:20:36 UTC+2, Chris Kinsalb a écrit :
> 
> > Hello group,
> > 
> > Here's what I am trying to achieve:
> > 
> > For testing purposes I created a default template which is inserted using  
> > the \_template  
> > endpoint into Elasticsearch. The template should be applied to all  
> > indexes  
> > starting with the string "docs" (for brevity I omitted some of the other  
> > fields)
> > 
> > {  
> > "template": "docs\*",  
> > "settings": {  
> > "query.default\_field": "@message",  
> > "action.auto\_create\_index": "+docs\*",  
> > "number\_of\_shards": 1,  
> > "number\_of\_replicas": 1,  
> > "routing.allocation.total\_shards\_per\_node": 1,  
> > "auto\_expand\_replicas": false,  
> > "index": {  
> > "analysis": {  
> > "analyzer": {  
> > "my\_doc\_analyser": {  
> > "type": "custom",  
> > "tokenizer": "whitespace",  
> > "filter": ["standard", "lowercase"]  
> > }  
> > }  
> > }  
> > }  
> > },  
> > "mappings": {  
> > "_default_": {  
> > "\_all": {  
> > "enabled": true  
> > },  
> > "\_source": {  
> > "compress": false  
> > },  
> > "properties": {  
> > "@time": {  
> > "type": "date",  
> > "index": "not\_analyzed"  
> > },  
> > "@message": {  
> > "type": "string",  
> > "index": "analyzed",  
> > "analyzer": "my\_doc\_analyser"  
> > }  
> > }  
> > }  
> > }  
> > }
> > 
> > Now, I insert a document into my elasticsearch instance, for instance  
> > having the  
> > text: (index is automatically created)
> > 
> > "Hello, how are you doing Donny"
> > 
> > I then issue a query\_string search to this endpoint:
> > 
> > [http://127.0.0.1:9200/docs-testicus/\_search](http://127.0.0.1:9200/docs-testicus/_search)
> > 
> > ... using this query in the body:
> > 
> > {  
> > "query": {  
> > "query\_string": {  
> > "query": "Donny"  
> > }  
> > }  
> > }
> > 
> > .. I then get the (1) result.
> > 
> > Now, I EXPECTED to also be able to issue searches with lowercased inputs,  
> > such as:
> > 
> > {  
> > "query": {  
> > "query\_string": {  
> > "query": "donny"  
> > }  
> > }  
> > }
> > 
> > However, this gives me 0 hits.
> > 
> > Can anyone explain WHY I cannot do lower-case searches when a lowercase  
> > filter is part  
> > of the analyzer?
> > 
> > When I tried testing my custom analyzer using the \_analyze endpoint, I  
> > can see  
> > that the input tokens are indeed turned into lowercase.
> > 
> > Cheers,
> > 
> > - Chris

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\
**Post date:** [May 23, 2013, 10:49am UTC](https://discuss.elastic.co/t/lowercase-searches-not-working/12087/4 "2013-05-23T10:49:27Z")

</div>

Chris - check the mapping and settings of your index.

I don't see anything that is obviously wrong, but you may have a typo  
somewhere which is producing results other than what you expect

clint

On 23 May 2013 10:09, Chris Kinsalb [hc.kinsalb@gmail.com](mailto:hc.kinsalb@gmail.com) wrote:

> Hello,
> 
> I just upgraded my test instance to 0.90.0 (used 0.20.6 previously) and am  
> still getting the same effect.
> 
> Did you do anything differently?  
> E.g. create the document type mapping for the index separately (i.e. by  
> explicitly creating the index?).
> 
> Cheerio,
> 
> - Chris
> 
> On Thursday, May 23, 2013 9:45:04 AM UTC+2, Franck GIRODON wrote:
> 
> > Hello, it's strange ... I have created the same index and the same seach  
> > and I have a result for "Donny" or "donny" ...
> > 
> > Le jeudi 23 mai 2013 09:20:36 UTC+2, Chris Kinsalb a écrit :
> > 
> > > Hello group,
> > > 
> > > Here's what I am trying to achieve:
> > > 
> > > For testing purposes I created a default template which is inserted  
> > > using the \_template  
> > > endpoint into Elasticsearch. The template should be applied to all  
> > > indexes  
> > > starting with the string "docs" (for brevity I omitted some of the other  
> > > fields)
> > > 
> > > {  
> > > "template": "docs\*",  
> > > "settings": {  
> > > "query.default\_field": "@message",  
> > > "action.auto\_create\_index": "+docs\*",  
> > > "number\_of\_shards": 1,  
> > > "number\_of\_replicas": 1,  
> > > "routing.allocation.total\_\*\*shards\_per\_node": 1,  
> > > "auto\_expand\_replicas": false,  
> > > "index": {  
> > > "analysis": {  
> > > "analyzer": {  
> > > "my\_doc\_analyser": {  
> > > "type": "custom",  
> > > "tokenizer": "whitespace",  
> > > "filter": ["standard", "lowercase"]  
> > > }  
> > > }  
> > > }  
> > > }  
> > > },  
> > > "mappings": {  
> > > "_default_": {  
> > > "\_all": {  
> > > "enabled": true  
> > > },  
> > > "\_source": {  
> > > "compress": false  
> > > },  
> > > "properties": {  
> > > "@time": {  
> > > "type": "date",  
> > > "index": "not\_analyzed"  
> > > },  
> > > "@message": {  
> > > "type": "string",  
> > > "index": "analyzed",  
> > > "analyzer": "my\_doc\_analyser"  
> > > }  
> > > }  
> > > }  
> > > }  
> > > }
> > > 
> > > Now, I insert a document into my elasticsearch instance, for instance  
> > > having the  
> > > text: (index is automatically created)
> > > 
> > > "Hello, how are you doing Donny"
> > > 
> > > I then issue a query\_string search to this endpoint:
> > > 
> > > [http://127.0.0.1:9200/docs-\*\*testicus/\_search](http://127.0.0.1:9200/docs-**testicus/_search)[http://127.0.0.1:9200/docs-testicus/\_search](http://127.0.0.1:9200/docs-testicus/_search)
> > > 
> > > ... using this query in the body:
> > > 
> > > {  
> > > "query": {  
> > > "query\_string": {  
> > > "query": "Donny"  
> > > }  
> > > }  
> > > }
> > > 
> > > .. I then get the (1) result.
> > > 
> > > Now, I EXPECTED to also be able to issue searches with lowercased  
> > > inputs, such as:
> > > 
> > > {  
> > > "query": {  
> > > "query\_string": {  
> > > "query": "donny"  
> > > }  
> > > }  
> > > }
> > > 
> > > However, this gives me 0 hits.
> > > 
> > > Can anyone explain WHY I cannot do lower-case searches when a lowercase  
> > > filter is part  
> > > of the analyzer?
> > > 
> > > When I tried testing my custom analyzer using the \_analyze endpoint, I  
> > > can see  
> > > that the input tokens are indeed turned into lowercase.
> > > 
> > > Cheers,
> > > 
> > > - Chris
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Chris\_Kinsalb](https://avatars.discourse-cdn.com/v4/letter/c/e99b99/32.png) [@Chris\_Kinsalb](https://discuss.elastic.co/u/Chris_Kinsalb)\
**Post date:** [May 23, 2013, 1:10pm UTC](https://discuss.elastic.co/t/lowercase-searches-not-working/12087/5 "2013-05-23T13:10:59Z")

</div>

Hi Clint,  
thanks for the check.

Now I actually found the solution, and it did not really make me laugh  
(especially considering the time that I spent figuring out why this  
happened) ;-).

In my first post I changed the config a bit, here's the "REAL"  
configuration for the template I used (there's only a SLIGHT difference  
anyway).  
Btw, eventually we will use this solution for storing log messages.

{  
"template": "logs\*",  
"settings": {  
"query.default\_field": "@message",  
"action.auto\_create\_index": "+logs\*",  
"number\_of\_shards": 1,  
"number\_of\_replicas": 1,  
"routing.allocation.total\_shards\_per\_node": 1,  
"auto\_expand\_replicas": false,  
"index": {  
"analysis": {  
"analyzer": {  
"my\_log\_analyser": {  
"tokenizer": "whitespace",  
"filter": [  
"standard",  
"lowercase"  
],  
"type": "custom"  
}  
}  
}  
}  
},  
"mappings": {  
"_default_": {  
"\_all": {  
"enabled": false  
},  
"\_source": {  
"compress": false  
},  
"dynamic\_templates": [  
{  
"fields\_template": {  
"mapping": {  
"type": "string",  
"index": "not\_analyzed"  
},  
"path\_match": "@fields.\*"  
}  
}  
],  
"properties": {  
"@time": {  
"type": "date",  
"index": "not\_analyzed"  
},  
"@message": {  
"type": "string",  
"index": "analyzed",  
"analyzer": "my\_log\_analyser"  
},  
"@fields": {  
"type": "object",  
"dynamic": true,  
"path": "full"  
}  
}  
}  
}  
}

Looking at the template you will spot that the only differences actually  
are the\* index prefix name\* to which this template is applied & the name of  
the log analyser.

To cut a long story short, the answer is that IFF (as in "if and only if")  
the index prefix name for a template starts with either  
_logs_\*  
or  
_logs-_\*

Elasticsearch gives me the finger and only does half of the analysis  
(apparently).  
Renaming the prefix to anything else, e.g. "narf" or "foo" or "docs" (as  
posted in my original message) will do the trick and everything works as  
expected.

This seems to have the word "bug" all over it, so I will post an issue on  
github with this.

Anyway, thanks a lot for your support!

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:35am UTC](https://discuss.elastic.co/t/lowercase-searches-not-working/12087/6 "2017-07-06T02:35:05Z")

</div>