# Searching not analyzed term name and value in case insensitive manner **URL:** \ **Category:** Elasticsearch\ **Created:** [May 20, 2016, 7:51am UTC](https://discuss.elastic.co/t/searching-not-analyzed-term-name-and-value-in-case-insensitive-manner/50537 "2016-05-20T07:51:38Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![nrmohta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nrmohta/32/120038_2.png) [@nrmohta](https://discuss.elastic.co/u/nrmohta)\ **Post date:** [May 20, 2016, 7:51am UTC](https://discuss.elastic.co/t/searching-not-analyzed-term-name-and-value-in-case-insensitive-manner/50537/1 "2016-05-20T07:51:38Z")
Searching term name and value in case insensitive manner Present structure - Field Mapping: "NAME": { "type": "string", "index": "not\_analyzed", "store": true } Say Sample Data in Field : Neeraj Problem : Search is now case sensitive, we need to have case insensitive behaviour Search NAME:Neeraj - we are getting result name:neeraj - no result NAME:NEERAJ - no result We want behaviour such that all above queries should return result. Please suggest options to do it. One option is to use standard analyzer with lowercase filter, but in that option I am not sure how to migrate old data. ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [May 24, 2016, 7:05am UTC](https://discuss.elastic.co/t/searching-not-analyzed-term-name-and-value-in-case-insensitive-manner/50537/2 "2016-05-24T07:05:18Z")
Hi @nrmohta, you can use the keyword tokenizer: ```auto PUT /my_index { "settings": { "analysis": { "analyzer": { "case_insensitive": { "tokenizer": "keyword", "filter": [ "lowercase" ] } } } }, "mappings": { "my_type": { "properties": { "name": { "type": "string", "analyzer": "case_insensitive" } } } } } ``` ```auto PUT /my_index/my_type/1 { "name": "Neeraj" } ``` ```auto GET /my_index/_search?q=name:Neeraj GET /my_index/_search?q=name:neeraj ``` For further details see [Elasticsearch: The Definitive Guide](https://www.elastic.co/guide/en/elasticsearch/guide/2.x/sorting-collations.html#case-insensitive-sorting). To your question for the migration path: When you change the analyzer, you have to reindex your old data. Daniel ---
**Author:** ![nrmohta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nrmohta/32/120038_2.png) [@nrmohta](https://discuss.elastic.co/u/nrmohta)\ **Post date:** [May 27, 2016, 12:16pm UTC](https://discuss.elastic.co/t/searching-not-analyzed-term-name-and-value-in-case-insensitive-manner/50537/3 "2016-05-27T12:16:51Z")
Thanks a lot Daniel for you suggestions. This answered query related to field value. However we also have requirement in which field name can also be search case insensitively. Please guide us. example: GET /my\_index/\_search?q=name:Neeraj GET /my\_index/\_search?q=NAME:Neeraj. Requirement - Both above queries should return the results. difference is in field name 'name' and 'NAME' ---
**Author:** ![anjith.p](https://avatars.discourse-cdn.com/v4/letter/a/a3d4f5/32.png) [@anjith.p](https://discuss.elastic.co/u/anjith.p)\ **Post date:** [June 15, 2016, 8:19am UTC](https://discuss.elastic.co/t/searching-not-analyzed-term-name-and-value-in-case-insensitive-manner/50537/4 "2016-06-15T08:19:29Z")
Is it posssible to achieve case insensitive searching using scripting? I looked at the docs but they were not clear. ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [June 15, 2016, 1:25pm UTC](https://discuss.elastic.co/t/searching-not-analyzed-term-name-and-value-in-case-insensitive-manner/50537/5 "2016-06-15T13:25:09Z")
Hi @nrmohta for some reason I missed that you asked again. /cc: @anjith.p Did you try this? This example works for me on Elasticsearch 2.3(i.e. I get search results): ```auto PUT /company/employee/1 { "first_name" : "John", "last_name" : "Smith", } GET /company/employee/_search?q:FiRsT_NaME=John&explain=true ``` Daniel ---
**Author:** ![anjith.p](https://avatars.discourse-cdn.com/v4/letter/a/a3d4f5/32.png) [@anjith.p](https://discuss.elastic.co/u/anjith.p)\ **Post date:** [June 15, 2016, 2:00pm UTC](https://discuss.elastic.co/t/searching-not-analyzed-term-name-and-value-in-case-insensitive-manner/50537/6 "2016-06-15T14:00:37Z")
Daniel , your test is workking because your strings are analyzed. But, in our caes we have non-analyzed strings and converting them to analyzed strings is not an option. ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [June 15, 2016, 2:22pm UTC](https://discuss.elastic.co/t/searching-not-analyzed-term-name-and-value-in-case-insensitive-manner/50537/7 "2016-06-15T14:22:34Z")
Hi @anjith.p, sorry for the confusion. In my example I was just demonstrating that the field name casing does not matter. The other issue has been resolved hopefully by my original answer, hence I did not include it in this example again. Daniel ---
**Author:** ![anjith.p](https://avatars.discourse-cdn.com/v4/letter/a/a3d4f5/32.png) [@anjith.p](https://discuss.elastic.co/u/anjith.p)\ **Post date:** [June 15, 2016, 2:27pm UTC](https://discuss.elastic.co/t/searching-not-analyzed-term-name-and-value-in-case-insensitive-manner/50537/8 "2016-06-15T14:27:28Z")
Thanks Daniel. Your original answer suggests that we should indeed use analyzeer. But, in our case, all our strings are not analyzed and we can't lose the original case. So, I'm wondering if there is anyway of achieving the case insensitive functionality for non-analyzed strings. May be through scripting? Thanks for your help. ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [June 15, 2016, 2:33pm UTC](https://discuss.elastic.co/t/searching-not-analyzed-term-name-and-value-in-case-insensitive-manner/50537/9 "2016-06-15T14:33:59Z")
Hi @anjith.p, you can use multi-fields and apply the analyzer just there. The reference documentation contains an [example for multi-fields](https://www.elastic.co/guide/en/elasticsearch/reference/2.3/_multi_fields.html#_multi_fields) but note that it's the other way around in the example (i.e. in your case you want the "main" field `not_analyzed` but the subfield `analyzed`). Does that work for you? Daniel ---
**Author:** ![anjith.p](https://avatars.discourse-cdn.com/v4/letter/a/a3d4f5/32.png) [@anjith.p](https://discuss.elastic.co/u/anjith.p)\ **Post date:** [June 15, 2016, 3:03pm UTC](https://discuss.elastic.co/t/searching-not-analyzed-term-name-and-value-in-case-insensitive-manner/50537/10 "2016-06-15T15:03:47Z")
What I understand from your answer is that I need to save my field as both analyzed and non-analyzed and analyzed will be used for case insensitve searching. Is my assumtion correct? If yes, won't that increase the storage by double. In our case, most of our fields are not-analyzed and we need to support case insensitive searching for them. Is using scripts(though I'm not clear about scripting at the moment) an option? ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [June 16, 2016, 6:19am UTC](https://discuss.elastic.co/t/searching-not-analyzed-term-name-and-value-in-case-insensitive-manner/50537/11 "2016-06-16T06:19:03Z")
Hi @anjith.p, that's correct, you need to separate fields but this does not necessarily mean that your index size doubles (although I guess it will be close to that if you have a lot of not\_analyzed fields). I suggest that you run a couple of experiments to determine the impact on index size (I also suggest you run a [force\_merge](https://www.elastic.co/guide/en/elasticsearch/reference/2.3/indices-forcemerge.html) at the end of your test to keep sizes comparable). Daniel ---
**Author:** ![anjith.p](https://avatars.discourse-cdn.com/v4/letter/a/a3d4f5/32.png) [@anjith.p](https://discuss.elastic.co/u/anjith.p)\ **Post date:** [June 16, 2016, 6:42am UTC](https://discuss.elastic.co/t/searching-not-analyzed-term-name-and-value-in-case-insensitive-manner/50537/12 "2016-06-16T06:42:28Z")
Thanks Daniel. We will go with your suggestions. But I even wanted to know if this functionality could be done through ES scripting. Any Ideas? Appreiciate your help. ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [June 16, 2016, 11:48am UTC](https://discuss.elastic.co/t/searching-not-analyzed-term-name-and-value-in-case-insensitive-manner/50537/13 "2016-06-16T11:48:59Z")
Hi @anjith.p, a scripting solution is possible but this will be **painfully slow** compared to a native search. Here is a working example (tested with Elasticsearch 2.3.3): Enable file based scripting in config/elasticsearch.yml: ```auto script.file: true ``` Add a file "match.groovy" in config/scripts/: ```auto doc['name'].value.toLowerCase() == 'bob' ``` Note: You can use parameters in scripts (see docs) but this is a minimalistic example so I've hardcoded the value but I hope you get the idea. Create an index: ```auto PUT cases { "mappings": { "people": { "properties": { "name": { "type": "string", "index": "not_analyzed" } } } } } ``` Now add Bob: ```auto POST /cases/people/1 { "name": "Bob" } ``` And search via a script query: ```auto GET /cases/people/_search { "query": { "bool": { "must": [ { "match_all": {} } ], "filter": { "script": { "script": { "file": "match" } } } } } } ``` You get: ```auto { "took": 202, "timed_out": false, "_shards": { "total": 5, "successful": 5, "failed": 0 }, "hits": { "total": 1, "max_score": 1, "hits": [ { "_index": "cases", "_type": "people", "_id": "1", "_score": 1, "_source": { "name": "Bob" } } ] } } ``` Now the same with a term query: ```auto GET /cases/people/_search { "query": { "term": { "name": { "value": "Bob" } } } } ``` ```auto { "took": 9, "timed_out": false, "_shards": { "total": 5, "successful": 5, "failed": 0 }, "hits": { "total": 1, "max_score": 0.30685282, "hits": [ { "_index": "cases", "_type": "people", "_id": "1", "_score": 0.30685282, "_source": { "name": "Bob" } } ] } } ``` I had just one document stored and you already see the performance difference: 9 ms vs 202 ms. Please keep this in mind and test the performance early because it will be much slower with this approach. Daniel ---
**Author:** ![anjith.p](https://avatars.discourse-cdn.com/v4/letter/a/a3d4f5/32.png) [@anjith.p](https://discuss.elastic.co/u/anjith.p)\ **Post date:** [June 16, 2016, 11:51am UTC](https://discuss.elastic.co/t/searching-not-analyzed-term-name-and-value-in-case-insensitive-manner/50537/14 "2016-06-16T11:51:06Z")
Thanks Daniel. Yes, I have just figured out that scripts actually work 🙂 Fortuantely, slow performance is OK in our use case. Really appreciate your response. I'll indeed write one article about ES scripting:It is really powerful 🙂 ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [June 16, 2016, 11:57am UTC](https://discuss.elastic.co/t/searching-not-analyzed-term-name-and-value-in-case-insensitive-manner/50537/15 "2016-06-16T11:57:48Z")
Hi @anjith.p, glad that I could help! 🙂 Can you share the link to the article here once you've published it? Daniel ---
**Author:** ![anjith.p](https://avatars.discourse-cdn.com/v4/letter/a/a3d4f5/32.png) [@anjith.p](https://discuss.elastic.co/u/anjith.p)\ **Post date:** [June 17, 2016, 4:43am UTC](https://discuss.elastic.co/t/searching-not-analyzed-term-name-and-value-in-case-insensitive-manner/50537/16 "2016-06-17T04:43:51Z")
I'll for sure. We have actually made the script "native" by using java plug-in and found out that it's performance is as good as regular search(without script) performance. ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [June 17, 2016, 5:52am UTC](https://discuss.elastic.co/t/searching-not-analyzed-term-name-and-value-in-case-insensitive-manner/50537/17 "2016-06-17T05:52:46Z")
Good to hear that the performance is sufficient (although I'm a bit surprised that it's really comparable to a non-script solution). ---
**Author:** ![anjith.p](https://avatars.discourse-cdn.com/v4/letter/a/a3d4f5/32.png) [@anjith.p](https://discuss.elastic.co/u/anjith.p)\ **Post date:** [June 17, 2016, 10:01am UTC](https://discuss.elastic.co/t/searching-not-analyzed-term-name-and-value-in-case-insensitive-manner/50537/18 "2016-06-17T10:01:07Z")
I suppose the query syntax you used is incorrect here. Correct query looks like the below: GET /company/employee/\_search?q=FiRsT\_NaME:John&explain=true And the above query doesn't return any results. ---
**Author:** ![danielmitterdorfer](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/danielmitterdorfer/32/110510_2.png) [@danielmitterdorfer](https://discuss.elastic.co/u/danielmitterdorfer)\ **Post date:** [June 17, 2016, 11:23am UTC](https://discuss.elastic.co/t/searching-not-analyzed-term-name-and-value-in-case-insensitive-manner/50537/19 "2016-06-17T11:23:29Z")
Not sure what I copy & pasted here because I try out all examples in Sense before posting. But your syntax is correct and the field name in the search must match exactly the field name in the mapping (that's the reason why you get no results). ---
**Author:** ![nrmohta](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nrmohta/32/120038_2.png) [@nrmohta](https://discuss.elastic.co/u/nrmohta)\ **Post date:** [June 20, 2016, 12:32pm UTC](https://discuss.elastic.co/t/searching-not-analyzed-term-name-and-value-in-case-insensitive-manner/50537/20 "2016-06-20T12:32:16Z")
To enable case insensitive search on field name - option we are thinking is to maintain the cache of all fields in memory and comparing search input field name against this cache list case insensitively. Fields which matches case insensitively, to be added in 'fields' query so that search will get perform on all those fields. Another option could be lower casing all fields while indexing, but as per our requirement we do want to support case sensitivity on field names. In case you have pointers to achieve our goal in a different way, please suggest. [Next page](https://discuss.elastic.co/t/searching-not-analyzed-term-name-and-value-in-case-insensitive-manner/50537.md?page=2)