# Case In/Sensitive Searching **URL:** \ **Category:** Elasticsearch\ **Created:** [September 23, 2010, 5:47pm UTC](https://discuss.elastic.co/t/case-in-sensitive-searching/3362 "2010-09-23T17:47:52Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![bailey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bailey/32/3278_2.png) [@bailey](https://discuss.elastic.co/u/bailey)\ **Post date:** [September 23, 2010, 5:47pm UTC](https://discuss.elastic.co/t/case-in-sensitive-searching/3362/1 "2010-09-23T17:47:52Z")
Hi All- I know that case insensitive searching has been covered in a couple other threads but I feel like this question is a little different. We have two fields that need to be case insensitive when we search for them as they need to be completely unique across the application, say fields "username" and "emailAddress". Setting the "username" field to be analyzed solves the issue for it as we only want alphanumeric characters to comprise the username (I feel like we would be in trouble though if we had to set it as "analyzed"). But if we set "emailAddress" to be analyzed will the default analyzer split it based on the "@" symbol (therefore returning anything that matches after the @ symbol)? Is there a configuration (or query parameters) to have a field set to "not\_analyzed" but still search case insensitively? Thanks! ---
**Author:** ![bailey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bailey/32/3278_2.png) [@bailey](https://discuss.elastic.co/u/bailey)\ **Post date:** [September 23, 2010, 5:50pm UTC](https://discuss.elastic.co/t/case-in-sensitive-searching/3362/2 "2010-09-23T17:50:52Z")
"(I feel like we would be in trouble though if we had to set it as "analyzed")." should read "(I feel like we would be in trouble though if we had to set it as "analyzed" and accepted any characters like dashes, underscores, etc...)." On Sep 23, 1:47 pm, bailey [matt.bai...@gmail.com](mailto:matt.bai...@gmail.com) wrote: > Hi All- > > I know that case insensitive searching has been covered in a couple > other threads but I feel like this question is a little different. > > We have two fields that need to be case insensitive when we search for > them as they need to be completely unique across the application, say > fields "username" and "emailAddress". > > Setting the "username" field to be analyzed solves the issue for it as > we only want alphanumeric characters to comprise the username (I feel > like we would be in trouble though if we had to set it as "analyzed"). > But if we set "emailAddress" to be analyzed will the default analyzer > split it based on the "@" symbol (therefore returning anything that > matches after the @ symbol)? > > Is there a configuration (or query parameters) to have a field set to > "not\_analyzed" but still search case insensitively? > > Thanks! ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [September 23, 2010, 6:00pm UTC](https://discuss.elastic.co/t/case-in-sensitive-searching/3362/3 "2010-09-23T18:00:03Z")
On Thu, 2010-09-23 at 10:50 -0700, bailey wrote: > "(I feel like we would be in trouble though if we had to set it as > "analyzed")." > > should read > > "(I feel like we would be in trouble though if we had to set it as > "analyzed" and accepted any characters like dashes, underscores, > etc...)." You will need to create a custom analyzer using the lowercase token filter: [http://www.elasticsearch.com/docs/elasticsearch/index\_modules/analysis/](http://www.elasticsearch.com/docs/elasticsearch/index_modules/analysis/) [http://www.elasticsearch.com/docs/elasticsearch/index\_modules/analysis/tokenfilter/lowercase](http://www.elasticsearch.com/docs/elasticsearch/index_modules/analysis/tokenfilter/lowercase) clint -- Web Announcements Limited is a company registered in England and Wales, with company number 05608868, with registered address at 10 Arvon Road, London, N5 1PR. ---
**Author:** ![ppearcy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppearcy/32/980_2.png) [@ppearcy](https://discuss.elastic.co/u/ppearcy)\ **Post date:** [September 23, 2010, 11:53pm UTC](https://discuss.elastic.co/t/case-in-sensitive-searching/3362/4 "2010-09-23T23:53:28Z")
FYI, here is the analyzer I created for this same case: index: analysis : analyzer : lowercase\_keyword : type : custom filter : [lowercase] tokenizer : keyword Had a little confusion, until I figured out that filter needed an array. Regards, Paul On Sep 23, 12:00 pm, Clinton Gormley [clin...@iannounce.co.uk](mailto:clin...@iannounce.co.uk) wrote: > On Thu, 2010-09-23 at 10:50 -0700, bailey wrote: > > > "(I feel like we would be in trouble though if we had to set it as > > "analyzed")." > > > should read > > > "(I feel like we would be in trouble though if we had to set it as > > "analyzed" and accepted any characters like dashes, underscores, > > etc...)." > > You will need to create a custom analyzer using the lowercase token > filter: > > [http://www.elasticsearch.com/docs/elasticsearch/index\_modules/analysis/http://www.elasticsearch.com/docs/elasticsearch/index\_modules/analysi](http://www.elasticsearch.com/docs/elasticsearch/index_modules/analysis/http://www.elasticsearch.com/docs/elasticsearch/index_modules/analysi)... > > clint > > -- > Web Announcements Limited is a company registered in England and Wales, > with company number 05608868, with registered address at 10 Arvon Road, > London, N5 1PR. ---
**Author:** ![bailey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bailey/32/3278_2.png) [@bailey](https://discuss.elastic.co/u/bailey)\ **Post date:** [September 24, 2010, 10:47pm UTC](https://discuss.elastic.co/t/case-in-sensitive-searching/3362/5 "2010-09-24T22:47:36Z")
Thanks Paul. If I was adding this to an Elasticsearch .properties file would the correct syntax be: index.analysis.analyzer.lowercase\_keyword.type=custom index.analysis.analyzer.lowercase\_keyword.filter=[lowercase] index.analysis.analyzer.lowercase\_keyword.tokenizer=keyword On Sep 23, 7:53 pm, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote: > FYI, here is the analyzer I created for this same case: > > index: > analysis : > analyzer : > lowercase\_keyword : > type : custom > filter : [lowercase] > tokenizer : keyword > > Had a little confusion, until I figured out that filter needed an > array. > > Regards, > Paul > > On Sep 23, 12:00 pm, Clinton Gormley [clin...@iannounce.co.uk](mailto:clin...@iannounce.co.uk) wrote: > > > On Thu, 2010-09-23 at 10:50 -0700, bailey wrote: > > > > > "(I feel like we would be in trouble though if we had to set it as > > > "analyzed")." > > > > should read > > > > "(I feel like we would be in trouble though if we had to set it as > > > "analyzed" and accepted any characters like dashes, underscores, > > > etc...)." > > > You will need to create a custom analyzer using the lowercase token > > filter: > > > [http://www.elasticsearch.com/docs/elasticsearch/index\_modules/analysi](http://www.elasticsearch.com/docs/elasticsearch/index_modules/analysi)...... > > > clint > > > -- > > Web Announcements Limited is a company registered in England and Wales, > > with company number 05608868, with registered address at 10 Arvon Road, > > London, N5 1PR. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [September 24, 2010, 10:57pm UTC](https://discuss.elastic.co/t/case-in-sensitive-searching/3362/6 "2010-09-24T22:57:54Z")
Properties files are a pain when it comes to array definitions, before I suggest the format, can you simply use yaml/json and load elasticsearch with it? On Sat, Sep 25, 2010 at 12:47 AM, bailey [matt.bailey@gmail.com](mailto:matt.bailey@gmail.com) wrote: > Thanks Paul. > If I was adding this to an Elasticsearch .properties file would the > correct syntax be: > > index.analysis.analyzer.lowercase\_keyword.type=custom > index.analysis.analyzer.lowercase\_keyword.filter=[lowercase] > index.analysis.analyzer.lowercase\_keyword.tokenizer=keyword > > On Sep 23, 7:53 pm, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote: > > > FYI, here is the analyzer I created for this same case: > > > > index: > > analysis : > > analyzer : > > lowercase\_keyword : > > type : custom > > filter : [lowercase] > > tokenizer : keyword > > > > Had a little confusion, until I figured out that filter needed an > > array. > > > > Regards, > > Paul > > > > On Sep 23, 12:00 pm, Clinton Gormley [clin...@iannounce.co.uk](mailto:clin...@iannounce.co.uk) wrote: > > > > > On Thu, 2010-09-23 at 10:50 -0700, bailey wrote: > > > > > > > "(I feel like we would be in trouble though if we had to set it as > > > > "analyzed")." > > > > > > should read > > > > > > "(I feel like we would be in trouble though if we had to set it as > > > > "analyzed" and accepted any characters like dashes, underscores, > > > > etc...)." > > > > > You will need to create a custom analyzer using the lowercase token > > > filter: > > > > > > > [http://www.elasticsearch.com/docs/elasticsearch/index\_modules/analysi](http://www.elasticsearch.com/docs/elasticsearch/index_modules/analysi).... > .. > > > > clint > > > > > -- > > > Web Announcements Limited is a company registered in England and Wales, > > > with company number 05608868, with registered address at 10 Arvon Road, > > > London, N5 1PR. ---
**Author:** ![James\_Cook](https://avatars.discourse-cdn.com/v4/letter/j/898d66/32.png) [@James\_Cook](https://discuss.elastic.co/u/James_Cook)\ **Post date:** [September 25, 2010, 4:53pm UTC](https://discuss.elastic.co/t/case-in-sensitive-searching/3362/7 "2010-09-25T16:53:36Z")
Hi shay, My ES instance is bootstrapped and configured via spring. I load the server properties from a properties file because that is what Spring supports. Is this array format you allude to able to be loaded using Java's property file support? On Sep 24, 2010 6:58 PM, "Shay Banon" [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) wrote: > Properties files are a pain when it comes to array definitions, before I > suggest the format, can you simply use yaml/json and load elasticsearch > with > it? > > On Sat, Sep 25, 2010 at 12:47 AM, bailey [matt.bailey@gmail.com](mailto:matt.bailey@gmail.com) wrote: > > > Thanks Paul. > > If I was adding this to an Elasticsearch .properties file would the > > correct syntax be: > > > > index.analysis.analyzer.lowercase\_keyword.type=custom > > index.analysis.analyzer.lowercase\_keyword.filter=[lowercase] > > index.analysis.analyzer.lowercase\_keyword.tokenizer=keyword > > > > On Sep 23, 7:53 pm, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote: > > > > > FYI, here is the analyzer I created for this same case: > > > > > > index: > > > analysis : > > > analyzer : > > > lowercase\_keyword : > > > type : custom > > > filter : [lowercase] > > > tokenizer : keyword > > > > > > Had a little confusion, until I figured out that filter needed an > > > array. > > > > > > Regards, > > > Paul > > > > > > On Sep 23, 12:00 pm, Clinton Gormley [clin...@iannounce.co.uk](mailto:clin...@iannounce.co.uk) wrote: > > > > > > > On Thu, 2010-09-23 at 10:50 -0700, bailey wrote: > > > > > > > > > "(I feel like we would be in trouble though if we had to set it as > > > > > "analyzed")." > > > > > > > > should read > > > > > > > > "(I feel like we would be in trouble though if we had to set it as > > > > > "analyzed" and accepted any characters like dashes, underscores, > > > > > etc...)." > > > > > > > You will need to create a custom analyzer using the lowercase token > > > > filter: > > > > > > > > > > > [http://www.elasticsearch.com/docs/elasticsearch/index\_modules/analysi](http://www.elasticsearch.com/docs/elasticsearch/index_modules/analysi).... > > .. > > > > > > clint > > > > > > > -- > > > > Web Announcements Limited is a company registered in England and > > > > Wales, > > > > with company number 05608868, with registered address at 10 Arvon > > > > Road, > > > > London, N5 1PR. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [September 25, 2010, 10:30pm UTC](https://discuss.elastic.co/t/case-in-sensitive-searching/3362/8 "2010-09-25T22:30:32Z")
Let me explain what I allude you are able to do (though not collude, which seems to be making twitter noise lately...): First, Spring does not imply using just properties files. Its your decision to use properties files as a configuration format or not, spring does not enforce you to use it. You can easily configure elasticsearch to load the settings from an external (classpath, or other) configuration file using json or yaml. Custom ${...} values will automatically be replaced with system properties. Check the SettingsLoader and different implementations. If you do want to use properties files, then suffixing the appropriate setting with .0, .1 and so on will imply its an array. Off, sorry, allude its an array. On Sat, Sep 25, 2010 at 6:53 PM, James Cook [jcook@tracermedia.com](mailto:jcook@tracermedia.com) wrote: > Hi shay, > My ES instance is bootstrapped and configured via spring. I load the server > properties from a properties file because that is what Spring supports. > > Is this array format you allude to able to be loaded using Java's property > file support? > > On Sep 24, 2010 6:58 PM, "Shay Banon" [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) > wrote: > > > Properties files are a pain when it comes to array definitions, before I > > suggest the format, can you simply use yaml/json and load elasticsearch > > with > > it? > > > > On Sat, Sep 25, 2010 at 12:47 AM, bailey [matt.bailey@gmail.com](mailto:matt.bailey@gmail.com) wrote: > > > > > Thanks Paul. > > > If I was adding this to an Elasticsearch .properties file would the > > > correct syntax be: > > > > > > index.analysis.analyzer.lowercase\_keyword.type=custom > > > index.analysis.analyzer.lowercase\_keyword.filter=[lowercase] > > > index.analysis.analyzer.lowercase\_keyword.tokenizer=keyword > > > > > > On Sep 23, 7:53 pm, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote: > > > > > > > FYI, here is the analyzer I created for this same case: > > > > > > > > index: > > > > analysis : > > > > analyzer : > > > > lowercase\_keyword : > > > > type : custom > > > > filter : [lowercase] > > > > tokenizer : keyword > > > > > > > > Had a little confusion, until I figured out that filter needed an > > > > array. > > > > > > > > Regards, > > > > Paul > > > > > > > > On Sep 23, 12:00 pm, Clinton Gormley [clin...@iannounce.co.uk](mailto:clin...@iannounce.co.uk) wrote: > > > > > > > > > On Thu, 2010-09-23 at 10:50 -0700, bailey wrote: > > > > > > > > > > > "(I feel like we would be in trouble though if we had to set it as > > > > > > "analyzed")." > > > > > > > > > > should read > > > > > > > > > > "(I feel like we would be in trouble though if we had to set it as > > > > > > "analyzed" and accepted any characters like dashes, underscores, > > > > > > etc...)." > > > > > > > > > You will need to create a custom analyzer using the lowercase token > > > > > filter: > > > > > > > > > > > > > > > [http://www.elasticsearch.com/docs/elasticsearch/index\_modules/analysi](http://www.elasticsearch.com/docs/elasticsearch/index_modules/analysi).. > > > .. > > > .. > > > > > > > > clint > > > > > > > > > -- > > > > > Web Announcements Limited is a company registered in England and > > > > > Wales, > > > > > with company number 05608868, with registered address at 10 Arvon > > > > > Road, > > > > > London, N5 1PR. ---
**Author:** ![James\_Cook](https://avatars.discourse-cdn.com/v4/letter/j/898d66/32.png) [@James\_Cook](https://discuss.elastic.co/u/James_Cook)\ **Post date:** [September 25, 2010, 10:49pm UTC](https://discuss.elastic.co/t/case-in-sensitive-searching/3362/9 "2010-09-25T22:49:27Z")
Thanks for sharing the arrays format. On Sat, Sep 25, 2010 at 6:30 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > Let me explain what I allude you are able to do (though not collude, which > seems to be making twitter noise lately...): > > First, Spring does not imply using just properties files. Its your decision > to use properties files as a configuration format or not, spring does not > enforce you to use it. > > You can easily configure elasticsearch to load the settings from an > external (classpath, or other) configuration file using json or yaml. Custom > ${...} values will automatically be replaced with system properties. Check > the SettingsLoader and different implementations. > > If you do want to use properties files, then suffixing > the appropriate setting with .0, .1 and so on will imply its an array. Off, > sorry, allude its an array. > > On Sat, Sep 25, 2010 at 6:53 PM, James Cook [jcook@tracermedia.com](mailto:jcook@tracermedia.com) wrote: > > > Hi shay, > > My ES instance is bootstrapped and configured via spring. I load the > > server properties from a properties file because that is what Spring > > supports. > > > > Is this array format you allude to able to be loaded using Java's property > > file support? > > > > On Sep 24, 2010 6:58 PM, "Shay Banon" [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) > > wrote: > > > > > Properties files are a pain when it comes to array definitions, before I > > > suggest the format, can you simply use yaml/json and load elasticsearch > > > with > > > it? > > > > > > On Sat, Sep 25, 2010 at 12:47 AM, bailey [matt.bailey@gmail.com](mailto:matt.bailey@gmail.com) wrote: > > > > > > > Thanks Paul. > > > > If I was adding this to an Elasticsearch .properties file would the > > > > correct syntax be: > > > > > > > > index.analysis.analyzer.lowercase\_keyword.type=custom > > > > index.analysis.analyzer.lowercase\_keyword.filter=[lowercase] > > > > index.analysis.analyzer.lowercase\_keyword.tokenizer=keyword > > > > > > > > On Sep 23, 7:53 pm, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote: > > > > > > > > > FYI, here is the analyzer I created for this same case: > > > > > > > > > > index: > > > > > analysis : > > > > > analyzer : > > > > > lowercase\_keyword : > > > > > type : custom > > > > > filter : [lowercase] > > > > > tokenizer : keyword > > > > > > > > > > Had a little confusion, until I figured out that filter needed an > > > > > array. > > > > > > > > > > Regards, > > > > > Paul > > > > > > > > > > On Sep 23, 12:00 pm, Clinton Gormley [clin...@iannounce.co.uk](mailto:clin...@iannounce.co.uk) > > > > > wrote: > > > > > > > > > > > On Thu, 2010-09-23 at 10:50 -0700, bailey wrote: > > > > > > > > > > > > > "(I feel like we would be in trouble though if we had to set it > > > > > > > as > > > > > > > "analyzed")." > > > > > > > > > > > > should read > > > > > > > > > > > > "(I feel like we would be in trouble though if we had to set it > > > > > > > as > > > > > > > "analyzed" and accepted any characters like dashes, underscores, > > > > > > > etc...)." > > > > > > > > > > > You will need to create a custom analyzer using the lowercase token > > > > > > filter: > > > > > > > > > > > > > > > [http://www.elasticsearch.com/docs/elasticsearch/index\_modules/analysi](http://www.elasticsearch.com/docs/elasticsearch/index_modules/analysi).... > > > > > > .. > > > > > > > > > > clint > > > > > > > > > > > -- > > > > > > Web Announcements Limited is a company registered in England and > > > > > > Wales, > > > > > > with company number 05608868, with registered address at 10 Arvon > > > > > > Road, > > > > > > London, N5 1PR. ---
**Author:** ![bailey](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bailey/32/3278_2.png) [@bailey](https://discuss.elastic.co/u/bailey)\ **Post date:** [September 26, 2010, 3:44pm UTC](https://discuss.elastic.co/t/case-in-sensitive-searching/3362/10 "2010-09-26T15:44:04Z")
Thanks for the input guys. I changed the .properties file to: index.analysis.analyzer.lowercase\_keyword.type=custom index.analysis.analyzer.lowercase\_keyword.filter.0=lowercase index.analysis.analyzer.lowercase\_keyword.tokenizer=keyword But I'm still getting some errors (i.e. "Analyzer [lowercase\_keyword] not found") when I associate a file to "analyzer":"lowercase\_keyword" in the mapping. Anything I'm missing here? On Sep 25, 6:49 pm, James Cook [jc...@tracermedia.com](mailto:jc...@tracermedia.com) wrote: > Thanks for sharing the arrays format. > > On Sat, Sep 25, 2010 at 6:30 PM, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com)wrote: > > > Let me explain what I allude you are able to do (though not collude, which > > seems to be making twitter noise lately...): > > > First, Spring does not imply using just properties files. Its your decision > > to use properties files as a configuration format or not, spring does not > > enforce you to use it. > > > You can easily configure elasticsearch to load the settings from an > > external (classpath, or other) configuration file using json or yaml. Custom > > ${...} values will automatically be replaced with system properties. Check > > the SettingsLoader and different implementations. > > > If you do want to use properties files, then suffixing > > the appropriate setting with .0, .1 and so on will imply its an array. Off, > > sorry, allude its an array. > > > On Sat, Sep 25, 2010 at 6:53 PM, James Cook [jc...@tracermedia.com](mailto:jc...@tracermedia.com) wrote: > > > > Hi shay, > > > My ES instance is bootstrapped and configured via spring. I load the > > > server properties from a properties file because that is what Spring > > > supports. > > > > Is this array format you allude to able to be loaded using Java's property > > > file support? > > > > On Sep 24, 2010 6:58 PM, "Shay Banon" [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) > > > wrote: > > > > > > > Properties files are a pain when it comes to array definitions, before I > > > > suggest the format, can you simply use yaml/json and load elasticsearch > > > > with > > > > it? > > > > > On Sat, Sep 25, 2010 at 12:47 AM, bailey [matt.bai...@gmail.com](mailto:matt.bai...@gmail.com) wrote: > > > > > > Thanks Paul. > > > > > If I was adding this to an Elasticsearch .properties file would the > > > > > correct syntax be: > > > > > > index.analysis.analyzer.lowercase\_keyword.type=custom > > > > > index.analysis.analyzer.lowercase\_keyword.filter=[lowercase] > > > > > index.analysis.analyzer.lowercase\_keyword.tokenizer=keyword > > > > > > On Sep 23, 7:53 pm, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote: > > > > > > > > > > > FYI, here is the analyzer I created for this same case: > > > > > > > index: > > > > > > analysis : > > > > > > analyzer : > > > > > > lowercase\_keyword : > > > > > > type : custom > > > > > > filter : [lowercase] > > > > > > tokenizer : keyword > > > > > > > Had a little confusion, until I figured out that filter needed an > > > > > > array. > > > > > > > Regards, > > > > > > Paul > > > > > > > On Sep 23, 12:00 pm, Clinton Gormley [clin...@iannounce.co.uk](mailto:clin...@iannounce.co.uk) > > > > > > wrote: > > > > > > > > On Thu, 2010-09-23 at 10:50 -0700, bailey wrote: > > > > > > > > > > > > > > > "(I feel like we would be in trouble though if we had to set it > > > > > > > > as > > > > > > > > "analyzed")." > > > > > > > > > should read > > > > > > > > > "(I feel like we would be in trouble though if we had to set it > > > > > > > > as > > > > > > > > "analyzed" and accepted any characters like dashes, underscores, > > > > > > > > etc...)." > > > > > > > > You will need to create a custom analyzer using the lowercase token > > > > > > > filter: > > > > [http://www.elasticsearch.com/docs/elasticsearch/index\_modules/analysi](http://www.elasticsearch.com/docs/elasticsearch/index_modules/analysi).... > > > > > > > > .. > > > > > > > > clint > > > > > > > > -- > > > > > > > Web Announcements Limited is a company registered in England and > > > > > > > Wales, > > > > > > > with company number 05608868, with registered address at 10 Arvon > > > > > > > Road, > > > > > > > London, N5 1PR. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [September 26, 2010, 8:47pm UTC](https://discuss.elastic.co/t/case-in-sensitive-searching/3362/11 "2010-09-26T20:47:24Z")
This settings seem ok. Can you recreate this with fresh elasticsearch install, rename elasticsearch.yml to elasticsearch.properties, place the relevant properties and gist a curl recreation? On Sun, Sep 26, 2010 at 5:44 PM, bailey [matt.bailey@gmail.com](mailto:matt.bailey@gmail.com) wrote: > Thanks for the input guys. > > I changed the .properties file to: > > index.analysis.analyzer.lowercase\_keyword.type=custom > index.analysis.analyzer.lowercase\_keyword.filter.0=lowercase > index.analysis.analyzer.lowercase\_keyword.tokenizer=keyword > > But I'm still getting some errors (i.e. "Analyzer [lowercase\_keyword] > not found") when I associate a file to "analyzer":"lowercase\_keyword" > in the mapping. > Anything I'm missing here? > > On Sep 25, 6:49 pm, James Cook [jc...@tracermedia.com](mailto:jc...@tracermedia.com) wrote: > > > Thanks for sharing the arrays format. > > > > On Sat, Sep 25, 2010 at 6:30 PM, Shay Banon \< > > [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com)\>wrote: > > > > > Let me explain what I allude you are able to do (though not collude, > > > which > > > seems to be making twitter noise lately...): > > > > > First, Spring does not imply using just properties files. Its your > > > decision > > > to use properties files as a configuration format or not, spring does > > > not > > > enforce you to use it. > > > > > You can easily configure elasticsearch to load the settings from an > > > external (classpath, or other) configuration file using json or yaml. > > > Custom > > > ${...} values will automatically be replaced with system properties. > > > Check > > > the SettingsLoader and different implementations. > > > > > If you do want to use properties files, then suffixing > > > the appropriate setting with .0, .1 and so on will imply its an array. > > > Off, > > > sorry, allude its an array. > > > > > On Sat, Sep 25, 2010 at 6:53 PM, James Cook [jc...@tracermedia.com](mailto:jc...@tracermedia.com) > > > wrote: > > > > > > Hi shay, > > > > My ES instance is bootstrapped and configured via spring. I load the > > > > server properties from a properties file because that is what Spring > > > > supports. > > > > > > Is this array format you allude to able to be loaded using Java's > > > > property > > > > file support? > > > > > > On Sep 24, 2010 6:58 PM, "Shay Banon" [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) > > > > wrote: > > > > > > > > > Properties files are a pain when it comes to array definitions, > > > > > before I > > > > > suggest the format, can you simply use yaml/json and load > > > > > elasticsearch > > > > > with > > > > > it? > > > > > > > On Sat, Sep 25, 2010 at 12:47 AM, bailey [matt.bai...@gmail.com](mailto:matt.bai...@gmail.com) > > > > > wrote: > > > > > > > > Thanks Paul. > > > > > > If I was adding this to an Elasticsearch .properties file would the > > > > > > correct syntax be: > > > > > > > > index.analysis.analyzer.lowercase\_keyword.type=custom > > > > > > index.analysis.analyzer.lowercase\_keyword.filter=[lowercase] > > > > > > index.analysis.analyzer.lowercase\_keyword.tokenizer=keyword > > > > > > > > On Sep 23, 7:53 pm, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote: > > > > > > > > > > > > > FYI, here is the analyzer I created for this same case: > > > > > > > > > index: > > > > > > > analysis : > > > > > > > analyzer : > > > > > > > lowercase\_keyword : > > > > > > > type : custom > > > > > > > filter : [lowercase] > > > > > > > tokenizer : keyword > > > > > > > > > Had a little confusion, until I figured out that filter needed an > > > > > > > array. > > > > > > > > > Regards, > > > > > > > Paul > > > > > > > > > On Sep 23, 12:00 pm, Clinton Gormley [clin...@iannounce.co.uk](mailto:clin...@iannounce.co.uk) > > > > > > > wrote: > > > > > > > > > > On Thu, 2010-09-23 at 10:50 -0700, bailey wrote: > > > > > > > > > > > > > > > > > "(I feel like we would be in trouble though if we had to set > > > > > > > > > it > > > > > > > > > as > > > > > > > > > "analyzed")." > > > > > > > > > > > should read > > > > > > > > > > > "(I feel like we would be in trouble though if we had to set > > > > > > > > > it > > > > > > > > > as > > > > > > > > > "analyzed" and accepted any characters like dashes, > > > > > > > > > underscores, > > > > > > > > > etc...)." > > > > > > > > > > You will need to create a custom analyzer using the lowercase > > > > > > > > token > > > > > > > > filter: > > > > > > > > [http://www.elasticsearch.com/docs/elasticsearch/index\_modules/analysi](http://www.elasticsearch.com/docs/elasticsearch/index_modules/analysi).... > > > > > > > .. > > > > > > > > > > clint > > > > > > > > > > -- > > > > > > > > Web Announcements Limited is a company registered in England > > > > > > > > and > > > > > > > > Wales, > > > > > > > > with company number 05608868, with registered address at 10 > > > > > > > > Arvon > > > > > > > > Road, > > > > > > > > London, N5 1PR. ---
**Author:** ![James\_Cook](https://avatars.discourse-cdn.com/v4/letter/j/898d66/32.png) [@James\_Cook](https://discuss.elastic.co/u/James_Cook)\ **Post date:** [September 27, 2010, 4:27pm UTC](https://discuss.elastic.co/t/case-in-sensitive-searching/3362/12 "2010-09-27T16:27:03Z")
The error was caused because we added the custom analyzer config to our ES server config only. The ES clients did not have this information and were the cause of the exception. On Sun, Sep 26, 2010 at 4:47 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > This settings seem ok. Can you recreate this with fresh elasticsearch > install, rename elasticsearch.yml to elasticsearch.properties, place the > relevant properties and gist a curl recreation? > > On Sun, Sep 26, 2010 at 5:44 PM, bailey [matt.bailey@gmail.com](mailto:matt.bailey@gmail.com) wrote: > > > Thanks for the input guys. > > > > I changed the .properties file to: > > > > index.analysis.analyzer.lowercase\_keyword.type=custom > > index.analysis.analyzer.lowercase\_keyword.filter.0=lowercase > > index.analysis.analyzer.lowercase\_keyword.tokenizer=keyword > > > > But I'm still getting some errors (i.e. "Analyzer [lowercase\_keyword] > > not found") when I associate a file to "analyzer":"lowercase\_keyword" > > in the mapping. > > Anything I'm missing here? > > > > On Sep 25, 6:49 pm, James Cook [jc...@tracermedia.com](mailto:jc...@tracermedia.com) wrote: > > > > > Thanks for sharing the arrays format. > > > > > > On Sat, Sep 25, 2010 at 6:30 PM, Shay Banon \< > > > [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com)\>wrote: > > > > > > > Let me explain what I allude you are able to do (though not collude, > > > > which > > > > seems to be making twitter noise lately...): > > > > > > > First, Spring does not imply using just properties files. Its your > > > > decision > > > > to use properties files as a configuration format or not, spring does > > > > not > > > > enforce you to use it. > > > > > > > You can easily configure elasticsearch to load the settings from an > > > > external (classpath, or other) configuration file using json or yaml. > > > > Custom > > > > ${...} values will automatically be replaced with system properties. > > > > Check > > > > the SettingsLoader and different implementations. > > > > > > > If you do want to use properties files, then suffixing > > > > the appropriate setting with .0, .1 and so on will imply its an array. > > > > Off, > > > > sorry, allude its an array. > > > > > > > On Sat, Sep 25, 2010 at 6:53 PM, James Cook [jc...@tracermedia.com](mailto:jc...@tracermedia.com) > > > > wrote: > > > > > > > > Hi shay, > > > > > My ES instance is bootstrapped and configured via spring. I load the > > > > > server properties from a properties file because that is what Spring > > > > > supports. > > > > > > > > Is this array format you allude to able to be loaded using Java's > > > > > property > > > > > file support? > > > > > > > > On Sep 24, 2010 6:58 PM, "Shay Banon" [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) > > > > > wrote: > > > > > > > > > > > Properties files are a pain when it comes to array definitions, > > > > > > before I > > > > > > suggest the format, can you simply use yaml/json and load > > > > > > elasticsearch > > > > > > with > > > > > > it? > > > > > > > > > On Sat, Sep 25, 2010 at 12:47 AM, bailey [matt.bai...@gmail.com](mailto:matt.bai...@gmail.com) > > > > > > wrote: > > > > > > > > > > Thanks Paul. > > > > > > > If I was adding this to an Elasticsearch .properties file would > > > > > > > the > > > > > > > correct syntax be: > > > > > > > > > > index.analysis.analyzer.lowercase\_keyword.type=custom > > > > > > > index.analysis.analyzer.lowercase\_keyword.filter=[lowercase] > > > > > > > index.analysis.analyzer.lowercase\_keyword.tokenizer=keyword > > > > > > > > > > On Sep 23, 7:53 pm, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote: > > > > > > > > > > > > > > > FYI, here is the analyzer I created for this same case: > > > > > > > > > > > index: > > > > > > > > analysis : > > > > > > > > analyzer : > > > > > > > > lowercase\_keyword : > > > > > > > > type : custom > > > > > > > > filter : [lowercase] > > > > > > > > tokenizer : keyword > > > > > > > > > > > Had a little confusion, until I figured out that filter needed > > > > > > > > an > > > > > > > > array. > > > > > > > > > > > Regards, > > > > > > > > Paul > > > > > > > > > > > On Sep 23, 12:00 pm, Clinton Gormley [clin...@iannounce.co.uk](mailto:clin...@iannounce.co.uk) > > > > > > > > wrote: > > > > > > > > > > > > On Thu, 2010-09-23 at 10:50 -0700, bailey wrote: > > > > > > > > > > > > > > > > > > > "(I feel like we would be in trouble though if we had to set > > > > > > > > > > it > > > > > > > > > > as > > > > > > > > > > "analyzed")." > > > > > > > > > > > > > should read > > > > > > > > > > > > > "(I feel like we would be in trouble though if we had to set > > > > > > > > > > it > > > > > > > > > > as > > > > > > > > > > "analyzed" and accepted any characters like dashes, > > > > > > > > > > underscores, > > > > > > > > > > etc...)." > > > > > > > > > > > > You will need to create a custom analyzer using the lowercase > > > > > > > > > token > > > > > > > > > filter: > > > > > > > > > > > > [http://www.elasticsearch.com/docs/elasticsearch/index\_modules/analysi](http://www.elasticsearch.com/docs/elasticsearch/index_modules/analysi).... > > > > > > > > > .. > > > > > > > > > > > > clint > > > > > > > > > > > > -- > > > > > > > > > Web Announcements Limited is a company registered in England > > > > > > > > > and > > > > > > > > > Wales, > > > > > > > > > with company number 05608868, with registered address at 10 > > > > > > > > > Arvon > > > > > > > > > Road, > > > > > > > > > London, N5 1PR. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [September 27, 2010, 5:18pm UTC](https://discuss.elastic.co/t/case-in-sensitive-searching/3362/13 "2010-09-27T17:18:26Z")
You can always provide the analyzes as part of the index configuration when you create an index (And not part of the node configuration), then it will propagate through the nodes. -shay.banon On Mon, Sep 27, 2010 at 6:27 PM, James Cook [jcook@tracermedia.com](mailto:jcook@tracermedia.com) wrote: > The error was caused because we added the custom analyzer config to our ES > server config only. The ES clients did not have this information and were > the cause of the exception. > > On Sun, Sep 26, 2010 at 4:47 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote: > > > This settings seem ok. Can you recreate this with fresh elasticsearch > > install, rename elasticsearch.yml to elasticsearch.properties, place the > > relevant properties and gist a curl recreation? > > > > On Sun, Sep 26, 2010 at 5:44 PM, bailey [matt.bailey@gmail.com](mailto:matt.bailey@gmail.com) wrote: > > > > > Thanks for the input guys. > > > > > > I changed the .properties file to: > > > > > > index.analysis.analyzer.lowercase\_keyword.type=custom > > > index.analysis.analyzer.lowercase\_keyword.filter.0=lowercase > > > index.analysis.analyzer.lowercase\_keyword.tokenizer=keyword > > > > > > But I'm still getting some errors (i.e. "Analyzer [lowercase\_keyword] > > > not found") when I associate a file to "analyzer":"lowercase\_keyword" > > > in the mapping. > > > Anything I'm missing here? > > > > > > On Sep 25, 6:49 pm, James Cook [jc...@tracermedia.com](mailto:jc...@tracermedia.com) wrote: > > > > > > > Thanks for sharing the arrays format. > > > > > > > > On Sat, Sep 25, 2010 at 6:30 PM, Shay Banon \< > > > > [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com)\>wrote: > > > > > > > > > Let me explain what I allude you are able to do (though not collude, > > > > > which > > > > > seems to be making twitter noise lately...): > > > > > > > > > First, Spring does not imply using just properties files. Its your > > > > > decision > > > > > to use properties files as a configuration format or not, spring does > > > > > not > > > > > enforce you to use it. > > > > > > > > > You can easily configure elasticsearch to load the settings from an > > > > > external (classpath, or other) configuration file using json or yaml. > > > > > Custom > > > > > ${...} values will automatically be replaced with system properties. > > > > > Check > > > > > the SettingsLoader and different implementations. > > > > > > > > > If you do want to use properties files, then suffixing > > > > > the appropriate setting with .0, .1 and so on will imply its an > > > > > array. Off, > > > > > sorry, allude its an array. > > > > > > > > > On Sat, Sep 25, 2010 at 6:53 PM, James Cook [jc...@tracermedia.com](mailto:jc...@tracermedia.com) > > > > > wrote: > > > > > > > > > > Hi shay, > > > > > > My ES instance is bootstrapped and configured via spring. I load the > > > > > > server properties from a properties file because that is what Spring > > > > > > supports. > > > > > > > > > > Is this array format you allude to able to be loaded using Java's > > > > > > property > > > > > > file support? > > > > > > > > > > On Sep 24, 2010 6:58 PM, "Shay Banon" \<[shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) > > > > > > > > > > wrote: > > > > > > > > > > > > > Properties files are a pain when it comes to array definitions, > > > > > > > before I > > > > > > > suggest the format, can you simply use yaml/json and load > > > > > > > elasticsearch > > > > > > > with > > > > > > > it? > > > > > > > > > > > On Sat, Sep 25, 2010 at 12:47 AM, bailey [matt.bai...@gmail.com](mailto:matt.bai...@gmail.com) > > > > > > > wrote: > > > > > > > > > > > > Thanks Paul. > > > > > > > > If I was adding this to an Elasticsearch .properties file would > > > > > > > > the > > > > > > > > correct syntax be: > > > > > > > > > > > > index.analysis.analyzer.lowercase\_keyword.type=custom > > > > > > > > index.analysis.analyzer.lowercase\_keyword.filter=[lowercase] > > > > > > > > index.analysis.analyzer.lowercase\_keyword.tokenizer=keyword > > > > > > > > > > > > On Sep 23, 7:53 pm, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote: > > > > > > > > > > > > > > > > > FYI, here is the analyzer I created for this same case: > > > > > > > > > > > > > index: > > > > > > > > > analysis : > > > > > > > > > analyzer : > > > > > > > > > lowercase\_keyword : > > > > > > > > > type : custom > > > > > > > > > filter : [lowercase] > > > > > > > > > tokenizer : keyword > > > > > > > > > > > > > Had a little confusion, until I figured out that filter needed > > > > > > > > > an > > > > > > > > > array. > > > > > > > > > > > > > Regards, > > > > > > > > > Paul > > > > > > > > > > > > > On Sep 23, 12:00 pm, Clinton Gormley [clin...@iannounce.co.uk](mailto:clin...@iannounce.co.uk) > > > > > > > > > wrote: > > > > > > > > > > > > > > On Thu, 2010-09-23 at 10:50 -0700, bailey wrote: > > > > > > > > > > > > > > > > > > > > > "(I feel like we would be in trouble though if we had to > > > > > > > > > > > set it > > > > > > > > > > > as > > > > > > > > > > > "analyzed")." > > > > > > > > > > > > > > > should read > > > > > > > > > > > > > > > "(I feel like we would be in trouble though if we had to > > > > > > > > > > > set it > > > > > > > > > > > as > > > > > > > > > > > "analyzed" and accepted any characters like dashes, > > > > > > > > > > > underscores, > > > > > > > > > > > etc...)." > > > > > > > > > > > > > > You will need to create a custom analyzer using the lowercase > > > > > > > > > > token > > > > > > > > > > filter: > > > > > > > > > > > > > > > > [http://www.elasticsearch.com/docs/elasticsearch/index\_modules/analysi](http://www.elasticsearch.com/docs/elasticsearch/index_modules/analysi).. > > > .. > > > > > > > > > > > .. > > > > > > > > > > > > > > clint > > > > > > > > > > > > > > -- > > > > > > > > > > Web Announcements Limited is a company registered in England > > > > > > > > > > and > > > > > > > > > > Wales, > > > > > > > > > > with company number 05608868, with registered address at 10 > > > > > > > > > > Arvon > > > > > > > > > > Road, > > > > > > > > > > London, N5 1PR. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:18am UTC](https://discuss.elastic.co/t/case-in-sensitive-searching/3362/14 "2017-07-06T04:18:53Z")