# Analyzers and Search **URL:** \ **Category:** Elasticsearch\ **Created:** [April 9, 2013, 6:45pm UTC](https://discuss.elastic.co/t/analyzers-and-search/11516 "2013-04-09T18:45:38Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![avin](https://avatars.discourse-cdn.com/v4/letter/a/ee7513/32.png) [@avin](https://discuss.elastic.co/u/avin)\ **Post date:** [April 9, 2013, 6:45pm UTC](https://discuss.elastic.co/t/analyzers-and-search/11516/1 "2013-04-09T18:45:38Z")
I use ES to store key value pairs. I haven't defined the mappings while creating the index. UpdateRequestBuilder up = elasticClient.prepareUpdate(indexName, indexType, key). ``` setConsistencyLevel(WriteConsistencyLevel.*DEFAULT* ``` ).setReplicationType(ReplicationType._ASYNC_) ``` .setDoc(map) .setUpsert(map); ``` Here map is my set of K-V pairs. One of the keys that I am storing is email and sometimes I also store values like URL and product id's which include non-alphabets. Now I dont use an analyzer and haven't defined my mappings so I assume the standard analyzer is used. So all my values are lower-cases and tokenized before indexing. Here is what is stored in the index: {"took":27,"timed\_out":false,"\_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":1,"max\_score":1.0,"hits":[{"\_index":"test","\_type":"test1","\_id":"097d4c2c-8c34-49bb-ac4b-977a5ea8abbf","\_score":1.0, "\_source" : {"ATID":"38318394","WTY":"Stocks","CREATED":"2013-04-09T18:30:52.845Z","CTN":"503-703-1115","EMAIL":"abcd.efgh@gmail.com","PRODUCT":"116/56L/A"}}]}} Now I find that I am not able to search for emails e.g. ([abcd.efgh@gmail.com](mailto:abcd.efgh@gmail.com)). I get false positives for "@gmail.com" , "abcd.efgh" , even [abcd@gmail.com](mailto:abcd@gmail.com). Is there a way I can use an analyzer just in search without changing my mapping definition. Scores dont matter to me as well so Filters are fine with me as well Query String:{ "bool" : { ``` "must" : [ { "match" : { "EMAIL" : { "query" : "abcd.efgh@gmail.com", "type" : "boolean" } } }, { "range" : { "CREATED" : { "from" : "2013-04-08", "to" : "2013-04-18", "include_lower" : true, "include_upper" : true } } } ] ``` } } Hits:1 { "bool" : { ``` "must" : [ { "match" : { "EMAIL" : { "query" : "abcd.efgh", "type" : "boolean" } } }, { "range" : { "CREATED" : { "from" : "2013-04-08", "to" : "2013-04-18", "include_lower" : true, "include_upper" : true } } } ] ``` } } Hits:1 Query String:{ "bool" : { ``` "must" : [ { "match" : { "EMAIL" : { "query" : "@gmail.com", "type" : "boolean" } } }, { "range" : { "CREATED" : { "from" : "2013-04-08", "to" : "2013-04-18", "include_lower" : true, "include_upper" : true } } } ] ``` } } Hits:1 { "bool" : { ``` "must" : [ { "match" : { "EMAIL" : { "query" : "abcd@gmail.com", "type" : "boolean" } } }, { "range" : { "CREATED" : { "from" : "2013-04-08", "to" : "2013-04-18", "include_lower" : true, "include_upper" : true } } } ] ``` } } Hits:1 -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [April 10, 2013, 7:34am UTC](https://discuss.elastic.co/t/analyzers-and-search/11516/2 "2013-04-10T07:34:18Z")
Hey the standard analyzer breaks email adresses and urls apart, you can find out by using the analyze API » curl -XGET 'localhost:9200/\_analyze?pretty&analyzer=standard' -d ' [foo@bar.de](mailto:foo@bar.de)' { "tokens" : [ { "token" : "foo", "start\_offset" : 0, "end\_offset" : 3, "type" : "", "position" : 1 }, { "token" : "[bar.de](http://bar.de)", "start\_offset" : 4, "end\_offset" : 10, "type" : "", "position" : 2 } ] } If you use the uax url email tokenizier, this does not happen, and emails/urls are indexed as one token » curl -XGET 'localhost:9200/\_analyze?pretty&tokenizer=uax\_url\_email' -d ' [foo@bar.de](mailto:foo@bar.de)' { "tokens" : [ { "token" : "[foo@bar.de](mailto:foo@bar.de)", "start\_offset" : 0, "end\_offset" : 10, "type" : "", "position" : 1 } ] } So use that tokenizer in your mapping in order to not split urls and emails and everything should work as you expect. More at > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. Hope this helps... --Alexander On Tue, Apr 9, 2013 at 8:45 PM, [avinsemail@gmail.com](mailto:avinsemail@gmail.com) wrote: > I use ES to store key value pairs. I haven't defined the mappings while > creating the index. > > UpdateRequestBuilder up = elasticClient.prepareUpdate(indexName, > indexType, key). > > ``` > setConsistencyLevel(WriteConsistencyLevel.*DEFAULT* > > ``` > > ).setReplicationType(ReplicationType._ASYNC_) > > ``` > .setDoc(map) > > .setUpsert(map); > > ``` > > Here map is my set of K-V pairs. > > One of the keys that I am storing is email and sometimes I also store > values like URL and product id's which include non-alphabets. Now I dont > use an analyzer and haven't defined my mappings so I assume the standard > analyzer is used. So all my values are lower-cases and tokenized before > indexing. > > Here is what is stored in the index: > > {"took":27,"timed\_out":false,"\_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":1,"max\_score":1.0,"hits":[{"\_index":"test","\_type":"test1","\_id":"097d4c2c-8c34-49bb-ac4b-977a5ea8abbf","\_score":1.0, "\_source" : {"[ATID":"38318394","WTY":"Stocks","CREATED":"2013-04-09T18:30:52.845Z","CTN":"503-703-1115","EMAIL":"abcd.efgh@gmail.com](mailto:ATID%22:%2238318394%22,%22WTY%22:%22Stocks%22,%22CREATED%22:%222013-04-09T18:30:52.845Z%22,%22CTN%22:%22503-703-1115%22,%22EMAIL%22:%22abcd.efgh@gmail.com)","PRODUCT":"116/56L/A"}}]}} > > Now I find that I am not able to search for emails e.g. ( > [abcd.efgh@gmail.com](mailto:abcd.efgh@gmail.com)). I get false positives for "@gmail.com" , > "abcd.efgh" , even [abcd@gmail.com](mailto:abcd@gmail.com). Is there a way I can use an analyzer > just in search without changing my mapping definition. Scores dont matter > to me as well so Filters are fine with me as well > > Query String:{ > > "bool" : { > > ``` > "must" : [ { > > "match" : { > > "EMAIL" : { > > "query" : "abcd.efgh@gmail.com", > > "type" : "boolean" > > } > > } > > }, { > > "range" : { > > "CREATED" : { > > "from" : "2013-04-08", > > "to" : "2013-04-18", > > "include_lower" : true, > > "include_upper" : true > > } > > } > > } ] > > ``` > > } > > } > > Hits:1 > > { > > "bool" : { > > ``` > "must" : [ { > > "match" : { > > "EMAIL" : { > > "query" : "abcd.efgh", > > "type" : "boolean" > > } > > } > > }, { > > "range" : { > > "CREATED" : { > > "from" : "2013-04-08", > > "to" : "2013-04-18", > > "include_lower" : true, > > "include_upper" : true > > } > > } > > } ] > > ``` > > } > > } > > Hits:1 > > Query String:{ > > "bool" : { > > ``` > "must" : [ { > > "match" : { > > "EMAIL" : { > > "query" : "@gmail.com", > > "type" : "boolean" > > } > > } > > }, { > > "range" : { > > "CREATED" : { > > "from" : "2013-04-08", > > "to" : "2013-04-18", > > "include_lower" : true, > > "include_upper" : true > > } > > } > > } ] > > ``` > > } > > } > > Hits:1 > > { > > "bool" : { > > ``` > "must" : [ { > > "match" : { > > "EMAIL" : { > > "query" : "abcd@gmail.com", > > "type" : "boolean" > > } > > } > > }, { > > "range" : { > > "CREATED" : { > > "from" : "2013-04-08", > > "to" : "2013-04-18", > > "include_lower" : true, > > "include_upper" : true > > } > > } > > } ] > > ``` > > } > > } > > Hits:1 > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:41am UTC](https://discuss.elastic.co/t/analyzers-and-search/11516/3 "2017-07-06T02:41:52Z")