# Uax\_url\_email tokenizer **URL:** \ **Category:** Elasticsearch\ **Created:** [November 6, 2012, 1:18pm UTC](https://discuss.elastic.co/t/uax-url-email-tokenizer/9581 "2012-11-06T13:18:31Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![smhdiu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/smhdiu/32/1808_2.png) [@smhdiu](https://discuss.elastic.co/u/smhdiu)\ **Post date:** [November 6, 2012, 1:18pm UTC](https://discuss.elastic.co/t/uax-url-email-tokenizer/9581/1 "2012-11-06T13:18:31Z")
Hello can we do uax\_url\_email tokenizer in java api ? if so then how to perform. please suggest. Thanks in advanced Mohsin H -- ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [November 6, 2012, 11:22pm UTC](https://discuss.elastic.co/t/uax-url-email-tokenizer/9581/2 "2012-11-06T23:22:41Z")
The easiest way to to create a custom analyzer that utilizes the uxl\_url\_email tokenizer and map that analyzer to a field. When that field is indexed/searched, the analyzer would be used. -- Ivan On Tue, Nov 6, 2012 at 5:18 AM, mohsin husen [smhdiu@gmail.com](mailto:smhdiu@gmail.com) wrote: > Hello > > can we do uax\_url\_email tokenizer in java api ? > if so then how to perform. > please suggest. > > Thanks in advanced > Mohsin H > > -- -- ---
**Author:** ![smhdiu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/smhdiu/32/1808_2.png) [@smhdiu](https://discuss.elastic.co/u/smhdiu)\ **Post date:** [November 7, 2012, 10:15am UTC](https://discuss.elastic.co/t/uax-url-email-tokenizer/9581/3 "2012-11-07T10:15:34Z")
Thanks Ivan for your reply. i have following java code for fetching the data from elasticsearch SearchRequestBuilder searchRequestBuilder = client.prepareSearch(indexName); searchRequestBuilder.setSearchType(SearchType.DEFAULT); searchRequestBuilder.setQuery(boolQuery() .should(textQuery("email", email)) .should(textQuery("firstName", firstName)) .should(textQuery("lastName", lastName)) .should(textQuery("keyword", keyword))); searchRequestBuilder.setFilter(FilterBuilders.andFilter().add(FilterBuilders. termFilter("email",email))); searchRequestBuilder.setFrom(0).setSize(40).setExplain(true); SearchResponse response = null; response = searchRequestBuilder.execute().actionGet(); problem with filter with email here. if i have only email or email with any other field i am getting empty result where as i hava data in email with that text. in other word combination with email is not giving any result if i am applying the filter. please suggest accordingly Thanks in advanced On Tuesday, 6 November 2012 23:22:45 UTC, Ivan Brusic wrote: > The easiest way to to create a custom analyzer that utilizes the > uxl\_url\_email tokenizer and map that analyzer to a field. When that field > is indexed/searched, the analyzer would be used. > > -- > Ivan > > On Tue, Nov 6, 2012 at 5:18 AM, mohsin husen \<[smh...@gmail.com](mailto:smh...@gmail.com)\ > > > wrote: > > > Hello > > > > can we do uax\_url\_email tokenizer in java api ? > > if so then how to perform. > > please suggest. > > > > Thanks in advanced > > Mohsin H > > > > -- -- ---
**Author:** ![smhdiu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/smhdiu/32/1808_2.png) [@smhdiu](https://discuss.elastic.co/u/smhdiu)\ **Post date:** [November 8, 2012, 9:54am UTC](https://discuss.elastic.co/t/uax-url-email-tokenizer/9581/4 "2012-11-08T09:54:05Z")
just to add that i am using default analyzer. is it happening due to analyzer ? please please advise ☹ Thanks Mohsin On Wednesday, 7 November 2012 10:15:34 UTC, mohsin husen wrote: > Thanks Ivan for your reply. > i have following java code for fetching the data from elasticsearch > > SearchRequestBuilder searchRequestBuilder = > client.prepareSearch(indexName); > searchRequestBuilder.setSearchType(SearchType.DEFAULT); > searchRequestBuilder.setQuery(boolQuery() > .should(textQuery("email", email)) > .should(textQuery("firstName", firstName)) > .should(textQuery("lastName", lastName)) > .should(textQuery("keyword", keyword))); > > searchRequestBuilder.setFilter(FilterBuilders.andFilter().add(FilterBuilders. > termFilter("email",email))); > searchRequestBuilder.setFrom(0).setSize(40).setExplain(true); > SearchResponse response = null; > response = searchRequestBuilder.execute().actionGet(); > > problem with filter with email here. > if i have only email or email with any other field i am getting empty > result where as i hava data in email with that text. > in other word combination with email is not giving any result if i am > applying the filter. > > please suggest accordingly > > Thanks in advanced > > On Tuesday, 6 November 2012 23:22:45 UTC, Ivan Brusic wrote: > > > The easiest way to to create a custom analyzer that utilizes the > > uxl\_url\_email tokenizer and map that analyzer to a field. When that field > > is indexed/searched, the analyzer would be used. > > > > -- > > Ivan > > > > On Tue, Nov 6, 2012 at 5:18 AM, mohsin husen [smh...@gmail.com](mailto:smh...@gmail.com) wrote: > > > > > Hello > > > > > > can we do uax\_url\_email tokenizer in java api ? > > > if so then how to perform. > > > please suggest. > > > > > > Thanks in advanced > > > Mohsin H > > > > > > -- -- ---
**Author:** ![Ivan](https://avatars.discourse-cdn.com/v4/letter/i/df788c/32.png) [@Ivan](https://discuss.elastic.co/u/Ivan)\ **Post date:** [November 8, 2012, 4:55pm UTC](https://discuss.elastic.co/t/uax-url-email-tokenizer/9581/5 "2012-11-08T16:55:29Z")
You would need a custom analyzer in order to use that tokenizer. > **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)** > > Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today. Analyzers and tokenizers are Lucene concepts, so you can learn more by looking into Lucene. Remember to have a consistent analyzer between indexing and querying. Elasticsearch mappings make this part easy. -- Ivan On Thu, Nov 8, 2012 at 1:54 AM, mohsin husen [smhdiu@gmail.com](mailto:smhdiu@gmail.com) wrote: > just to add that i am using default analyzer. > is it happening due to analyzer ? > > please please advise ☹ > > Thanks > Mohsin > > On Wednesday, 7 November 2012 10:15:34 UTC, mohsin husen wrote: > > > Thanks Ivan for your reply. > > i have following java code for fetching the data from elasticsearch > > > > SearchRequestBuilder searchRequestBuilder = > > client.prepareSearch(\*\*indexName); > > searchRequestBuilder.\*\*setSearchType(SearchType.\*\*DEFAULT); > > searchRequestBuilder.\*\*setQuery(boolQuery() > > .should(textQuery("email", email)) > > .should(textQuery("firstName", firstName)) > > .should(textQuery("lastName", lastName)) > > .should(textQuery("keyword", keyword))); > > searchRequestBuilder.**setFilter(FilterBuilders.** > > andFilter().add(\*\*FilterBuilders. termFilter("email",email))); > > searchRequestBuilder.setFrom(\*\*0).setSize(40).setExplain(**true); > > SearchResponse response = null; > > response = searchRequestBuilder.execute()**.actionGet(); > > > > problem with filter with email here. > > if i have only email or email with any other field i am getting empty > > result where as i hava data in email with that text. > > in other word combination with email is not giving any result if i am > > applying the filter. > > > > please suggest accordingly > > > > Thanks in advanced > > > > On Tuesday, 6 November 2012 23:22:45 UTC, Ivan Brusic wrote: > > > > > The easiest way to to create a custom analyzer that utilizes the > > > uxl\_url\_email tokenizer and map that analyzer to a field. When that field > > > is indexed/searched, the analyzer would be used. > > > > > > -- > > > Ivan > > > > > > On Tue, Nov 6, 2012 at 5:18 AM, mohsin husen [smh...@gmail.com](mailto:smh...@gmail.com) wrote: > > > > > > > Hello > > > > > > > > can we do uax\_url\_email tokenizer in java api ? > > > > if so then how to perform. > > > > please suggest. > > > > > > > > Thanks in advanced > > > > Mohsin H > > > > > > > > -- > > > > > > -- -- ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [November 9, 2012, 7:48pm UTC](https://discuss.elastic.co/t/uax-url-email-tokenizer/9581/6 "2012-11-09T19:48:11Z")
Standard analyzer splits emails addresses into terms. So [elasticsearch@googlegroups.com](mailto:elasticsearch@googlegroups.com) is indexed as two terms: "elasticsearch" and "[googlegroups.com](http://googlegroups.com)". The email filter that you use FilterBuilders.termFilter("email",email) it searching for the entire email as a single term. There are two ways to solve this problem. One is to use uax\_url\_email (you can find an example here [http://stackoverflow.com/questions/13173185/not-analyzed-is-not-working-as-expected · GitHub](https://gist.github.com/3993902)) Another solution is to replace termFilter with queryFilter(matchPhraseQuery("email", email)). On Thursday, November 8, 2012 4:54:05 AM UTC-5, mohsin husen wrote: > just to add that i am using default analyzer. > is it happening due to analyzer ? > > please please advise ☹ > > Thanks > Mohsin > > On Wednesday, 7 November 2012 10:15:34 UTC, mohsin husen wrote: > > > Thanks Ivan for your reply. > > i have following java code for fetching the data from elasticsearch > > > > SearchRequestBuilder searchRequestBuilder = > > client.prepareSearch(indexName); > > searchRequestBuilder.setSearchType(SearchType.DEFAULT); > > searchRequestBuilder.setQuery(boolQuery() > > .should(textQuery("email", email)) > > .should(textQuery("firstName", firstName)) > > .should(textQuery("lastName", lastName)) > > .should(textQuery("keyword", keyword))); > > > > searchRequestBuilder.setFilter(FilterBuilders.andFilter().add(FilterBuilders. > > termFilter("email",email))); > > searchRequestBuilder.setFrom(0).setSize(40).setExplain(true); > > SearchResponse response = null; > > response = searchRequestBuilder.execute().actionGet(); > > > > problem with filter with email here. > > if i have only email or email with any other field i am getting empty > > result where as i hava data in email with that text. > > in other word combination with email is not giving any result if i am > > applying the filter. > > > > please suggest accordingly > > > > Thanks in advanced > > > > On Tuesday, 6 November 2012 23:22:45 UTC, Ivan Brusic wrote: > > > > > The easiest way to to create a custom analyzer that utilizes the > > > uxl\_url\_email tokenizer and map that analyzer to a field. When that field > > > is indexed/searched, the analyzer would be used. > > > > > > -- > > > Ivan > > > > > > On Tue, Nov 6, 2012 at 5:18 AM, mohsin husen [smh...@gmail.com](mailto:smh...@gmail.com) wrote: > > > > > > > Hello > > > > > > > > can we do uax\_url\_email tokenizer in java api ? > > > > if so then how to perform. > > > > please suggest. > > > > > > > > Thanks in advanced > > > > Mohsin H > > > > > > > > -- -- ---
**Author:** ![smhdiu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/smhdiu/32/1808_2.png) [@smhdiu](https://discuss.elastic.co/u/smhdiu)\ **Post date:** [November 12, 2012, 5:43pm UTC](https://discuss.elastic.co/t/uax-url-email-tokenizer/9581/7 "2012-11-12T17:43:45Z")
Thanks Igor and Ivan... with your guidence i creted the mappings as follows ` curl -XPUT 'http://localhost:9200/test/?pretty=1' -d ' { "mappings" : { "1" : { // this is for type 1 "properties" : { "authorOfAbstracts" : { "type" : "string" }, "firstName" : { "type" : "string","search_analyzer" : "name_analyzer","index_analyzer" : "name_analyzer"}, "lastName" : { "type" : "string","search_analyzer" : "name_analyzer","index_analyzer" : "name_analyzer"}, "email" : { "type" : "string", "search_analyzer" : "email_analyzer", "index_analyzer" : "email_analyzer" } } } }, "settings" : { "analysis" : { "analyzer" : { "name_analyzer" : { "filter" : [ "standard", "lowercase", "asciifolding" ], "type" : "custom", "tokenizer" : "standard" }, "email_analyzer" : { "type" : "custom", "tokenizer" : "uax_url_email" } ` `} }} }' ` now problem with this is when i am adding data to index=test and type=1, its working fine but when i m inserting the data to type=2 its not working as expected to be work with email tokenizer. its shows following mapping... in which we can clearly see for type=1 data will be indexed and searched as per the mapping but what about the type=2 ? can we achieve same mapping in all the type in one index? {"test":{ "2":{"properties":{"authorOfAbstracts":{"type":"string"},"email":{"type":"string"},"firstName":{"type":"string"},"lastName":{"type":"string"}}}, "1":{"properties":{"authorOfAbstracts":{"type":"string"},"email":{"type":"string","analyzer":"email\_analyzer"},"firstName":{"type":"string","analyzer":"name\_analyzer"},"lastName":{"type":"string","analyzer":"name\_analyzer"}}} }} Thanks Mohsin On Friday, 9 November 2012 19:48:11 UTC, Igor Motov wrote: > Standard analyzer splits emails addresses into terms. So > [elasti...@googlegroups.com](mailto:elasti...@googlegroups.com) \ is indexed as two terms: > "elasticsearch" and "[googlegroups.com](http://googlegroups.com)". The email filter that you > use FilterBuilders.termFilter("email",email) it searching for the entire > email as a single term. There are two ways to solve this problem. One is to > use uax\_url\_email (you can find an example here > [http://stackoverflow.com/questions/13173185/not-analyzed-is-not-working-as-expected · GitHub](https://gist.github.com/3993902)) Another solution is to replace > termFilter with queryFilter(matchPhraseQuery("email", email)). > > On Thursday, November 8, 2012 4:54:05 AM UTC-5, mohsin husen wrote: > > > just to add that i am using default analyzer. > > is it happening due to analyzer ? > > > > please please advise ☹ > > > > Thanks > > Mohsin > > > > On Wednesday, 7 November 2012 10:15:34 UTC, mohsin husen wrote: > > > > > Thanks Ivan for your reply. > > > i have following java code for fetching the data from elasticsearch > > > > > > SearchRequestBuilder searchRequestBuilder = > > > client.prepareSearch(indexName); > > > searchRequestBuilder.setSearchType(SearchType.DEFAULT); > > > searchRequestBuilder.setQuery(boolQuery() > > > .should(textQuery("email", email)) > > > .should(textQuery("firstName", firstName)) > > > .should(textQuery("lastName", lastName)) > > > .should(textQuery("keyword", keyword))); > > > > > > searchRequestBuilder.setFilter(FilterBuilders.andFilter().add(FilterBuilders. > > > termFilter("email",email))); > > > searchRequestBuilder.setFrom(0).setSize(40).setExplain(true); > > > SearchResponse response = null; > > > response = searchRequestBuilder.execute().actionGet(); > > > > > > problem with filter with email here. > > > if i have only email or email with any other field i am getting empty > > > result where as i hava data in email with that text. > > > in other word combination with email is not giving any result if i am > > > applying the filter. > > > > > > please suggest accordingly > > > > > > Thanks in advanced > > > > > > On Tuesday, 6 November 2012 23:22:45 UTC, Ivan Brusic wrote: > > > > > > > The easiest way to to create a custom analyzer that utilizes the > > > > uxl\_url\_email tokenizer and map that analyzer to a field. When that field > > > > is indexed/searched, the analyzer would be used. > > > > > > > > -- > > > > Ivan > > > > > > > > On Tue, Nov 6, 2012 at 5:18 AM, mohsin husen [smh...@gmail.com](mailto:smh...@gmail.com) wrote: > > > > > > > > > Hello > > > > > > > > > > can we do uax\_url\_email tokenizer in java api ? > > > > > if so then how to perform. > > > > > please suggest. > > > > > > > > > > Thanks in advanced > > > > > Mohsin H > > > > > > > > > > -- -- ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [November 12, 2012, 11:22pm UTC](https://discuss.elastic.co/t/uax-url-email-tokenizer/9581/8 "2012-11-12T23:22:52Z")
Sorry, I don't think I quite understand what you are trying to achieve here. If you are looking for some automation in assigning mappings to fields, take a look at Dynamic Templates[http://www.elasticsearch.org/guide/reference/mapping/root-object-type.html](http://www.elasticsearch.org/guide/reference/mapping/root-object-type.html)and Index Templates API[http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html](http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html) . On Monday, November 12, 2012 12:43:45 PM UTC-5, mohsin husen wrote: > Thanks Igor and Ivan... > with your guidence i creted the mappings as follows > > ` > curl -XPUT 'http://localhost:9200/test/?pretty=1' -d ' > { > "mappings" : { > "1" : { // this is for type 1 > "properties" : { > "authorOfAbstracts" : { > "type" : "string" > }, > "firstName" : { > "type" : "string","search_analyzer" : > "name_analyzer","index_analyzer" : "name_analyzer"}, > "lastName" : { > "type" : "string","search_analyzer" : > "name_analyzer","index_analyzer" : "name_analyzer"}, > "email" : { > "type" : "string", > "search_analyzer" : "email_analyzer", > "index_analyzer" : "email_analyzer" > } > } > } > }, > "settings" : { > "analysis" : { > "analyzer" : { > "name_analyzer" : { > "filter" : [ > "standard", > "lowercase", > "asciifolding" > ], > "type" : "custom", > "tokenizer" : "standard" > }, > "email_analyzer" : { > "type" : "custom", > "tokenizer" : "uax_url_email" > } > ` > > `} > }} > }' > ` > > now problem with this is when i am adding data to index=test and type=1, > its working fine but when i m inserting the data to type=2 its not working > as expected to be work with email tokenizer. > > its shows following mapping... in which we can clearly see for type=1 data > will be indexed and searched as per the mapping but what about the type=2 ? > can we achieve same mapping in all the type in one index? > > {"test":{ > > "2":{"properties":{"authorOfAbstracts":{"type":"string"},"email":{"type":"string"},"firstName":{"type":"string"},"lastName":{"type":"string"}}}, > > "1":{"properties":{"authorOfAbstracts":{"type":"string"},"email":{"type":"string","analyzer":"email\_analyzer"},"firstName":{"type":"string","analyzer":"name\_analyzer"},"lastName":{"type":"string","analyzer":"name\_analyzer"}}} > }} > > Thanks > Mohsin > > On Friday, 9 November 2012 19:48:11 UTC, Igor Motov wrote: > > > Standard analyzer splits emails addresses into terms. So > > [elasti...@googlegroups.com](mailto:elasti...@googlegroups.com) is indexed as two terms: "elasticsearch" and " > > [googlegroups.com](http://googlegroups.com)". The email filter that you > > use FilterBuilders.termFilter("email",email) it searching for the entire > > email as a single term. There are two ways to solve this problem. One is to > > use uax\_url\_email (you can find an example here > > [http://stackoverflow.com/questions/13173185/not-analyzed-is-not-working-as-expected · GitHub](https://gist.github.com/3993902)) Another solution is to replace > > termFilter with queryFilter(matchPhraseQuery("email", email)). > > > > On Thursday, November 8, 2012 4:54:05 AM UTC-5, mohsin husen wrote: > > > > > just to add that i am using default analyzer. > > > is it happening due to analyzer ? > > > > > > please please advise ☹ > > > > > > Thanks > > > Mohsin > > > > > > On Wednesday, 7 November 2012 10:15:34 UTC, mohsin husen wrote: > > > > > > > Thanks Ivan for your reply. > > > > i have following java code for fetching the data from elasticsearch > > > > > > > > SearchRequestBuilder searchRequestBuilder = > > > > client.prepareSearch(indexName); > > > > searchRequestBuilder.setSearchType(SearchType.DEFAULT); > > > > searchRequestBuilder.setQuery(boolQuery() > > > > .should(textQuery("email", email)) > > > > .should(textQuery("firstName", firstName)) > > > > .should(textQuery("lastName", lastName)) > > > > .should(textQuery("keyword", keyword))); > > > > > > > > searchRequestBuilder.setFilter(FilterBuilders.andFilter().add(FilterBuilders. > > > > termFilter("email",email))); > > > > searchRequestBuilder.setFrom(0).setSize(40).setExplain(true); > > > > SearchResponse response = null; > > > > response = searchRequestBuilder.execute().actionGet(); > > > > > > > > problem with filter with email here. > > > > if i have only email or email with any other field i am getting empty > > > > result where as i hava data in email with that text. > > > > in other word combination with email is not giving any result if i am > > > > applying the filter. > > > > > > > > please suggest accordingly > > > > > > > > Thanks in advanced > > > > > > > > On Tuesday, 6 November 2012 23:22:45 UTC, Ivan Brusic wrote: > > > > > > > > > The easiest way to to create a custom analyzer that utilizes the > > > > > uxl\_url\_email tokenizer and map that analyzer to a field. When that field > > > > > is indexed/searched, the analyzer would be used. > > > > > > > > > > -- > > > > > Ivan > > > > > > > > > > On Tue, Nov 6, 2012 at 5:18 AM, mohsin husen [smh...@gmail.com](mailto:smh...@gmail.com) wrote: > > > > > > > > > > > Hello > > > > > > > > > > > > can we do uax\_url\_email tokenizer in java api ? > > > > > > if so then how to perform. > > > > > > please suggest. > > > > > > > > > > > > Thanks in advanced > > > > > > Mohsin H > > > > > > > > > > > > -- -- ---
**Author:** ![smhdiu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/smhdiu/32/1808_2.png) [@smhdiu](https://discuss.elastic.co/u/smhdiu)\ **Post date:** [November 13, 2012, 10:41am UTC](https://discuss.elastic.co/t/uax-url-email-tokenizer/9581/9 "2012-11-13T10:41:18Z")
Sorry for the confusion Igor. suppose i have a index as books and their types are departmentid. so if i want to create the mapping for bookid bookname price author author\_email etc like below bookid - string , lowercase bookname - string, lowercase price - long author - string lowercase author\_email - string, uax\_url\_email tokenizer now problem with this mapping is that it is asking for the type while creating the mapping and settings. if i will create mapping with departmentid - 1 then whatever goes in that department has those mappings and settings which we created. but now when we add data to second depatmentid -2 it takes by default values. which is quite right as we havent created mappings and settings for this type. so my question is how to create the mapping and the settings that can be available for all the types by default. let me know if you still have doubt. Thanks in advanced. On Monday, 12 November 2012 23:22:52 UTC, Igor Motov wrote: > Sorry, I don't think I quite understand what you are trying > to achieve here. If you are looking for some automation in assigning > mappings to fields, take a look at Dynamic Templates[http://www.elasticsearch.org/guide/reference/mapping/root-object-type.html](http://www.elasticsearch.org/guide/reference/mapping/root-object-type.html)and Index > Templates API[http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html](http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html) > . > > On Monday, November 12, 2012 12:43:45 PM UTC-5, mohsin husen wrote: > > > Thanks Igor and Ivan... > > with your guidence i creted the mappings as follows > > > > ` > > curl -XPUT 'http://localhost:9200/test/?pretty=1' -d ' > > { > > "mappings" : { > > "1" : { // this is for type 1 > > "properties" : { > > "authorOfAbstracts" : { > > "type" : "string" > > }, > > "firstName" : { > > "type" : "string","search_analyzer" : > > "name_analyzer","index_analyzer" : "name_analyzer"}, > > "lastName" : { > > "type" : "string","search_analyzer" : > > "name_analyzer","index_analyzer" : "name_analyzer"}, > > "email" : { > > "type" : "string", > > "search_analyzer" : "email_analyzer", > > "index_analyzer" : "email_analyzer" > > } > > } > > } > > }, > > "settings" : { > > "analysis" : { > > "analyzer" : { > > "name_analyzer" : { > > "filter" : [ > > "standard", > > "lowercase", > > "asciifolding" > > ], > > "type" : "custom", > > "tokenizer" : "standard" > > }, > > "email_analyzer" : { > > "type" : "custom", > > "tokenizer" : "uax_url_email" > > } > > ` > > > > `} > > }} > > }' > > ` > > > > now problem with this is when i am adding data to index=test and type=1, > > its working fine but when i m inserting the data to type=2 its not working > > as expected to be work with email tokenizer. > > > > its shows following mapping... in which we can clearly see for type=1 > > data will be indexed and searched as per the mapping but what about the > > type=2 ? > > can we achieve same mapping in all the type in one index? > > > > {"test":{ > > > > "2":{"properties":{"authorOfAbstracts":{"type":"string"},"email":{"type":"string"},"firstName":{"type":"string"},"lastName":{"type":"string"}}}, > > > > "1":{"properties":{"authorOfAbstracts":{"type":"string"},"email":{"type":"string","analyzer":"email\_analyzer"},"firstName":{"type":"string","analyzer":"name\_analyzer"},"lastName":{"type":"string","analyzer":"name\_analyzer"}}} > > }} > > > > Thanks > > Mohsin > > > > On Friday, 9 November 2012 19:48:11 UTC, Igor Motov wrote: > > > > > Standard analyzer splits emails addresses into terms. So > > > [elasti...@googlegroups.com](mailto:elasti...@googlegroups.com) is indexed as two terms: "elasticsearch" and > > > "[googlegroups.com](http://googlegroups.com)". The email filter that you > > > use FilterBuilders.termFilter("email",email) it searching for the entire > > > email as a single term. There are two ways to solve this problem. One is to > > > use uax\_url\_email (you can find an example here > > > [http://stackoverflow.com/questions/13173185/not-analyzed-is-not-working-as-expected · GitHub](https://gist.github.com/3993902)) Another solution is to replace > > > termFilter with queryFilter(matchPhraseQuery("email", email)). > > > > > > On Thursday, November 8, 2012 4:54:05 AM UTC-5, mohsin husen wrote: > > > > > > > just to add that i am using default analyzer. > > > > is it happening due to analyzer ? > > > > > > > > please please advise ☹ > > > > > > > > Thanks > > > > Mohsin > > > > > > > > On Wednesday, 7 November 2012 10:15:34 UTC, mohsin husen wrote: > > > > > > > > > Thanks Ivan for your reply. > > > > > i have following java code for fetching the data from elasticsearch > > > > > > > > > > SearchRequestBuilder searchRequestBuilder = > > > > > client.prepareSearch(indexName); > > > > > searchRequestBuilder.setSearchType(SearchType.DEFAULT); > > > > > searchRequestBuilder.setQuery(boolQuery() > > > > > .should(textQuery("email", email)) > > > > > .should(textQuery("firstName", firstName)) > > > > > .should(textQuery("lastName", lastName)) > > > > > .should(textQuery("keyword", keyword))); > > > > > > > > > > searchRequestBuilder.setFilter(FilterBuilders.andFilter().add(FilterBuilders. > > > > > termFilter("email",email))); > > > > > searchRequestBuilder.setFrom(0).setSize(40).setExplain(true); > > > > > SearchResponse response = null; > > > > > response = searchRequestBuilder.execute().actionGet(); > > > > > > > > > > problem with filter with email here. > > > > > if i have only email or email with any other field i am getting empty > > > > > result where as i hava data in email with that text. > > > > > in other word combination with email is not giving any result if i am > > > > > applying the filter. > > > > > > > > > > please suggest accordingly > > > > > > > > > > Thanks in advanced > > > > > > > > > > On Tuesday, 6 November 2012 23:22:45 UTC, Ivan Brusic wrote: > > > > > > > > > > > The easiest way to to create a custom analyzer that utilizes the > > > > > > uxl\_url\_email tokenizer and map that analyzer to a field. When that field > > > > > > is indexed/searched, the analyzer would be used. > > > > > > > > > > > > -- > > > > > > Ivan > > > > > > > > > > > > On Tue, Nov 6, 2012 at 5:18 AM, mohsin husen [smh...@gmail.com](mailto:smh...@gmail.com)wrote: > > > > > > > > > > > > > Hello > > > > > > > > > > > > > > can we do uax\_url\_email tokenizer in java api ? > > > > > > > if so then how to perform. > > > > > > > please suggest. > > > > > > > > > > > > > > Thanks in advanced > > > > > > > Mohsin H > > > > > > > > > > > > > > -- -- ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [November 13, 2012, 11:49am UTC](https://discuss.elastic.co/t/uax-url-email-tokenizer/9581/10 "2012-11-13T11:49:04Z")
Why cannot you just use a field for departmentid instead of creating a whole new type? Type is somewhat heavy construct because of mappings that are created, maintained and sent between nodes. On Tuesday, November 13, 2012 5:41:18 AM UTC-5, mohsin husen wrote: > Sorry for the confusion Igor. > suppose i have a index as books and their types are departmentid. > > so if i want to create the mapping for bookid bookname price author > author\_email etc like below > > bookid - string , lowercase > bookname - string, lowercase > price - long > author - string lowercase > author\_email - string, uax\_url\_email tokenizer > > now problem with this mapping is that it is asking for the type while > creating the mapping and settings. > if i will create mapping with departmentid - 1 then whatever goes in that > department has those mappings and settings which we created. > > but now when we add data to second depatmentid -2 it takes by default > values. which is quite right as we havent created mappings and settings for > this type. > > so my question is how to create the mapping and the settings that can be > available for all the types by default. > > let me know if you still have doubt. > > Thanks in advanced. > > On Monday, 12 November 2012 23:22:52 UTC, Igor Motov wrote: > > > Sorry, I don't think I quite understand what you are trying > > to achieve here. If you are looking for some automation in assigning > > mappings to fields, take a look at Dynamic Templates[http://www.elasticsearch.org/guide/reference/mapping/root-object-type.html](http://www.elasticsearch.org/guide/reference/mapping/root-object-type.html)and Index > > Templates API[http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html](http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html) > > . > > > > On Monday, November 12, 2012 12:43:45 PM UTC-5, mohsin husen wrote: > > > > > Thanks Igor and Ivan... > > > with your guidence i creted the mappings as follows > > > > > > ` > > > curl -XPUT 'http://localhost:9200/test/?pretty=1' -d ' > > > { > > > "mappings" : { > > > "1" : { // this is for type 1 > > > "properties" : { > > > "authorOfAbstracts" : { > > > "type" : "string" > > > }, > > > "firstName" : { > > > "type" : "string","search_analyzer" : > > > "name_analyzer","index_analyzer" : "name_analyzer"}, > > > "lastName" : { > > > "type" : "string","search_analyzer" : > > > "name_analyzer","index_analyzer" : "name_analyzer"}, > > > "email" : { > > > "type" : "string", > > > "search_analyzer" : "email_analyzer", > > > "index_analyzer" : "email_analyzer" > > > } > > > } > > > } > > > }, > > > "settings" : { > > > "analysis" : { > > > "analyzer" : { > > > "name_analyzer" : { > > > "filter" : [ > > > "standard", > > > "lowercase", > > > "asciifolding" > > > ], > > > "type" : "custom", > > > "tokenizer" : "standard" > > > }, > > > "email_analyzer" : { > > > "type" : "custom", > > > "tokenizer" : "uax_url_email" > > > } > > > ` > > > > > > `} > > > }} > > > }' > > > ` > > > > > > now problem with this is when i am adding data to index=test and type=1, > > > its working fine but when i m inserting the data to type=2 its not working > > > as expected to be work with email tokenizer. > > > > > > its shows following mapping... in which we can clearly see for type=1 > > > data will be indexed and searched as per the mapping but what about the > > > type=2 ? > > > can we achieve same mapping in all the type in one index? > > > > > > {"test":{ > > > > > > "2":{"properties":{"authorOfAbstracts":{"type":"string"},"email":{"type":"string"},"firstName":{"type":"string"},"lastName":{"type":"string"}}}, > > > > > > "1":{"properties":{"authorOfAbstracts":{"type":"string"},"email":{"type":"string","analyzer":"email\_analyzer"},"firstName":{"type":"string","analyzer":"name\_analyzer"},"lastName":{"type":"string","analyzer":"name\_analyzer"}}} > > > }} > > > > > > Thanks > > > Mohsin > > > > > > On Friday, 9 November 2012 19:48:11 UTC, Igor Motov wrote: > > > > > > > Standard analyzer splits emails addresses into terms. So > > > > [elasti...@googlegroups.com](mailto:elasti...@googlegroups.com) is indexed as two terms: "elasticsearch" > > > > and "[googlegroups.com](http://googlegroups.com)". The email filter that you > > > > use FilterBuilders.termFilter("email",email) it searching for the entire > > > > email as a single term. There are two ways to solve this problem. One is to > > > > use uax\_url\_email (you can find an example here > > > > [http://stackoverflow.com/questions/13173185/not-analyzed-is-not-working-as-expected · GitHub](https://gist.github.com/3993902)) Another solution is to replace > > > > termFilter with queryFilter(matchPhraseQuery("email", email)). > > > > > > > > On Thursday, November 8, 2012 4:54:05 AM UTC-5, mohsin husen wrote: > > > > > > > > > just to add that i am using default analyzer. > > > > > is it happening due to analyzer ? > > > > > > > > > > please please advise ☹ > > > > > > > > > > Thanks > > > > > Mohsin > > > > > > > > > > On Wednesday, 7 November 2012 10:15:34 UTC, mohsin husen wrote: > > > > > > > > > > > Thanks Ivan for your reply. > > > > > > i have following java code for fetching the data from elasticsearch > > > > > > > > > > > > SearchRequestBuilder searchRequestBuilder = > > > > > > client.prepareSearch(indexName); > > > > > > searchRequestBuilder.setSearchType(SearchType.DEFAULT); > > > > > > searchRequestBuilder.setQuery(boolQuery() > > > > > > .should(textQuery("email", email)) > > > > > > .should(textQuery("firstName", firstName)) > > > > > > .should(textQuery("lastName", lastName)) > > > > > > .should(textQuery("keyword", keyword))); > > > > > > > > > > > > searchRequestBuilder.setFilter(FilterBuilders.andFilter().add(FilterBuilders. > > > > > > termFilter("email",email))); > > > > > > searchRequestBuilder.setFrom(0).setSize(40).setExplain(true); > > > > > > SearchResponse response = null; > > > > > > response = searchRequestBuilder.execute().actionGet(); > > > > > > > > > > > > problem with filter with email here. > > > > > > if i have only email or email with any other field i am getting empty > > > > > > result where as i hava data in email with that text. > > > > > > in other word combination with email is not giving any result if i am > > > > > > applying the filter. > > > > > > > > > > > > please suggest accordingly > > > > > > > > > > > > Thanks in advanced > > > > > > > > > > > > On Tuesday, 6 November 2012 23:22:45 UTC, Ivan Brusic wrote: > > > > > > > > > > > > > The easiest way to to create a custom analyzer that utilizes the > > > > > > > uxl\_url\_email tokenizer and map that analyzer to a field. When that field > > > > > > > is indexed/searched, the analyzer would be used. > > > > > > > > > > > > > > -- > > > > > > > Ivan > > > > > > > > > > > > > > On Tue, Nov 6, 2012 at 5:18 AM, mohsin husen [smh...@gmail.com](mailto:smh...@gmail.com)wrote: > > > > > > > > > > > > > > > Hello > > > > > > > > > > > > > > > > can we do uax\_url\_email tokenizer in java api ? > > > > > > > > if so then how to perform. > > > > > > > > please suggest. > > > > > > > > > > > > > > > > Thanks in advanced > > > > > > > > Mohsin H > > > > > > > > > > > > > > > > -- -- ---
**Author:** ![smhdiu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/smhdiu/32/1808_2.png) [@smhdiu](https://discuss.elastic.co/u/smhdiu)\ **Post date:** [November 13, 2012, 12:27pm UTC](https://discuss.elastic.co/t/uax-url-email-tokenizer/9581/11 "2012-11-13T12:27:39Z")
Hello Igor, Thanks for the revert back. i am lil bit confused in creating the type, i have following situation. i have book database which has books of different department like Computer, Account etc.. now each department has millions of books. so what I thought was like creating the main index as books and then divide the department as type so while searching particular book in department we can only go to specific department id(type) rather than searching entire index of books. so just curious about the approach, will it degrade more performance in compare to the single type? Thanks in advanced Mohsin On Tuesday, 13 November 2012 11:49:04 UTC, Igor Motov wrote: > Why cannot you just use a field for departmentid instead of creating a > whole new type? Type is somewhat heavy construct because of mappings that > are created, maintained and sent between nodes. > > On Tuesday, November 13, 2012 5:41:18 AM UTC-5, mohsin husen wrote: > > > Sorry for the confusion Igor. > > suppose i have a index as books and their types are departmentid. > > > > so if i want to create the mapping for bookid bookname price author > > author\_email etc like below > > > > bookid - string , lowercase > > bookname - string, lowercase > > price - long > > author - string lowercase > > author\_email - string, uax\_url\_email tokenizer > > > > now problem with this mapping is that it is asking for the type while > > creating the mapping and settings. > > if i will create mapping with departmentid - 1 then whatever goes in that > > department has those mappings and settings which we created. > > > > but now when we add data to second depatmentid -2 it takes by default > > values. which is quite right as we havent created mappings and settings for > > this type. > > > > so my question is how to create the mapping and the settings that can be > > available for all the types by default. > > > > let me know if you still have doubt. > > > > Thanks in advanced. > > > > On Monday, 12 November 2012 23:22:52 UTC, Igor Motov wrote: > > > > > Sorry, I don't think I quite understand what you are trying > > > to achieve here. If you are looking for some automation in assigning > > > mappings to fields, take a look at Dynamic Templates[http://www.elasticsearch.org/guide/reference/mapping/root-object-type.html](http://www.elasticsearch.org/guide/reference/mapping/root-object-type.html)and Index > > > Templates API[http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html](http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html) > > > . > > > > > > On Monday, November 12, 2012 12:43:45 PM UTC-5, mohsin husen wrote: > > > > > > > Thanks Igor and Ivan... > > > > with your guidence i creted the mappings as follows > > > > > > > > ` > > > > curl -XPUT 'http://localhost:9200/test/?pretty=1' -d ' > > > > { > > > > "mappings" : { > > > > "1" : { // this is for type 1 > > > > "properties" : { > > > > "authorOfAbstracts" : { > > > > "type" : "string" > > > > }, > > > > "firstName" : { > > > > "type" : "string","search_analyzer" : > > > > "name_analyzer","index_analyzer" : "name_analyzer"}, > > > > "lastName" : { > > > > "type" : "string","search_analyzer" : > > > > "name_analyzer","index_analyzer" : "name_analyzer"}, > > > > "email" : { > > > > "type" : "string", > > > > "search_analyzer" : "email_analyzer", > > > > "index_analyzer" : "email_analyzer" > > > > } > > > > } > > > > } > > > > }, > > > > "settings" : { > > > > "analysis" : { > > > > "analyzer" : { > > > > "name_analyzer" : { > > > > "filter" : [ > > > > "standard", > > > > "lowercase", > > > > "asciifolding" > > > > ], > > > > "type" : "custom", > > > > "tokenizer" : "standard" > > > > }, > > > > "email_analyzer" : { > > > > "type" : "custom", > > > > "tokenizer" : "uax_url_email" > > > > } > > > > ` > > > > > > > > `} > > > > }} > > > > }' > > > > ` > > > > > > > > now problem with this is when i am adding data to index=test and > > > > type=1, its working fine but when i m inserting the data to type=2 its not > > > > working as expected to be work with email tokenizer. > > > > > > > > its shows following mapping... in which we can clearly see for type=1 > > > > data will be indexed and searched as per the mapping but what about the > > > > type=2 ? > > > > can we achieve same mapping in all the type in one index? > > > > > > > > {"test":{ > > > > > > > > "2":{"properties":{"authorOfAbstracts":{"type":"string"},"email":{"type":"string"},"firstName":{"type":"string"},"lastName":{"type":"string"}}}, > > > > > > > > "1":{"properties":{"authorOfAbstracts":{"type":"string"},"email":{"type":"string","analyzer":"email\_analyzer"},"firstName":{"type":"string","analyzer":"name\_analyzer"},"lastName":{"type":"string","analyzer":"name\_analyzer"}}} > > > > }} > > > > > > > > Thanks > > > > Mohsin > > > > > > > > On Friday, 9 November 2012 19:48:11 UTC, Igor Motov wrote: > > > > > > > > > Standard analyzer splits emails addresses into terms. So > > > > > [elasti...@googlegroups.com](mailto:elasti...@googlegroups.com) is indexed as two terms: "elasticsearch" > > > > > and "[googlegroups.com](http://googlegroups.com)". The email filter that you > > > > > use FilterBuilders.termFilter("email",email) it searching for the entire > > > > > email as a single term. There are two ways to solve this problem. One is to > > > > > use uax\_url\_email (you can find an example here > > > > > [http://stackoverflow.com/questions/13173185/not-analyzed-is-not-working-as-expected · GitHub](https://gist.github.com/3993902)) Another solution is to replace > > > > > termFilter with queryFilter(matchPhraseQuery("email", email)). > > > > > > > > > > On Thursday, November 8, 2012 4:54:05 AM UTC-5, mohsin husen wrote: > > > > > > > > > > > just to add that i am using default analyzer. > > > > > > is it happening due to analyzer ? > > > > > > > > > > > > please please advise ☹ > > > > > > > > > > > > Thanks > > > > > > Mohsin > > > > > > > > > > > > On Wednesday, 7 November 2012 10:15:34 UTC, mohsin husen wrote: > > > > > > > > > > > > > Thanks Ivan for your reply. > > > > > > > i have following java code for fetching the data from elasticsearch > > > > > > > > > > > > > > SearchRequestBuilder searchRequestBuilder = > > > > > > > client.prepareSearch(indexName); > > > > > > > searchRequestBuilder.setSearchType(SearchType.DEFAULT); > > > > > > > searchRequestBuilder.setQuery(boolQuery() > > > > > > > .should(textQuery("email", email)) > > > > > > > .should(textQuery("firstName", firstName)) > > > > > > > .should(textQuery("lastName", lastName)) > > > > > > > .should(textQuery("keyword", keyword))); > > > > > > > > > > > > > > searchRequestBuilder.setFilter(FilterBuilders.andFilter().add(FilterBuilders. > > > > > > > termFilter("email",email))); > > > > > > > searchRequestBuilder.setFrom(0).setSize(40).setExplain(true); > > > > > > > SearchResponse response = null; > > > > > > > response = searchRequestBuilder.execute().actionGet(); > > > > > > > > > > > > > > problem with filter with email here. > > > > > > > if i have only email or email with any other field i am getting > > > > > > > empty result where as i hava data in email with that text. > > > > > > > in other word combination with email is not giving any result if i > > > > > > > am applying the filter. > > > > > > > > > > > > > > please suggest accordingly > > > > > > > > > > > > > > Thanks in advanced > > > > > > > > > > > > > > On Tuesday, 6 November 2012 23:22:45 UTC, Ivan Brusic wrote: > > > > > > > > > > > > > > > The easiest way to to create a custom analyzer that utilizes the > > > > > > > > uxl\_url\_email tokenizer and map that analyzer to a field. When that field > > > > > > > > is indexed/searched, the analyzer would be used. > > > > > > > > > > > > > > > > -- > > > > > > > > Ivan > > > > > > > > > > > > > > > > On Tue, Nov 6, 2012 at 5:18 AM, mohsin husen [smh...@gmail.com](mailto:smh...@gmail.com)wrote: > > > > > > > > > > > > > > > > > Hello > > > > > > > > > > > > > > > > > > can we do uax\_url\_email tokenizer in java api ? > > > > > > > > > if so then how to perform. > > > > > > > > > please suggest. > > > > > > > > > > > > > > > > > > Thanks in advanced > > > > > > > > > Mohsin H > > > > > > > > > > > > > > > > > > -- -- ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [November 13, 2012, 1:06pm UTC](https://discuss.elastic.co/t/uax-url-email-tokenizer/9581/12 "2012-11-13T13:06:09Z")
Documents of all types are indexed into the same index. There is special \_type [http://www.elasticsearch.org/guide/reference/mapping/type-field.html](http://www.elasticsearch.org/guide/reference/mapping/type-field.html)field that is indexed and used to query documents of a certain type. So, using different types for different departments doesn't give you any performance advantages comparing to indexing departmentid as not\_analyzed field, but there is some overhead of handling additional types. If you want to limit your searches to a single department you will need to create one index per department or consider using departmentid as a routing value. My advice, however, would be to just use a single index for now, get more comfortable with elasticsearch, and optimize it later, when it will become an issue and you will have some real data and queries to test different optimizations on. On Tuesday, November 13, 2012 7:27:39 AM UTC-5, mohsin husen wrote: > Hello Igor, > > Thanks for the revert back. > i am lil bit confused in creating the type, i have following situation. > > i have book database which has books of different department like > Computer, Account etc.. > now each department has millions of books. so what I thought was like > creating the main index as books and then divide the department as type so > while searching particular book in department we can only go to specific > department id(type) rather than searching entire index of books. > > so just curious about the approach, > will it degrade more performance in compare to the single type? > > Thanks in advanced > Mohsin > > On Tuesday, 13 November 2012 11:49:04 UTC, Igor Motov wrote: > > > Why cannot you just use a field for departmentid instead of creating a > > whole new type? Type is somewhat heavy construct because of mappings that > > are created, maintained and sent between nodes. > > > > On Tuesday, November 13, 2012 5:41:18 AM UTC-5, mohsin husen wrote: > > > > > Sorry for the confusion Igor. > > > suppose i have a index as books and their types are departmentid. > > > > > > so if i want to create the mapping for bookid bookname price author > > > author\_email etc like below > > > > > > bookid - string , lowercase > > > bookname - string, lowercase > > > price - long > > > author - string lowercase > > > author\_email - string, uax\_url\_email tokenizer > > > > > > now problem with this mapping is that it is asking for the type while > > > creating the mapping and settings. > > > if i will create mapping with departmentid - 1 then whatever goes in > > > that department has those mappings and settings which we created. > > > > > > but now when we add data to second depatmentid -2 it takes by default > > > values. which is quite right as we havent created mappings and settings for > > > this type. > > > > > > so my question is how to create the mapping and the settings that can be > > > available for all the types by default. > > > > > > let me know if you still have doubt. > > > > > > Thanks in advanced. > > > > > > On Monday, 12 November 2012 23:22:52 UTC, Igor Motov wrote: > > > > > > > Sorry, I don't think I quite understand what you are trying > > > > to achieve here. If you are looking for some automation in assigning > > > > mappings to fields, take a look at Dynamic Templates[http://www.elasticsearch.org/guide/reference/mapping/root-object-type.html](http://www.elasticsearch.org/guide/reference/mapping/root-object-type.html)and Index > > > > Templates API[http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html](http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html) > > > > . > > > > > > > > On Monday, November 12, 2012 12:43:45 PM UTC-5, mohsin husen wrote: > > > > > > > > > Thanks Igor and Ivan... > > > > > with your guidence i creted the mappings as follows > > > > > > > > > > ` > > > > > curl -XPUT 'http://localhost:9200/test/?pretty=1' -d ' > > > > > { > > > > > "mappings" : { > > > > > "1" : { // this is for type 1 > > > > > "properties" : { > > > > > "authorOfAbstracts" : { > > > > > "type" : "string" > > > > > }, > > > > > "firstName" : { > > > > > "type" : "string","search_analyzer" : > > > > > "name_analyzer","index_analyzer" : "name_analyzer"}, > > > > > "lastName" : { > > > > > "type" : "string","search_analyzer" : > > > > > "name_analyzer","index_analyzer" : "name_analyzer"}, > > > > > "email" : { > > > > > "type" : "string", > > > > > "search_analyzer" : "email_analyzer", > > > > > "index_analyzer" : "email_analyzer" > > > > > } > > > > > } > > > > > } > > > > > }, > > > > > "settings" : { > > > > > "analysis" : { > > > > > "analyzer" : { > > > > > "name_analyzer" : { > > > > > "filter" : [ > > > > > "standard", > > > > > "lowercase", > > > > > "asciifolding" > > > > > ], > > > > > "type" : "custom", > > > > > "tokenizer" : "standard" > > > > > }, > > > > > "email_analyzer" : { > > > > > "type" : "custom", > > > > > "tokenizer" : "uax_url_email" > > > > > } > > > > > ` > > > > > > > > > > `} > > > > > }} > > > > > }' > > > > > ` > > > > > > > > > > now problem with this is when i am adding data to index=test and > > > > > type=1, its working fine but when i m inserting the data to type=2 its not > > > > > working as expected to be work with email tokenizer. > > > > > > > > > > its shows following mapping... in which we can clearly see for type=1 > > > > > data will be indexed and searched as per the mapping but what about the > > > > > type=2 ? > > > > > can we achieve same mapping in all the type in one index? > > > > > > > > > > {"test":{ > > > > > > > > > > "2":{"properties":{"authorOfAbstracts":{"type":"string"},"email":{"type":"string"},"firstName":{"type":"string"},"lastName":{"type":"string"}}}, > > > > > > > > > > "1":{"properties":{"authorOfAbstracts":{"type":"string"},"email":{"type":"string","analyzer":"email\_analyzer"},"firstName":{"type":"string","analyzer":"name\_analyzer"},"lastName":{"type":"string","analyzer":"name\_analyzer"}}} > > > > > }} > > > > > > > > > > Thanks > > > > > Mohsin > > > > > > > > > > On Friday, 9 November 2012 19:48:11 UTC, Igor Motov wrote: > > > > > > > > > > > Standard analyzer splits emails addresses into terms. So > > > > > > [elasti...@googlegroups.com](mailto:elasti...@googlegroups.com) is indexed as two terms: "elasticsearch" > > > > > > and "[googlegroups.com](http://googlegroups.com)". The email filter that you > > > > > > use FilterBuilders.termFilter("email",email) it searching for the entire > > > > > > email as a single term. There are two ways to solve this problem. One is to > > > > > > use uax\_url\_email (you can find an example here > > > > > > [http://stackoverflow.com/questions/13173185/not-analyzed-is-not-working-as-expected · GitHub](https://gist.github.com/3993902)) Another solution is to replace > > > > > > termFilter with queryFilter(matchPhraseQuery("email", email)). > > > > > > > > > > > > On Thursday, November 8, 2012 4:54:05 AM UTC-5, mohsin husen wrote: > > > > > > > > > > > > > just to add that i am using default analyzer. > > > > > > > is it happening due to analyzer ? > > > > > > > > > > > > > > please please advise ☹ > > > > > > > > > > > > > > Thanks > > > > > > > Mohsin > > > > > > > > > > > > > > On Wednesday, 7 November 2012 10:15:34 UTC, mohsin husen wrote: > > > > > > > > > > > > > > > Thanks Ivan for your reply. > > > > > > > > i have following java code for fetching the data from elasticsearch > > > > > > > > > > > > > > > > SearchRequestBuilder searchRequestBuilder = > > > > > > > > client.prepareSearch(indexName); > > > > > > > > searchRequestBuilder.setSearchType(SearchType.DEFAULT); > > > > > > > > searchRequestBuilder.setQuery(boolQuery() > > > > > > > > .should(textQuery("email", email)) > > > > > > > > .should(textQuery("firstName", firstName)) > > > > > > > > .should(textQuery("lastName", lastName)) > > > > > > > > .should(textQuery("keyword", keyword))); > > > > > > > > > > > > > > > > searchRequestBuilder.setFilter(FilterBuilders.andFilter().add(FilterBuilders. > > > > > > > > termFilter("email",email))); > > > > > > > > > > > > > > > > searchRequestBuilder.setFrom(0).setSize(40).setExplain(true); > > > > > > > > SearchResponse response = null; > > > > > > > > response = searchRequestBuilder.execute().actionGet(); > > > > > > > > > > > > > > > > problem with filter with email here. > > > > > > > > if i have only email or email with any other field i am getting > > > > > > > > empty result where as i hava data in email with that text. > > > > > > > > in other word combination with email is not giving any result if i > > > > > > > > am applying the filter. > > > > > > > > > > > > > > > > please suggest accordingly > > > > > > > > > > > > > > > > Thanks in advanced > > > > > > > > > > > > > > > > On Tuesday, 6 November 2012 23:22:45 UTC, Ivan Brusic wrote: > > > > > > > > > > > > > > > > > The easiest way to to create a custom analyzer that utilizes the > > > > > > > > > uxl\_url\_email tokenizer and map that analyzer to a field. When that field > > > > > > > > > is indexed/searched, the analyzer would be used. > > > > > > > > > > > > > > > > > > -- > > > > > > > > > Ivan > > > > > > > > > > > > > > > > > > On Tue, Nov 6, 2012 at 5:18 AM, mohsin husen [smh...@gmail.com](mailto:smh...@gmail.com)wrote: > > > > > > > > > > > > > > > > > > > Hello > > > > > > > > > > > > > > > > > > > > can we do uax\_url\_email tokenizer in java api ? > > > > > > > > > > if so then how to perform. > > > > > > > > > > please suggest. > > > > > > > > > > > > > > > > > > > > Thanks in advanced > > > > > > > > > > Mohsin H > > > > > > > > > > > > > > > > > > > > -- -- ---
**Author:** ![smhdiu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/smhdiu/32/1808_2.png) [@smhdiu](https://discuss.elastic.co/u/smhdiu)\ **Post date:** [November 14, 2012, 12:31pm UTC](https://discuss.elastic.co/t/uax-url-email-tokenizer/9581/13 "2012-11-14T12:31:06Z")
Igor ! Thanks a Ton! Really appreciated your response. It was really helpful. will try single index approach for now as u suggested. Thanks again. On Tuesday, 13 November 2012 13:06:10 UTC, Igor Motov wrote: > Documents of all types are indexed into the same index. There is special > \_type[http://www.elasticsearch.org/guide/reference/mapping/type-field.html](http://www.elasticsearch.org/guide/reference/mapping/type-field.html)field that is indexed and used to query documents of a certain type. So, > using different types for different departments doesn't give you any > performance advantages comparing to indexing departmentid as not\_analyzed > field, but there is some overhead of handling additional types. > > If you want to limit your searches to a single department you will need to > create one index per department or consider using departmentid as a routing > value. My advice, however, would be to just use a single index for now, get > more comfortable with elasticsearch, and optimize it later, when it will > become an issue and you will have some real data and queries to test > different optimizations on. > > On Tuesday, November 13, 2012 7:27:39 AM UTC-5, mohsin husen wrote: > > > Hello Igor, > > > > Thanks for the revert back. > > i am lil bit confused in creating the type, i have following situation. > > > > i have book database which has books of different department like > > Computer, Account etc.. > > now each department has millions of books. so what I thought was like > > creating the main index as books and then divide the department as type so > > while searching particular book in department we can only go to specific > > department id(type) rather than searching entire index of books. > > > > so just curious about the approach, > > will it degrade more performance in compare to the single type? > > > > Thanks in advanced > > Mohsin > > > > On Tuesday, 13 November 2012 11:49:04 UTC, Igor Motov wrote: > > > > > Why cannot you just use a field for departmentid instead of creating a > > > whole new type? Type is somewhat heavy construct because of mappings that > > > are created, maintained and sent between nodes. > > > > > > On Tuesday, November 13, 2012 5:41:18 AM UTC-5, mohsin husen wrote: > > > > > > > Sorry for the confusion Igor. > > > > suppose i have a index as books and their types are departmentid. > > > > > > > > so if i want to create the mapping for bookid bookname price author > > > > author\_email etc like below > > > > > > > > bookid - string , lowercase > > > > bookname - string, lowercase > > > > price - long > > > > author - string lowercase > > > > author\_email - string, uax\_url\_email tokenizer > > > > > > > > now problem with this mapping is that it is asking for the type while > > > > creating the mapping and settings. > > > > if i will create mapping with departmentid - 1 then whatever goes in > > > > that department has those mappings and settings which we created. > > > > > > > > but now when we add data to second depatmentid -2 it takes by default > > > > values. which is quite right as we havent created mappings and settings for > > > > this type. > > > > > > > > so my question is how to create the mapping and the settings that can > > > > be available for all the types by default. > > > > > > > > let me know if you still have doubt. > > > > > > > > Thanks in advanced. > > > > > > > > On Monday, 12 November 2012 23:22:52 UTC, Igor Motov wrote: > > > > > > > > > Sorry, I don't think I quite understand what you are trying > > > > > to achieve here. If you are looking for some automation in assigning > > > > > mappings to fields, take a look at Dynamic Templates[http://www.elasticsearch.org/guide/reference/mapping/root-object-type.html](http://www.elasticsearch.org/guide/reference/mapping/root-object-type.html)and Index > > > > > Templates API[http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html](http://www.elasticsearch.org/guide/reference/api/admin-indices-templates.html) > > > > > . > > > > > > > > > > On Monday, November 12, 2012 12:43:45 PM UTC-5, mohsin husen wrote: > > > > > > > > > > > Thanks Igor and Ivan... > > > > > > with your guidence i creted the mappings as follows > > > > > > > > > > > > ` > > > > > > curl -XPUT 'http://localhost:9200/test/?pretty=1' -d ' > > > > > > { > > > > > > "mappings" : { > > > > > > "1" : { // this is for type 1 > > > > > > "properties" : { > > > > > > "authorOfAbstracts" : { > > > > > > "type" : "string" > > > > > > }, > > > > > > "firstName" : { > > > > > > "type" : "string","search_analyzer" : > > > > > > "name_analyzer","index_analyzer" : "name_analyzer"}, > > > > > > "lastName" : { > > > > > > "type" : "string","search_analyzer" : > > > > > > "name_analyzer","index_analyzer" : "name_analyzer"}, > > > > > > "email" : { > > > > > > "type" : "string", > > > > > > "search_analyzer" : "email_analyzer", > > > > > > "index_analyzer" : "email_analyzer" > > > > > > } > > > > > > } > > > > > > } > > > > > > }, > > > > > > "settings" : { > > > > > > "analysis" : { > > > > > > "analyzer" : { > > > > > > "name_analyzer" : { > > > > > > "filter" : [ > > > > > > "standard", > > > > > > "lowercase", > > > > > > "asciifolding" > > > > > > ], > > > > > > "type" : "custom", > > > > > > "tokenizer" : "standard" > > > > > > }, > > > > > > "email_analyzer" : { > > > > > > "type" : "custom", > > > > > > "tokenizer" : "uax_url_email" > > > > > > } > > > > > > ` > > > > > > > > > > > > `} > > > > > > }} > > > > > > }' > > > > > > ` > > > > > > > > > > > > now problem with this is when i am adding data to index=test and > > > > > > type=1, its working fine but when i m inserting the data to type=2 its not > > > > > > working as expected to be work with email tokenizer. > > > > > > > > > > > > its shows following mapping... in which we can clearly see for type=1 > > > > > > data will be indexed and searched as per the mapping but what about the > > > > > > type=2 ? > > > > > > can we achieve same mapping in all the type in one index? > > > > > > > > > > > > {"test":{ > > > > > > > > > > > > "2":{"properties":{"authorOfAbstracts":{"type":"string"},"email":{"type":"string"},"firstName":{"type":"string"},"lastName":{"type":"string"}}}, > > > > > > > > > > > > "1":{"properties":{"authorOfAbstracts":{"type":"string"},"email":{"type":"string","analyzer":"email\_analyzer"},"firstName":{"type":"string","analyzer":"name\_analyzer"},"lastName":{"type":"string","analyzer":"name\_analyzer"}}} > > > > > > }} > > > > > > > > > > > > Thanks > > > > > > Mohsin > > > > > > > > > > > > On Friday, 9 November 2012 19:48:11 UTC, Igor Motov wrote: > > > > > > > > > > > > > Standard analyzer splits emails addresses into terms. So > > > > > > > [elasti...@googlegroups.com](mailto:elasti...@googlegroups.com) is indexed as two terms: "elasticsearch" > > > > > > > and "[googlegroups.com](http://googlegroups.com)". The email filter that you > > > > > > > use FilterBuilders.termFilter("email",email) it searching for the entire > > > > > > > email as a single term. There are two ways to solve this problem. One is to > > > > > > > use uax\_url\_email (you can find an example here > > > > > > > [http://stackoverflow.com/questions/13173185/not-analyzed-is-not-working-as-expected · GitHub](https://gist.github.com/3993902)) Another solution is to replace > > > > > > > termFilter with queryFilter(matchPhraseQuery("email", email)). > > > > > > > > > > > > > > On Thursday, November 8, 2012 4:54:05 AM UTC-5, mohsin husen wrote: > > > > > > > > > > > > > > > just to add that i am using default analyzer. > > > > > > > > is it happening due to analyzer ? > > > > > > > > > > > > > > > > please please advise ☹ > > > > > > > > > > > > > > > > Thanks > > > > > > > > Mohsin > > > > > > > > > > > > > > > > On Wednesday, 7 November 2012 10:15:34 UTC, mohsin husen wrote: > > > > > > > > > > > > > > > > > Thanks Ivan for your reply. > > > > > > > > > i have following java code for fetching the data from elasticsearch > > > > > > > > > > > > > > > > > > SearchRequestBuilder searchRequestBuilder = > > > > > > > > > client.prepareSearch(indexName); > > > > > > > > > searchRequestBuilder.setSearchType(SearchType.DEFAULT); > > > > > > > > > searchRequestBuilder.setQuery(boolQuery() > > > > > > > > > .should(textQuery("email", email)) > > > > > > > > > .should(textQuery("firstName", firstName)) > > > > > > > > > .should(textQuery("lastName", lastName)) > > > > > > > > > .should(textQuery("keyword", keyword))); > > > > > > > > > > > > > > > > > > searchRequestBuilder.setFilter(FilterBuilders.andFilter().add(FilterBuilders. > > > > > > > > > termFilter("email",email))); > > > > > > > > > > > > > > > > > > searchRequestBuilder.setFrom(0).setSize(40).setExplain(true); > > > > > > > > > SearchResponse response = null; > > > > > > > > > response = searchRequestBuilder.execute().actionGet(); > > > > > > > > > > > > > > > > > > problem with filter with email here. > > > > > > > > > if i have only email or email with any other field i am getting > > > > > > > > > empty result where as i hava data in email with that text. > > > > > > > > > in other word combination with email is not giving any result if i > > > > > > > > > am applying the filter. > > > > > > > > > > > > > > > > > > please suggest accordingly > > > > > > > > > > > > > > > > > > Thanks in advanced > > > > > > > > > > > > > > > > > > On Tuesday, 6 November 2012 23:22:45 UTC, Ivan Brusic wrote: > > > > > > > > > > > > > > > > > > > The easiest way to to create a custom analyzer that utilizes the > > > > > > > > > > uxl\_url\_email tokenizer and map that analyzer to a field. When that field > > > > > > > > > > is indexed/searched, the analyzer would be used. > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Ivan > > > > > > > > > > > > > > > > > > > > On Tue, Nov 6, 2012 at 5:18 AM, mohsin husen [smh...@gmail.com](mailto:smh...@gmail.com)wrote: > > > > > > > > > > > > > > > > > > > > > Hello > > > > > > > > > > > > > > > > > > > > > > can we do uax\_url\_email tokenizer in java api ? > > > > > > > > > > > if so then how to perform. > > > > > > > > > > > please suggest. > > > > > > > > > > > > > > > > > > > > > > Thanks in advanced > > > > > > > > > > > Mohsin H > > > > > > > > > > > > > > > > > > > > > > -- -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:04am UTC](https://discuss.elastic.co/t/uax-url-email-tokenizer/9581/14 "2017-07-06T03:04:22Z")