# Trying to Solve Relevancy and Performance Problem **URL:** \ **Category:** Elasticsearch\ **Created:** [May 8, 2013, 5:38am UTC](https://discuss.elastic.co/t/trying-to-solve-relevancy-and-performance-problem/11868 "2013-05-08T05:38:56Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![Accu\_Search](https://avatars.discourse-cdn.com/v4/letter/a/e0b2c6/32.png) [@Accu\_Search](https://discuss.elastic.co/u/Accu_Search)\ **Post date:** [May 8, 2013, 5:38am UTC](https://discuss.elastic.co/t/trying-to-solve-relevancy-and-performance-problem/11868/1 "2013-05-08T05:38:56Z")
Currently I have around 10 million records indexed, which mainly is user meta data like firstname and lastname with following mapping: _Field Mapping:_ "user" : {"\_source" : { "enabled" : true }, "properties" : { "uniqueid" : { "type" : "string", "index" : "not\_analyzed"}, "fname" : { "type" : "string", "index" : "analyzed", "analyzer" : "standard" }, "lname" : { "type" : "string", "index" : "analyzed", "analyzer" : "standard" } . . . } } I currently have a user lookup feature which lets users type in one or more words to lookup user meta data without specifying what field is it searching for. It is sort to Prefix search. For example user can search for "Chris Mos" and the expected results should be in following order: 1. Chris Mos - Record that has fname as Chris and lname as Mos 2. Chris SomeLastName - Records that have fname as Chris 3. SomeFirstName Mos - Records that have lname as Mos 4. Christopher Mosby - Records henceforth should have either Chris or Mos in prefix I am firing following query which 95% of the time satisfies the criteria 1 but rest of the results dont necessary follow the order of criteria defined: {explain:"false", query:{"query\_string" : {"query" : "Chris\* Mos\*","fields" : ["fname ^1.5", "lname ^1.0"],"use\_dis\_max" : true,"analyze\_wildcard" : true}}} Couple of problems that i am trying to solve here: 1. _Relevancy_ - Is there a better way to query elastic search or given the scenario should i be using different analyzer?I t 2. _Performance_ - With records growing at very fast pace, i feel that doing wild card search isn't the best approach. Is there an alternate way in which i can query or if i can make use of other analyzers? I tried nGram analyzer before but the issue i was facing was that it was also returning records which had "Chris" as mid-text. For example records with fname as "a_chris_ty" was also returned. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\ **Post date:** [May 8, 2013, 7:56am UTC](https://discuss.elastic.co/t/trying-to-solve-relevancy-and-performance-problem/11868/2 "2013-05-08T07:56:40Z")
Try with Edgengram. Also have a look at multimatch query instead of query string. And don't use wildcards. Here is what you can read in doc: "Note this query can be slow, as it needs to iterate over many terms." -- David 😉 Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs Le 8 mai 2013 à 07:38, Accu Search [accusearch83@gmail.com](mailto:accusearch83@gmail.com) a écrit : Currently I have around 10 million records indexed, which mainly is user meta data like firstname and lastname with following mapping: Field Mapping: "user" : {"\_source" : { "enabled" : true }, "properties" : { "uniqueid" : { "type" : "string", "index" : "not\_analyzed"}, "fname" : { "type" : "string", "index" : "analyzed", "analyzer" : "standard" }, "lname" : { "type" : "string", "index" : "analyzed", "analyzer" : "standard" } . . . } } I currently have a user lookup feature which lets users type in one or more words to lookup user meta data without specifying what field is it searching for. It is sort to Prefix search. For example user can search for "Chris Mos" and the expected results should be in following order: 1. Chris Mos - Record that has fname as Chris and lname as Mos 2. Chris SomeLastName - Records that have fname as Chris 3. SomeFirstName Mos - Records that have lname as Mos 4. Christopher Mosby - Records henceforth should have either Chris or Mos in prefix I am firing following query which 95% of the time satisfies the criteria 1 but rest of the results dont necessary follow the order of criteria defined: {explain:"false", query:{"query\_string" : {"query" : "Chris\* Mos\*","fields" : ["fname ^1.5", "lname ^1.0"],"use\_dis\_max" : true,"analyze\_wildcard" : true}}} Couple of problems that i am trying to solve here: 1. Relevancy - Is there a better way to query elastic search or given the scenario should i be using different analyzer?I t 2. Performance - With records growing at very fast pace, i feel that doing wild card search isn't the best approach. Is there an alternate way in which i can query or if i can make use of other analyzers? I tried nGram analyzer before but the issue i was facing was that it was also returning records which had "Chris" as mid-text. For example records with fname as "achristy" was also returned. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Accu\_Search](https://avatars.discourse-cdn.com/v4/letter/a/e0b2c6/32.png) [@Accu\_Search](https://discuss.elastic.co/u/Accu_Search)\ **Post date:** [May 9, 2013, 5:25am UTC](https://discuss.elastic.co/t/trying-to-solve-relevancy-and-performance-problem/11868/3 "2013-05-09T05:25:04Z")
Thanks David. One more question, one of the future enhancement would be something like auto suggest and minimum characters for auto suggest is 2. Hence i am planning to set min gram size to 2 but i am in conundrum for max gram size. Given that i am indexing names, so i was thinking of settting max size to 40, is setting max gram to high value recommended? Or is it usually closer to min gram size? On Wednesday, May 8, 2013 12:56:40 AM UTC-7, David Pilato wrote: > Try with Edgengram. > Also have a look at multimatch query instead of query string. > And don't use wildcards. Here is what you can read in doc: "Note this > query can be slow, as it needs to iterate over many terms." > > -- > David 😉 > Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs > > Le 8 mai 2013 à 07:38, Accu Search \<[accuse...@gmail.com](mailto:accuse...@gmail.com) \\> a > écrit : > > Currently I have around 10 million records indexed, which mainly is user > meta data like firstname and lastname with following mapping: > > _Field Mapping:_ > > "user" : {"\_source" : { "enabled" : true }, > "properties" : { > "uniqueid" : { "type" : "string", "index" : "not\_analyzed"}, > "fname" : { "type" : "string", "index" : "analyzed", > "analyzer" : "standard" }, > "lname" : { "type" : "string", "index" : "analyzed", > "analyzer" : "standard" } > . > . > . > } > } > > I currently have a user lookup feature which lets users type in one or > more words to lookup user meta data without specifying what field is it > searching for. It is sort to Prefix search. For example user can search > for "Chris Mos" and the expected results should be in following order: > > 1. Chris Mos - Record that has fname as Chris and lname as Mos > > 2. Chris SomeLastName - Records that have fname as Chris > > 3. SomeFirstName Mos - Records that have lname as Mos > > 4. Christopher Mosby - Records henceforth should have either Chris or Mos > in prefix > > I am firing following query which 95% of the time satisfies the criteria 1 > but rest of the results dont necessary follow the order of criteria defined: > > {explain:"false", query:{"query\_string" : {"query" : "Chris\* > Mos\*","fields" : ["fname ^1.5", "lname ^1.0"],"use\_dis\_max" : > true,"analyze\_wildcard" : true}}} > > Couple of problems that i am trying to solve here: > > 1. _Relevancy_ - Is there a better way to query Elasticsearch or given > the scenario should i be using different analyzer?I t > > 2. _Performance_ - With records growing at very fast pace, i feel that > doing wild card search isn't the best approach. Is there an alternate way > in which i can query or if i can make use of other analyzers? I tried > nGram analyzer before but the issue i was facing was that it was also > returning records which had "Chris" as mid-text. For example records with > fname as "a_chris_ty" was also returned. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:37am UTC](https://discuss.elastic.co/t/trying-to-solve-relevancy-and-performance-problem/11868/4 "2017-07-06T02:37:27Z")