# Advanced scoring of muti field searching (only count a token once) **URL:** \ **Category:** Elasticsearch\ **Created:** [June 24, 2013, 11:04pm UTC](https://discuss.elastic.co/t/advanced-scoring-of-muti-field-searching-only-count-a-token-once/12530 "2013-06-24T23:04:44Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![mtthrok](https://avatars.discourse-cdn.com/v4/letter/m/8c91f0/32.png) [@mtthrok](https://discuss.elastic.co/u/mtthrok)\ **Post date:** [June 24, 2013, 11:04pm UTC](https://discuss.elastic.co/t/advanced-scoring-of-muti-field-searching-only-count-a-token-once/12530/1 "2013-06-24T23:04:44Z")
Hi, I'm trying to create a search engine over users that searches over people's names and other metadata like where they work (so a user could query "tim cook apple"). I have a mapping schema (pasted below) with a first\_name, last\_name, and affils column (which will usually be a list of affiliations like "apple"). Each field is indexed with the full tokens as well as an additional "partial" field that has ngrams. To query against it, I was originally using a bool/should that hit each of the fields (like in [http://elasticsearch-users.115913.n3.nabble.com/help-needed-with-the-query-td3177477.html#a3178856](http://elasticsearch-users.115913.n3.nabble.com/help-needed-with-the-query-td3177477.html#a3178856)). The issue is that if someone was named "Tim Cook" and worked at "Tim Cook Design" they would come up to the top, even for "Tim Cook Apple". In our case, we really only want to count each token to the score once so that Tim Cook at Tim Cook Design scores as well as Tim Cook at Apple for the query "Tim Cook." I switched to a query\_string query (pasted below) which does better but still is giving more weight to those cases (now "Tim Cook Apple" works, but "Tim Cook" still gives more weight to the one at Tim Cook Deisgn) . So, how can I customize the scoring of query\_search (or another multi field query) to only let a token contribue to the score in one field? Thanks! I'm pretty new to elasticsearch/lucene, so sorry if this is obvious. =========== Mapping setup: curl -XPUT '[http://127.0.0.1:9200/people\_search/?pretty=1](http://127.0.0.1:9200/people_search/?pretty=1)' -d ' { "mappings" : { "person" : { "properties" : { "\_id" : { "type" : "integer", "index" : "no" }, "last\_name" : { "fields" : { "partial" : { "search\_analyzer" : "full\_name", "index\_analyzer" : "partial\_name", "type" : "string" }, "last\_name" : { "type" : "string", "analyzer" : "full\_name" } }, "type" : "multi\_field" }, "first\_name" : { "fields" : { "partial" : { "search\_analyzer" : "partial\_name\_search", "index\_analyzer" : "partial\_name", "type" : "string" }, "first\_name" : { "type" : "string", "analyzer" : "full\_name" } }, "type" : "multi\_field" }, "affils" : { "fields" : { "partial" : { "search\_analyzer" : "full\_name", "index\_analyzer" : "partial\_name", "type" : "string" }, "names" : { "type" : "string", "analyzer" : "full\_name" } }, "type" : "multi\_field" } ``` } } ``` }, "settings" : { "analysis" : { "filter" : { "name\_ngrams" : { "side" : "front", "max\_gram" : 10, "min\_gram" : 2, "type" : "edgeNGram" }, "name\_ngrams\_search" : { "side" : "front", "max\_gram" : 10, "min\_gram" : 2, "type" : "edgeNGram" } }, "analyzer" : { "full\_name" : { "filter" : [ "standard", "lowercase", "asciifolding" ], "type" : "custom", "tokenizer" : "standard" }, "partial\_name" : { "filter" : [ "standard", "lowercase", "asciifolding", "name\_ngrams" ], "type" : "custom", "tokenizer" : "standard" }, "partial\_name\_search" : { "filter" : [ "standard", "lowercase", "asciifolding", "name\_ngrams\_search" ], "type" : "custom", "tokenizer" : "standard" } ``` } } ``` } } ' ============= Insert some data curl -XPOST '[http://127.0.0.1:9200/\_bulk?pretty=1](http://127.0.0.1:9200/_bulk?pretty=1)' -d ' {"index" : {"\_index" : "people\_search", "\_type" : "person", "\_id" : 1}} {"\_id" : 1, "last\_name" : "Cook", "first\_name" : "Tim", "affils":["Apple"]} {"index" : {"\_index" : "people\_search", "\_type" : "person", "\_id" : 2}} {"\_id" : 2, "last\_name" : "Cook", "first\_name" : "Tim", "affils":["Tim Cook Design", "Random co"]} ' =============== Query: curl -XPOST 'people\_search/person/\_search?search\_type=dfs\_query\_then\_fetch' -d" {'query': {'query\_string': {'fields': ['first\_name.partial', 'first\_name.first\_name^1.5', 'last\_name.partial', 'last\_name.last\_name^1.5', 'affils.partial', 'affils.names^1.5'], 'query': 'Tim Cook', 'use\_dis\_max': True}}} " -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![mtthrok](https://avatars.discourse-cdn.com/v4/letter/m/8c91f0/32.png) [@mtthrok](https://discuss.elastic.co/u/mtthrok)\ **Post date:** [June 24, 2013, 11:37pm UTC](https://discuss.elastic.co/t/advanced-scoring-of-muti-field-searching-only-count-a-token-once/12530/2 "2013-06-24T23:37:08Z")
I guess I could create a new indexed field that uses an analyzer that returns the unique tokens from the first\_name, last\_name, and affils fields and just query against that. Is that the best way to do it? On Monday, June 24, 2013 4:04:44 PM UTC-7, [mtt...@gmail.com](mailto:mtt...@gmail.com) wrote: > Hi, > > I'm trying to create a search engine over users that searches over > people's names and other metadata like where they work (so a user could > query "tim cook apple"). I have a mapping schema (pasted below) with a > first\_name, last\_name, and affils column (which will usually be a list of > affiliations like "apple"). Each field is indexed with the full tokens as > well as an additional "partial" field that has ngrams. > > To query against it, I was originally using a bool/should that hit each of > the fields (like in > [http://elasticsearch-users.115913.n3.nabble.com/help-needed-with-the-query-td3177477.html#a3178856](http://elasticsearch-users.115913.n3.nabble.com/help-needed-with-the-query-td3177477.html#a3178856)). > The issue is that if someone was named "Tim Cook" and worked at "Tim Cook > Design" they would come up to the top, even for "Tim Cook Apple". In our > case, we really only want to count each token to the score once so that Tim > Cook at Tim Cook Design scores as well as Tim Cook at Apple for the query > "Tim Cook." I switched to a query\_string query (pasted below) which does > better but still is giving more weight to those cases (now "Tim Cook Apple" > works, but "Tim Cook" still gives more weight to the one at Tim Cook > Deisgn) . > > So, how can I customize the scoring of query\_search (or another multi > field query) to only let a token contribue to the score in one field? > > Thanks! I'm pretty new to elasticsearch/lucene, so sorry if this is > obvious. > > =========== > Mapping setup: > curl -XPUT '[http://127.0.0.1:9200/people\_search/?pretty=1](http://127.0.0.1:9200/people_search/?pretty=1)' -d ' > { > "mappings" : { > "person" : { > "properties" : { > "\_id" : { > "type" : "integer", > "index" : "no" > }, > "last\_name" : { > "fields" : { > "partial" : { > "search\_analyzer" : "full\_name", > "index\_analyzer" : "partial\_name", > "type" : "string" > }, > "last\_name" : { > "type" : "string", > "analyzer" : "full\_name" > } > }, > "type" : "multi\_field" > }, > "first\_name" : { > "fields" : { > "partial" : { > "search\_analyzer" : "partial\_name\_search", > "index\_analyzer" : "partial\_name", > "type" : "string" > }, > "first\_name" : { > "type" : "string", > "analyzer" : "full\_name" > } > }, > "type" : "multi\_field" > }, > "affils" : { > "fields" : { > "partial" : { > "search\_analyzer" : "full\_name", > "index\_analyzer" : "partial\_name", > "type" : "string" > }, > "names" : { > "type" : "string", > "analyzer" : "full\_name" > } > }, > "type" : "multi\_field" > } > > ``` > } > } > > ``` > > }, > "settings" : { > "analysis" : { > "filter" : { > "name\_ngrams" : { > "side" : "front", > "max\_gram" : 10, > "min\_gram" : 2, > "type" : "edgeNGram" > }, > "name\_ngrams\_search" : { > "side" : "front", > "max\_gram" : 10, > "min\_gram" : 2, > "type" : "edgeNGram" > } > }, > "analyzer" : { > "full\_name" : { > "filter" : [ > "standard", > "lowercase", > "asciifolding" > ], > "type" : "custom", > "tokenizer" : "standard" > }, > "partial\_name" : { > "filter" : [ > "standard", > "lowercase", > "asciifolding", > "name\_ngrams" > ], > "type" : "custom", > "tokenizer" : "standard" > }, > "partial\_name\_search" : { > "filter" : [ > "standard", > "lowercase", > "asciifolding", > "name\_ngrams\_search" > ], > "type" : "custom", > "tokenizer" : "standard" > } > > ``` > } > } > > ``` > > } > } > ' > > ============= > Insert some data > curl -XPOST '[http://127.0.0.1:9200/\_bulk?pretty=1](http://127.0.0.1:9200/_bulk?pretty=1)' -d ' > {"index" : {"\_index" : "people\_search", "\_type" : "person", "\_id" : 1}} > {"\_id" : 1, "last\_name" : "Cook", "first\_name" : "Tim", "affils":["Apple"]} > {"index" : {"\_index" : "people\_search", "\_type" : "person", "\_id" : 2}} > {"\_id" : 2, "last\_name" : "Cook", "first\_name" : "Tim", "affils":["Tim > Cook Design", "Random co"]} > ' > > =============== > Query: > curl -XPOST > 'people\_search/person/\_search?search\_type=dfs\_query\_then\_fetch' -d" > {'query': {'query\_string': {'fields': ['first\_name.partial', > 'first\_name.first\_name^1.5', > 'last\_name.partial', > 'last\_name.last\_name^1.5', > 'affils.partial', > 'affils.names^1.5'], > 'query': 'Tim Cook', > 'use\_dis\_max': True}}} > " -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![germap](https://avatars.discourse-cdn.com/v4/letter/g/4da419/32.png) [@germap](https://discuss.elastic.co/u/germap)\ **Post date:** [August 20, 2014, 2:38pm UTC](https://discuss.elastic.co/t/advanced-scoring-of-muti-field-searching-only-count-a-token-once/12530/3 "2014-08-20T14:38:11Z")
Hi, just for the record, did you finally manage to solve this problem? The docs [1] say the default behavior ("best\_fields") for multi\_match queries is to only take the highest score into account, you can configure it to include the score from other fields though. I was facing a similar issue and could overcome it by boosting the most relevant field with a "^2". Regards, ## Germán [1] [Elastic — The Search AI Company | Elastic](http://www.elasticsearch.org/blog/multi-field-search-just-got-better/) El lunes, 24 de junio de 2013 18:37:08 UTC-5, [mtt...@gmail.com](mailto:mtt...@gmail.com) escribió: > I guess I could create a new indexed field that uses an analyzer that > returns the unique tokens from the first\_name, last\_name, and affils fields > and just query against that. Is that the best way to do it? > > On Monday, June 24, 2013 4:04:44 PM UTC-7, [mtt...@gmail.com](mailto:mtt...@gmail.com) wrote: > > > Hi, > > > > I'm trying to create a search engine over users that searches over > > people's names and other metadata like where they work (so a user could > > query "tim cook apple"). I have a mapping schema (pasted below) with a > > first\_name, last\_name, and affils column (which will usually be a list of > > affiliations like "apple"). Each field is indexed with the full tokens as > > well as an additional "partial" field that has ngrams. > > > > To query against it, I was originally using a bool/should that hit each > > of the fields (like in > > [http://elasticsearch-users.115913.n3.nabble.com/help-needed-with-the-query-td3177477.html#a3178856](http://elasticsearch-users.115913.n3.nabble.com/help-needed-with-the-query-td3177477.html#a3178856)). > > The issue is that if someone was named "Tim Cook" and worked at "Tim Cook > > Design" they would come up to the top, even for "Tim Cook Apple". In our > > case, we really only want to count each token to the score once so that Tim > > Cook at Tim Cook Design scores as well as Tim Cook at Apple for the query > > "Tim Cook." I switched to a query\_string query (pasted below) which does > > better but still is giving more weight to those cases (now "Tim Cook Apple" > > works, but "Tim Cook" still gives more weight to the one at Tim Cook > > Deisgn) . > > > > So, how can I customize the scoring of query\_search (or another multi > > field query) to only let a token contribue to the score in one field? > > > > Thanks! I'm pretty new to elasticsearch/lucene, so sorry if this is > > obvious. > > > > =========== > > Mapping setup: > > curl -XPUT '[http://127.0.0.1:9200/people\_search/?pretty=1](http://127.0.0.1:9200/people_search/?pretty=1)' -d ' > > { > > "mappings" : { > > "person" : { > > "properties" : { > > "\_id" : { > > "type" : "integer", > > "index" : "no" > > }, > > "last\_name" : { > > "fields" : { > > "partial" : { > > "search\_analyzer" : "full\_name", > > "index\_analyzer" : "partial\_name", > > "type" : "string" > > }, > > "last\_name" : { > > "type" : "string", > > "analyzer" : "full\_name" > > } > > }, > > "type" : "multi\_field" > > }, > > "first\_name" : { > > "fields" : { > > "partial" : { > > "search\_analyzer" : "partial\_name\_search", > > "index\_analyzer" : "partial\_name", > > "type" : "string" > > }, > > "first\_name" : { > > "type" : "string", > > "analyzer" : "full\_name" > > } > > }, > > "type" : "multi\_field" > > }, > > "affils" : { > > "fields" : { > > "partial" : { > > "search\_analyzer" : "full\_name", > > "index\_analyzer" : "partial\_name", > > "type" : "string" > > }, > > "names" : { > > "type" : "string", > > "analyzer" : "full\_name" > > } > > }, > > "type" : "multi\_field" > > } > > > > ``` > > } > > } > > > > ``` > > > > }, > > "settings" : { > > "analysis" : { > > "filter" : { > > "name\_ngrams" : { > > "side" : "front", > > "max\_gram" : 10, > > "min\_gram" : 2, > > "type" : "edgeNGram" > > }, > > "name\_ngrams\_search" : { > > "side" : "front", > > "max\_gram" : 10, > > "min\_gram" : 2, > > "type" : "edgeNGram" > > } > > }, > > "analyzer" : { > > "full\_name" : { > > "filter" : [ > > "standard", > > "lowercase", > > "asciifolding" > > ], > > "type" : "custom", > > "tokenizer" : "standard" > > }, > > "partial\_name" : { > > "filter" : [ > > "standard", > > "lowercase", > > "asciifolding", > > "name\_ngrams" > > ], > > "type" : "custom", > > "tokenizer" : "standard" > > }, > > "partial\_name\_search" : { > > "filter" : [ > > "standard", > > "lowercase", > > "asciifolding", > > "name\_ngrams\_search" > > ], > > "type" : "custom", > > "tokenizer" : "standard" > > } > > > > ``` > > } > > } > > > > ``` > > > > } > > } > > ' > > > > ============= > > Insert some data > > curl -XPOST '[http://127.0.0.1:9200/\_bulk?pretty=1](http://127.0.0.1:9200/_bulk?pretty=1)' -d ' > > {"index" : {"\_index" : "people\_search", "\_type" : "person", "\_id" : 1}} > > {"\_id" : 1, "last\_name" : "Cook", "first\_name" : "Tim", > > "affils":["Apple"]} > > {"index" : {"\_index" : "people\_search", "\_type" : "person", "\_id" : 2}} > > {"\_id" : 2, "last\_name" : "Cook", "first\_name" : "Tim", "affils":["Tim > > Cook Design", "Random co"]} > > ' > > > > =============== > > Query: > > curl -XPOST > > 'people\_search/person/\_search?search\_type=dfs\_query\_then\_fetch' -d" > > {'query': {'query\_string': {'fields': ['first\_name.partial', > > 'first\_name.first\_name^1.5', > > 'last\_name.partial', > > 'last\_name.last\_name^1.5', > > 'affils.partial', > > 'affils.names^1.5'], > > 'query': 'Tim Cook', > > 'use\_dis\_max': True}}} > > " -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/eaeb62d6-beea-4e53-a1f3-7fc3ce972145%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/eaeb62d6-beea-4e53-a1f3-7fc3ce972145%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:07am UTC](https://discuss.elastic.co/t/advanced-scoring-of-muti-field-searching-only-count-a-token-once/12530/4 "2017-07-06T01:07:25Z")