# Cross\_fields and boost - Is it feasible? **URL:** \ **Category:** Elasticsearch\ **Created:** [June 1, 2018, 12:14pm UTC](https://discuss.elastic.co/t/cross-fields-and-boost-is-it-feasible/134219 "2018-06-01T12:14:53Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![lapilit](https://avatars.discourse-cdn.com/v4/letter/l/7ba0ec/32.png) [@lapilit](https://discuss.elastic.co/u/lapilit)\ **Post date:** [June 1, 2018, 12:14pm UTC](https://discuss.elastic.co/t/cross-fields-and-boost-is-it-feasible/134219/1 "2018-06-01T12:14:53Z")
Does it work-as-expected in common sense, when boosting individual fields of `cross_fields` query? It looks like the following [page](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-multi-match-query.html#type-cross-fields) states it's impractical to use boosting of cross\_fields query: > Note that cross\_fields is usually only useful on short string fields that all have a boost of 1. Otherwise boosts, term freqs and length normalization contribute to the score in such a way that the blending of term statistics is not meaningful anymore. While the following [page](https://www.elastic.co/blog/multi-field-search-just-got-better) points the boosting as an advantage over "one big field": > Individual fields can be boosted, which can’t be done with the copy\_to field. ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [June 1, 2018, 12:37pm UTC](https://discuss.elastic.co/t/cross-fields-and-boost-is-it-feasible/134219/2 "2018-06-01T12:37:09Z")
Cross fields is designed to select "the right" field for each word. So given a search for "mark" in both the forename and surname fields it will prefer the more-common forename field over the surname. It does this by tweaking the reported DF values for surname:mark so that it is _one doc more_ than the most-likely field of forename. Incrementing DF by one subtly reduces IDF to make surname:mark rank lower than forename:mark. The other words in the query e.g. "harwood" will be still be rewarded more highly than mark because it is a rarer word. All the subtlety of this per-word score-tweaking is undone if you just slap a global boost on a field. ---
**Author:** ![lapilit](https://avatars.discourse-cdn.com/v4/letter/l/7ba0ec/32.png) [@lapilit](https://discuss.elastic.co/u/lapilit)\ **Post date:** [June 1, 2018, 1:18pm UTC](https://discuss.elastic.co/t/cross-fields-and-boost-is-it-feasible/134219/3 "2018-06-01T13:18:56Z")
@Mark_Harwood Thanks for response Generally speaking, you say it's impractical to use `boost` option within `cross_fields` query. **Could you be so kind to confirm this?** If the above is true, then here are the questions: 1. Why `boost` option is formally allowed? 2. Do you think previously mentioned quote misleads the usage of boosting inside `cross_fields` query? [(source page)](https://www.elastic.co/blog/multi-field-search-just-got-better) > Individual fields can be boosted, which can’t be done with the copy\_to field. ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [June 1, 2018, 2:12pm UTC](https://discuss.elastic.co/t/cross-fields-and-boost-is-it-feasible/134219/4 "2018-06-01T14:12:39Z")
> [@lapilit](#): > > Why `boost` option is formally allowed? To support very rare cases where you just want to make use of cross-field's down-playing of rare-but-wrong fields - the original discussion [is here](https://github.com/elastic/elasticsearch/pull/12294#issuecomment-125879989) Personally speaking I was all for throwing errors if users tried to set a boost on a cross\_fields query. ---
**Author:** ![lapilit](https://avatars.discourse-cdn.com/v4/letter/l/7ba0ec/32.png) [@lapilit](https://discuss.elastic.co/u/lapilit)\ **Post date:** [June 4, 2018, 2:32pm UTC](https://discuss.elastic.co/t/cross-fields-and-boost-is-it-feasible/134219/5 "2018-06-04T14:32:17Z")
@Mark_Harwood Thanks for the link to original discussion, I've read it throughout. Still, I want to make sure I get it right: - `cross-field` is absolutely valid to use **on analyzed and/or long-length and/or short-length fields** like "description" (according to @Clinton_Gormley1 's [comment](https://github.com/elastic/elasticsearch/pull/12294#issuecomment-128084157) ) - it's valid to apply `boost` option to thwart the effect of TF and normalization factor (according to @jpountz [comment](https://github.com/elastic/elasticsearch/pull/12294#issuecomment-125879989) ) If the above is true, User-supplied field-level boosts can be used as a tool to give some priorities on the fields, while IDF-scores are even out by `cross-field`. E.g. we search for "Will Smith" and want to give a priority for "description" field over "first name", "last name" and "biography" fields. ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [June 4, 2018, 2:55pm UTC](https://discuss.elastic.co/t/cross-fields-and-boost-is-it-feasible/134219/6 "2018-06-04T14:55:30Z")
In practice, the following statements hold true for most search apps: 1. Users are lazy. We can't be bothered to put the right words into the right input boxes in a structured form so we reach for typing multiple keywords into a single search box. So no useful context is provided for each word. Equally, users are unlikely to specify "boosts" for fields and certainly not per-search-term field boosts. 2. Application developers are lazy. We won't parse the user's search input, understand each search term provided and for each determine which field is the most likely target and provide query-specific boosts for each term and field pair. We won't consider if the sequence of the words is important and if they represent a phrase. We'll just chuck the user's search text at the search engine and hope for the best (sometimes developers will use a global "template" query with boosts for what they consider to be generally important fields but this is not sensitive to the actual search words a user provides). Cross fields is trying to do the analysis that application developers fail to do in 2 - understand the actual query terms being used on a per-search basis and then decide on the boosts required to tailor that specific query. ---
**Author:** ![lapilit](https://avatars.discourse-cdn.com/v4/letter/l/7ba0ec/32.png) [@lapilit](https://discuss.elastic.co/u/lapilit)\ **Post date:** [June 4, 2018, 3:14pm UTC](https://discuss.elastic.co/t/cross-fields-and-boost-is-it-feasible/134219/7 "2018-06-04T15:14:59Z")
@Mark_Harwood I got your point and common case of Enterprise world. I just want to make sure the things mentioned in my previous post are valid. I'm confused... E.g., I read that [original discussion](https://github.com/elastic/elasticsearch/pull/12294#issuecomment-125879989) and still see the following on [Elasticsearch Reference page](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-multi-match-query.html#type-cross-fields): > Note that cross\_fields is usually only useful on short string fields that all have a boost of 1. It seems the above quotation is not quite true... ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [June 4, 2018, 3:21pm UTC](https://discuss.elastic.co/t/cross-fields-and-boost-is-it-feasible/134219/8 "2018-06-04T15:21:55Z")
> [@lapilit](#): > > It seems the above quotation is not quite true... I'm not so sure. "Boost of 1" means the default boost i.e. no alteration to the input score. So the lazy user and lazy developer have both opted to run with defaults. "Short string fields" means things like firstname, surname or address where a search for "john smith" would be really bad if you didn't have cross\_fields tweaking the IDF for you to stop "address:smith street" being ranked top. ---
**Author:** ![lapilit](https://avatars.discourse-cdn.com/v4/letter/l/7ba0ec/32.png) [@lapilit](https://discuss.elastic.co/u/lapilit)\ **Post date:** [June 4, 2018, 3:36pm UTC](https://discuss.elastic.co/t/cross-fields-and-boost-is-it-feasible/134219/9 "2018-06-04T15:36:40Z")
> [@Mark\_Harwood](#): > > "Short string fields" means things like firstname, surname or address where a search for "john smith" would be really bad if you didn't have cross\_fields tweaking the IDF for you to stop "address:smith street" being ranked top. Still I don't see the problem here: We're using `cross_fields`, so "address" field won't be biased (IDF score is even out). Now, I just make "description" field to be a priority field over the rest of the fields. Why can't I benefit from `boost` option in this scenario? ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [June 4, 2018, 3:52pm UTC](https://discuss.elastic.co/t/cross-fields-and-boost-is-it-feasible/134219/10 "2018-06-04T15:52:53Z")
> [@lapilit](#): > > Why can't I benefit from `boost` option in this scenario? The questionable "benefit" of your boost is the ability to provide a "one-size-fits-all-search-terms" multiplier that is not tweaked for each term in the user's input. It does not matter if the user's search, when tokenized, contains what cross\_field can clearly see as both an address and a name - your ham-fisted boost is going to try steer both terms in the direction of a particular field. It's like taking an airhorn to a classical music concert - you can do it but it's undoing some of the subtlety in the process. ---
**Author:** ![lapilit](https://avatars.discourse-cdn.com/v4/letter/l/7ba0ec/32.png) [@lapilit](https://discuss.elastic.co/u/lapilit)\ **Post date:** [June 5, 2018, 11:58am UTC](https://discuss.elastic.co/t/cross-fields-and-boost-is-it-feasible/134219/11 "2018-06-05T11:58:27Z")
@Mark_Harwood Thanks for detailed explanation. It confirms what I already understood. > [@Mark\_Harwood](#): > > All the subtlety of this per-word score-tweaking is undone if you just slap a global boost on a field. **As I see, it's actually not undone, right?** E.g. if user-boost is provided for "first name", it evens out IDF-scores for every query-term and only then applies the boost for "first name" field (for every query-term). At the same time, user-boost is not provided for "last name", so it evens out IDF-scores for "last name" field and the boost "1" remains by default. Also, I still don't get why can't we use `cross_fields` on big-free-text fields. ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [June 5, 2018, 8:46pm UTC](https://discuss.elastic.co/t/cross-fields-and-boost-is-it-feasible/134219/12 "2018-06-05T20:46:13Z")
> [@lapilit](#): > > if user-boost is provided for "first name", My point is user-supplied boosts typically don’t consider if the actual search terms provided in a particular user query are surnames, forenames or a mix of both. They just apply a lazy one size-fits-all policy of “I think field X is great”. (Correct me if I’m wrong in my assumption about your app). Such a boosting policy undoes the minute scoring changes cross-fields does to understand each term in the query and favour what has been determined as the right field context for that word. Like I say - you’re free to do what you want though so if you want to go ahead and supply your own boost, more power to you. ---
**Author:** ![lapilit](https://avatars.discourse-cdn.com/v4/letter/l/7ba0ec/32.png) [@lapilit](https://discuss.elastic.co/u/lapilit)\ **Post date:** [June 7, 2018, 8:28am UTC](https://discuss.elastic.co/t/cross-fields-and-boost-is-it-feasible/134219/13 "2018-06-07T08:28:58Z")
> [@Mark\_Harwood](#): > > They just apply a lazy one size-fits-all policy of “I think field X is great”. (Correct me if I’m wrong in my assumption about your app) In our app, neither end-user nor developer knows which specific query-term is appropriate for specific column, even if developer is looking at some query by hand. End-user's "friendly-query" is searched over "internal/very technical" columns. > [@Mark\_Harwood](#): > > We can't be bothered to put the right words into the right input boxes in a structured form so we reach for typing multiple keywords into a single search box. So as I see, the case you wrote is different from our app's case mentioned above: there is no relation between mimic "input boxes" and "most-likely column"; at the same time, we'd like to even out IDF-scores between columns. ---
**Author:** ![Mark\_Harwood](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mark_harwood/32/10538_2.png) [@Mark\_Harwood](https://discuss.elastic.co/u/Mark_Harwood)\ **Post date:** [June 7, 2018, 9:07am UTC](https://discuss.elastic.co/t/cross-fields-and-boost-is-it-feasible/134219/14 "2018-06-07T09:07:53Z")
> [@lapilit](#): > > there is no relation between mimic "input boxes" and "most-likely column". Sorry, I don't know what you mean by "mimic input boxes". My assumption is your app is like most search apps - there's a single search input box into which people type one or more words. If so, then it sounds like what you have been asking for is based on the following principles: 1. We have a favourite field we always want to boost regardless of what a user typed. 2. We make no attempt to favour which field each individual search term naturally belongs to 3. We like to think some of the search words in a query are more important than others If you didn't want rule 3) then you could wrap each search term in a `constant_score` query to dispense with IDF completely - because your app knows best (see rule 1) However, I assume you want rule 3 because users can type a mix of terms in a query eg (`rareWord, commonWord`) and given the choice you prefer docs to match `rareWord` rather than only match `commonWord`. To have this preference you need to know which of these words is rarer - using IDF. The key question is _which field do you use to obtain this IDF_? The sensible answer is "the most likely one" which is what cross-fields does (other search types typically favour the least likely). So, bringing this all together - you propose a system that prefers `rareWord` over `commonWord` based on an understanding of "correct" context (thanks to cross fields) but your rule 1 will just boost "hits in field X" because you think that's a generally good thing to do for all queries, _regardless_ of the context of each search term. ---
**Author:** ![lapilit](https://avatars.discourse-cdn.com/v4/letter/l/7ba0ec/32.png) [@lapilit](https://discuss.elastic.co/u/lapilit)\ **Post date:** [June 7, 2018, 12:42pm UTC](https://discuss.elastic.co/t/cross-fields-and-boost-is-it-feasible/134219/15 "2018-06-07T12:42:33Z")
Yes, this is how I see my system. Thanks! I've advanced my understanding of how cross fields works. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2018, 12:42pm UTC](https://discuss.elastic.co/t/cross-fields-and-boost-is-it-feasible/134219/16 "2018-07-05T12:42:35Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.