# Custom Filters Score Help **URL:** \ **Category:** Elasticsearch\ **Created:** [November 14, 2012, 1:54pm UTC](https://discuss.elastic.co/t/custom-filters-score-help/9715 "2012-11-14T13:54:48Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Brandon\_Hilkert](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brandon_hilkert/32/2182_2.png) [@Brandon\_Hilkert](https://discuss.elastic.co/u/Brandon_Hilkert)\ **Post date:** [November 14, 2012, 1:54pm UTC](https://discuss.elastic.co/t/custom-filters-score-help/9715/1 "2012-11-14T13:54:48Z")
I'm using a custom filters score query to boost results that have terms fb\_connections in common. I'm finding that results with 4 terms in common in the fb\_connections field aren't necessarily weighted higher than those with 1 common fb\_connection. Is there a way with this query to boost it multiple times for the count of common term matches? { "query": { "filtered": { "query": { "custom\_filters\_score": { "query": { "query\_string": { "query": "Apple" } }, "filters": [ { "filter": { "terms": { "fb\_connections": [ "100", "101" ] } }, "boost": 5 } ], "score\_mode": "total" } }, "filter": { "and": [ { "term": { "is\_active": true } }, { "not": { "terms": { "id": [ 1 ] } } }, { "not": { "terms": { "fb\_user\_id": [ "100", "101" ] } } } ] } } }, "size": 8, "fields": [ "id", "fb\_user\_id", "linkedin\_id" ] } -- ---
**Author:** ![simonw\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/simonw_2/32/1130_2.png) [@simonw\_2](https://discuss.elastic.co/u/simonw_2)\ **Post date:** [November 15, 2012, 9:22am UTC](https://discuss.elastic.co/t/custom-filters-score-help/9715/2 "2012-11-15T09:22:55Z")
hey, when you say terms in common do you refer to the terms in your filter? ie. you filter for '100','101', '102' so if a document has all 3 of them you want a higher boost than for documents that only have 2 of those terms? simon On Wednesday, November 14, 2012 2:54:48 PM UTC+1, Brandon Hilkert wrote: > I'm using a custom filters score query to boost results that have terms > fb\_connections in common. I'm finding that results with 4 terms in common > in the fb\_connections field aren't necessarily weighted higher than those > with 1 common fb\_connection. Is there a way with this query to boost it > multiple times for the count of common term matches? > > { > "query": { > "filtered": { > "query": { > "custom\_filters\_score": { > "query": { > "query\_string": { > "query": "Apple" > } > }, > "filters": [ > { > "filter": { > "terms": { > "fb\_connections": [ > "100", > "101" > ] > } > }, > "boost": 5 > } > ], > "score\_mode": "total" > } > }, > "filter": { > "and": [ > { > "term": { > "is\_active": true > } > }, > { > "not": { > "terms": { > "id": [ > 1 > ] > } > } > }, > { > "not": { > "terms": { > "fb\_user\_id": [ > "100", > "101" > ] > } > } > } > ] > } > } > }, > "size": 8, > "fields": [ > "id", > "fb\_user\_id", > "linkedin\_id" > ] > } -- ---
**Author:** ![Brandon\_Hilkert](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brandon_hilkert/32/2182_2.png) [@Brandon\_Hilkert](https://discuss.elastic.co/u/Brandon_Hilkert)\ **Post date:** [November 15, 2012, 10:58am UTC](https://discuss.elastic.co/t/custom-filters-score-help/9715/3 "2012-11-15T10:58:11Z")
Exactly. On Nov 15, 2012, at 4:23 AM, simonw [simon.willnauer@elasticsearch.com](mailto:simon.willnauer@elasticsearch.com) wrote: hey, when you say terms in common do you refer to the terms in your filter? ie. you filter for '100','101', '102' so if a document has all 3 of them you want a higher boost than for documents that only have 2 of those terms? simon On Wednesday, November 14, 2012 2:54:48 PM UTC+1, Brandon Hilkert wrote: > I'm using a custom filters score query to boost results that have terms > fb\_connections in common. I'm finding that results with 4 terms in common > in the fb\_connections field aren't necessarily weighted higher than those > with 1 common fb\_connection. Is there a way with this query to boost it > multiple times for the count of common term matches? > > { > "query": { > "filtered": { > "query": { > "custom\_filters\_score": { > "query": { > "query\_string": { > "query": "Apple" > } > }, > "filters": [ > { > "filter": { > "terms": { > "fb\_connections": [ > "100", > "101" > ] > } > }, > "boost": 5 > } > ], > "score\_mode": "total" > } > }, > "filter": { > "and": [ > { > "term": { > "is\_active": true > } > }, > { > "not": { > "terms": { > "id": [ > 1 > ] > } > } > }, > { > "not": { > "terms": { > "fb\_user\_id": [ > "100", > "101" > ] > } > } > } > ] > } > } > }, > "size": 8, > "fields": [ > "id", > "fb\_user\_id", > "linkedin\_id" > ] > } > > -- -- ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [November 15, 2012, 7:45pm UTC](https://discuss.elastic.co/t/custom-filters-score-help/9715/4 "2012-11-15T19:45:48Z")
Your filter in custom\_filters\_score is "binary" it ether matches and then the document gets a boost of 5, or it doesn't and the document doesn't get any boost. To have the behavior that Simon described you need to have multiple filters (one for each term) and use "total" or "multiply" score mode to accumulate boosts from all matching filters. On Thursday, November 15, 2012 5:58:22 AM UTC-5, Brandon Hilkert wrote: > Exactly. > > On Nov 15, 2012, at 4:23 AM, simonw \<[simon.w...@elasticsearch.com](mailto:simon.w...@elasticsearch.com)\\> > wrote: > > hey, > > when you say terms in common do you refer to the terms in your filter? ie. > you filter for '100','101', '102' so if a document has all 3 of them you > want a higher boost than for documents that only have 2 of those terms? > > simon > > On Wednesday, November 14, 2012 2:54:48 PM UTC+1, Brandon Hilkert wrote: > > > I'm using a custom filters score query to boost results that have terms > > fb\_connections in common. I'm finding that results with 4 terms in common > > in the fb\_connections field aren't necessarily weighted higher than those > > with 1 common fb\_connection. Is there a way with this query to boost it > > multiple times for the count of common term matches? > > > > { > > "query": { > > "filtered": { > > "query": { > > "custom\_filters\_score": { > > "query": { > > "query\_string": { > > "query": "Apple" > > } > > }, > > "filters": [ > > { > > "filter": { > > "terms": { > > "fb\_connections": [ > > "100", > > "101" > > ] > > } > > }, > > "boost": 5 > > } > > ], > > "score\_mode": "total" > > } > > }, > > "filter": { > > "and": [ > > { > > "term": { > > "is\_active": true > > } > > }, > > { > > "not": { > > "terms": { > > "id": [ > > 1 > > ] > > } > > } > > }, > > { > > "not": { > > "terms": { > > "fb\_user\_id": [ > > "100", > > "101" > > ] > > } > > } > > } > > ] > > } > > } > > }, > > "size": 8, > > "fields": [ > > "id", > > "fb\_user\_id", > > "linkedin\_id" > > ] > > } > > > > -- -- ---
**Author:** ![Brandon\_Hilkert](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brandon_hilkert/32/2182_2.png) [@Brandon\_Hilkert](https://discuss.elastic.co/u/Brandon_Hilkert)\ **Post date:** [November 15, 2012, 7:53pm UTC](https://discuss.elastic.co/t/custom-filters-score-help/9715/5 "2012-11-15T19:53:33Z")
Ahh....ok. That makes sense. I'll give that a shot. Thanks! On Nov 15, 2012, at 2:45 PM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote: Your filter in custom\_filters\_score is "binary" it ether matches and then the document gets a boost of 5, or it doesn't and the document doesn't get any boost. To have the behavior that Simon described you need to have multiple filters (one for each term) and use "total" or "multiply" score mode to accumulate boosts from all matching filters. On Thursday, November 15, 2012 5:58:22 AM UTC-5, Brandon Hilkert wrote: > Exactly. > > On Nov 15, 2012, at 4:23 AM, simonw \<[simon.w...@elasticsearch.com](mailto:simon.w...@elasticsearch.com)\\> > wrote: > > hey, > > when you say terms in common do you refer to the terms in your filter? ie. > you filter for '100','101', '102' so if a document has all 3 of them you > want a higher boost than for documents that only have 2 of those terms? > > simon > > On Wednesday, November 14, 2012 2:54:48 PM UTC+1, Brandon Hilkert wrote: > > > I'm using a custom filters score query to boost results that have terms > > fb\_connections in common. I'm finding that results with 4 terms in common > > in the fb\_connections field aren't necessarily weighted higher than those > > with 1 common fb\_connection. Is there a way with this query to boost it > > multiple times for the count of common term matches? > > > > { > > "query": { > > "filtered": { > > "query": { > > "custom\_filters\_score": { > > "query": { > > "query\_string": { > > "query": "Apple" > > } > > }, > > "filters": [ > > { > > "filter": { > > "terms": { > > "fb\_connections": [ > > "100", > > "101" > > ] > > } > > }, > > "boost": 5 > > } > > ], > > "score\_mode": "total" > > } > > }, > > "filter": { > > "and": [ > > { > > "term": { > > "is\_active": true > > } > > }, > > { > > "not": { > > "terms": { > > "id": [ > > 1 > > ] > > } > > } > > }, > > { > > "not": { > > "terms": { > > "fb\_user\_id": [ > > "100", > > "101" > > ] > > } > > } > > } > > ] > > } > > } > > }, > > "size": 8, > > "fields": [ > > "id", > > "fb\_user\_id", > > "linkedin\_id" > > ] > > } > > > > -- > > -- -- ---
**Author:** ![Brandon\_Hilkert](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/brandon_hilkert/32/2182_2.png) [@Brandon\_Hilkert](https://discuss.elastic.co/u/Brandon_Hilkert)\ **Post date:** [November 27, 2012, 7:33pm UTC](https://discuss.elastic.co/t/custom-filters-score-help/9715/6 "2012-11-27T19:33:31Z")
As a follow up to my initial question, I implemented iterating through the connections to a boost times the number of matching terms. Everything works, but these are social connections (LinkedIn, Facebook, Twitter,...). I found today that some of our productions users have quite a few connections, so I'm getting the following error: \*nested: TooManyClauses[maxClauseCount is set to 1024]; \* \* \* Is there a better way to go about searching for a string, and boosting results with common terms based on the number of common terms? On Thu, Nov 15, 2012 at 2:53 PM, Brandon Hilkert [brandon@meeteor.com](mailto:brandon@meeteor.com)wrote: > Ahh....ok. That makes sense. I'll give that a shot. Thanks! > > On Nov 15, 2012, at 2:45 PM, Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com) wrote: > > Your filter in custom\_filters\_score is "binary" it ether matches and then > the document gets a boost of 5, or it doesn't and the document doesn't get > any boost. To have the behavior that Simon described you need to > have multiple filters (one for each term) and use "total" or "multiply" > score mode to accumulate boosts from all matching filters. > > On Thursday, November 15, 2012 5:58:22 AM UTC-5, Brandon Hilkert wrote: > > > Exactly. > > > > On Nov 15, 2012, at 4:23 AM, simonw [simon.w...@elasticsearch.com](mailto:simon.w...@elasticsearch.com) wrote: > > > > hey, > > > > when you say terms in common do you refer to the terms in your filter? > > ie. you filter for '100','101', '102' so if a document has all 3 of them > > you want a higher boost than for documents that only have 2 of those terms? > > > > simon > > > > On Wednesday, November 14, 2012 2:54:48 PM UTC+1, Brandon Hilkert wrote: > > > > > I'm using a custom filters score query to boost results that have terms > > > fb\_connections in common. I'm finding that results with 4 terms in common > > > in the fb\_connections field aren't necessarily weighted higher than those > > > with 1 common fb\_connection. Is there a way with this query to boost it > > > multiple times for the count of common term matches? > > > > > > { > > > "query": { > > > "filtered": { > > > "query": { > > > "custom\_filters\_score": { > > > "query": { > > > "query\_string": { > > > "query": "Apple" > > > } > > > }, > > > "filters": [ > > > { > > > "filter": { > > > "terms": { > > > "fb\_connections": [ > > > "100", > > > "101" > > > ] > > > } > > > }, > > > "boost": 5 > > > } > > > ], > > > "score\_mode": "total" > > > } > > > }, > > > "filter": { > > > "and": [ > > > { > > > "term": { > > > "is\_active": true > > > } > > > }, > > > { > > > "not": { > > > "terms": { > > > "id": [ > > > 1 > > > ] > > > } > > > } > > > }, > > > { > > > "not": { > > > "terms": { > > > "fb\_user\_id": [ > > > "100", > > > "101" > > > ] > > > } > > > } > > > } > > > ] > > > } > > > } > > > }, > > > "size": 8, > > > "fields": [ > > > "id", > > > "fb\_user\_id", > > > "linkedin\_id" > > > ] > > > } > > > > > > -- > > > > -- -- ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [November 27, 2012, 7:46pm UTC](https://discuss.elastic.co/t/custom-filters-score-help/9715/7 "2012-11-27T19:46:42Z")
First of all, you can increase the maxClauseCount using indices.query.bool.max\_clause\_count setting. But to answer you question, depending on the number of connections and number of search results, it might make sense to try switching to custom\_score and boost connections by comparing connection id with a hashmap of connection that you want to boost. For a few connections, it will be slower than custom\_filters\_score but for users with vary large number of connections it might be a faster solution. On Tuesday, November 27, 2012 2:33:31 PM UTC-5, Brandon Hilkert wrote: > As a follow up to my initial question, I implemented iterating through the > connections to a boost times the number of matching terms. Everything > works, but these are social connections (LinkedIn, Facebook, Twitter,...). > I found today that some of our productions users have quite a few > connections, so I'm getting the following error: > > \*nested: TooManyClauses[maxClauseCount is set to 1024]; \* > \* > \* > Is there a better way to go about searching for a string, and boosting > results with common terms based on the number of common terms? > > On Thu, Nov 15, 2012 at 2:53 PM, Brandon Hilkert \<[bra...@meeteor.com](mailto:bra...@meeteor.com)\ > > > wrote: > > > Ahh....ok. That makes sense. I'll give that a shot. Thanks! > > > > On Nov 15, 2012, at 2:45 PM, Igor Motov \<[imo...@gmail.com](mailto:imo...@gmail.com) \\> > > wrote: > > > > Your filter in custom\_filters\_score is "binary" it ether matches and then > > the document gets a boost of 5, or it doesn't and the document doesn't get > > any boost. To have the behavior that Simon described you need to > > have multiple filters (one for each term) and use "total" or "multiply" > > score mode to accumulate boosts from all matching filters. > > > > On Thursday, November 15, 2012 5:58:22 AM UTC-5, Brandon Hilkert wrote: > > > > > Exactly. > > > > > > On Nov 15, 2012, at 4:23 AM, simonw [simon.w...@elasticsearch.com](mailto:simon.w...@elasticsearch.com) > > > wrote: > > > > > > hey, > > > > > > when you say terms in common do you refer to the terms in your filter? > > > ie. you filter for '100','101', '102' so if a document has all 3 of them > > > you want a higher boost than for documents that only have 2 of those terms? > > > > > > simon > > > > > > On Wednesday, November 14, 2012 2:54:48 PM UTC+1, Brandon Hilkert wrote: > > > > > > > I'm using a custom filters score query to boost results that have terms > > > > fb\_connections in common. I'm finding that results with 4 terms in common > > > > in the fb\_connections field aren't necessarily weighted higher than those > > > > with 1 common fb\_connection. Is there a way with this query to boost it > > > > multiple times for the count of common term matches? > > > > > > > > { > > > > "query": { > > > > "filtered": { > > > > "query": { > > > > "custom\_filters\_score": { > > > > "query": { > > > > "query\_string": { > > > > "query": "Apple" > > > > } > > > > }, > > > > "filters": [ > > > > { > > > > "filter": { > > > > "terms": { > > > > "fb\_connections": [ > > > > "100", > > > > "101" > > > > ] > > > > } > > > > }, > > > > "boost": 5 > > > > } > > > > ], > > > > "score\_mode": "total" > > > > } > > > > }, > > > > "filter": { > > > > "and": [ > > > > { > > > > "term": { > > > > "is\_active": true > > > > } > > > > }, > > > > { > > > > "not": { > > > > "terms": { > > > > "id": [ > > > > 1 > > > > ] > > > > } > > > > } > > > > }, > > > > { > > > > "not": { > > > > "terms": { > > > > "fb\_user\_id": [ > > > > "100", > > > > "101" > > > > ] > > > > } > > > > } > > > > } > > > > ] > > > > } > > > > } > > > > }, > > > > "size": 8, > > > > "fields": [ > > > > "id", > > > > "fb\_user\_id", > > > > "linkedin\_id" > > > > ] > > > > } > > > > > > > > -- > > > > > > -- -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:02am UTC](https://discuss.elastic.co/t/custom-filters-score-help/9715/8 "2017-07-06T03:02:35Z")