# Custom\_score requires specific attribute ordering **URL:** \ **Category:** Elasticsearch\ **Created:** [November 30, 2010, 11:00am UTC](https://discuss.elastic.co/t/custom-score-requires-specific-attribute-ordering/3610 "2010-11-30T11:00:05Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Timber](https://avatars.discourse-cdn.com/v4/letter/t/5daacb/32.png) [@Timber](https://discuss.elastic.co/u/Timber)\ **Post date:** [November 30, 2010, 11:00am UTC](https://discuss.elastic.co/t/custom-score-requires-specific-attribute-ordering/3610/1 "2010-11-30T11:00:05Z")
Hi, I have the following query fragment which works fine in ES 0.13.0: ``` "query" : { "custom_score" : { "filtered" : { "query" : { "bool" : { "must_not" : { "term" : { "group.vip" : "T" } }, "must" : { "match_all" : { } } } }, "params" : { "categoryId" : "77205" }, "lang" : "js", "script" : "parseFloat(eval(doc['catSequences'].value) ``` [categoryId])", "filter" : { "term" : { "category.pathIds" : "77205" } } ``` } } } ``` But if the "script" element comes after the "filter" object then the I get the error: QueryParsingException[[books] [custom\_score] requires 'script' field]; Unfortunately I don't have any control over the order object attributes are generated ( I don't think that order is significant in json). My question is should Elastic Search not care about the order or do I change my json library? Thanks Tim ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [November 30, 2010, 11:55am UTC](https://discuss.elastic.co/t/custom-score-requires-specific-attribute-ordering/3610/2 "2010-11-30T11:55:22Z")
Hi Time You are not nesting your query correctly. You have: "query": { "custom\_score": { "filtered": { "query": {...}, "params": {...}, "langs": {...}, "script": {...}, "filter": {...} It should be: "query": { "custom\_score": { "query": { "filtered": { "filter": {..}, "query": {...}} }, "params": {...}, "langs": {...}, "script": {...}, See here for more [http://www.elasticsearch.com/docs/elasticsearch/rest\_api/query\_dsl/custom\_score\_query](http://www.elasticsearch.com/docs/elasticsearch/rest_api/query_dsl/custom_score_query) Also, consider using constant\_score instead of match\_all: [http://www.elasticsearch.com/docs/elasticsearch/rest\_api/query\_dsl/constant\_score\_query](http://www.elasticsearch.com/docs/elasticsearch/rest_api/query_dsl/constant_score_query) clint ---
**Author:** ![Timber](https://avatars.discourse-cdn.com/v4/letter/t/5daacb/32.png) [@Timber](https://discuss.elastic.co/u/Timber)\ **Post date:** [November 30, 2010, 12:45pm UTC](https://discuss.elastic.co/t/custom-score-requires-specific-attribute-ordering/3610/3 "2010-11-30T12:45:44Z")
Thanks for pointing that out. The query is now working fine ``` "query" : { "custom_score" : { "query" : { "bool" : { "must_not" : { "term" : { "group.vip" : "T" } }, "must" : { "constant_score" : { "filter" : { "term" : { "category.pathIds" : "77705" } } } } } }, "script" : "parseFloat(eval(doc['catSequences'].value) ``` [categoryId])", "params" : { "categoryId" : 77705 }, "lang" : "js" } On Nov 30, 11:55 am, Clinton Gormley [clin...@iannounce.co.uk](mailto:clin...@iannounce.co.uk) wrote: > Hi Time > > You are not nesting your query correctly. > > You have: > > "query": { > "custom\_score": { > "filtered": { > "query": {...}, > "params": {...}, > "langs": {...}, > "script": {...}, > "filter": {...} > > It should be: > > "query": { > "custom\_score": { > "query": { > "filtered": { "filter": {..}, "query": {...}} > }, > "params": {...}, > "langs": {...}, > "script": {...}, > > See here for morehttp://www.elasticsearch.com/docs/elasticsearch/rest\_api/query\_dsl/cu... > > Also, consider using constant\_score instead of match\_all:[http://www.elasticsearch.com/docs/elasticsearch/rest\_api/query\_dsl/co](http://www.elasticsearch.com/docs/elasticsearch/rest_api/query_dsl/co)... > > clint ---
**Author:** ![Clinton\_Gormley](https://avatars.discourse-cdn.com/v4/letter/c/50afbb/32.png) [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)\ **Post date:** [November 30, 2010, 1:08pm UTC](https://discuss.elastic.co/t/custom-score-requires-specific-attribute-ordering/3610/4 "2010-11-30T13:08:29Z")
On Tue, 2010-11-30 at 04:45 -0800, Timber wrote: > Thanks for pointing that out. The query is now working fine You can reduce that further to this: "query" : { "custom\_score" : { "query" : { "bool" : { "must\_not" : { "term" : { "group.vip" : "T" } }, "must" : { "term" : { "category.pathIds" : "77705" } } } }, "script" : "parseFloat(eval(doc['catSequences'].value)[categoryId])", "params" : { "categoryId" : 77705 }, "lang" : "js" } } clint ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [November 30, 2010, 9:25pm UTC](https://discuss.elastic.co/t/custom-score-requires-specific-attribute-ordering/3610/5 "2010-11-30T21:25:41Z")
Hi, Great that its working. I am not sure I got the script you are executing, what are you trying to do? (just want to see if it can be optimized). On Tue, Nov 30, 2010 at 3:08 PM, Clinton Gormley [clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk)wrote: > On Tue, 2010-11-30 at 04:45 -0800, Timber wrote: > > > Thanks for pointing that out. The query is now working fine > > You can reduce that further to this: > > "query" : { > "custom\_score" : { > "query" : { > "bool" : { > "must\_not" : { > "term" : { > "group.vip" : "T" > } > }, > "must" : { > "term" : { > "category.pathIds" : "77705" > } > } > } > }, > "script" : > "parseFloat(eval(doc['catSequences'].value)[categoryId])", > "params" : { > "categoryId" : 77705 > }, > "lang" : "js" > } > } > > clint ---
**Author:** ![Timber](https://avatars.discourse-cdn.com/v4/letter/t/5daacb/32.png) [@Timber](https://discuss.elastic.co/u/Timber)\ **Post date:** [November 30, 2010, 10:33pm UTC](https://discuss.elastic.co/t/custom-score-requires-specific-attribute-ordering/3610/6 "2010-11-30T22:33:22Z")
Hi Shay, I am trying too allow the products in each of our categories to be ordered in a category specific order. The first page of two of products in each category browsed are to be displayed in an order determined by our marketing team. When a search term is used we just use relevance. The catSequences field is indexed as "catSequences": "({76715:373.0,48124:141.0,76704:62.0,48122:325.0,76710:21.0})" which puts each product in an ordered position in the category. This is obviously a sparse mapping. On a side note I had no luck using groovy as the scripting language. Thanks Tim On Nov 30, 9:25 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > Hi, > > Great that its working. I am not sure I got the script you are executing, > what are you trying to do? (just want to see if it can be optimized). > > On Tue, Nov 30, 2010 at 3:08 PM, Clinton Gormley [clin...@iannounce.co.uk](mailto:clin...@iannounce.co.uk)wrote: > > > On Tue, 2010-11-30 at 04:45 -0800, Timber wrote: > > > > > Thanks for pointing that out. The query is now working fine > > > You can reduce that further to this: > > > "query" : { > > "custom\_score" : { > > "query" : { > > "bool" : { > > "must\_not" : { > > "term" : { > > "group.vip" : "T" > > } > > }, > > "must" : { > > "term" : { > > "category.pathIds" : "77705" > > } > > } > > } > > }, > > "script" : > > "parseFloat(eval(doc['catSequences'].value)[categoryId])", > > "params" : { > > "categoryId" : 77705 > > }, > > "lang" : "js" > > } > > } > > > clint ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [December 1, 2010, 12:43am UTC](https://discuss.elastic.co/t/custom-score-requires-specific-attribute-ordering/3610/7 "2010-12-01T00:43:24Z")
I think I understand, not sure I understand the script though (maybe gmail garbled it up). What was the problem with groovy? Also, you can sort on several fields, not just one, maybe that would help as well? On Wed, Dec 1, 2010 at 12:33 AM, Timber [tim.berkeley@gmail.com](mailto:tim.berkeley@gmail.com) wrote: > Hi Shay, > > I am trying too allow the products in each of our categories to be > ordered in a category specific order. The first page of two of > products in each category browsed are to be displayed in an order > determined by our marketing team. When a search term is used we just > use relevance. > > The catSequences field is indexed as "catSequences": > "({76715:373.0,48124:141.0,76704:62.0,48122:325.0,76710:21.0})" which > puts each product in an ordered position in the category. This is > obviously a sparse mapping. > > On a side note I had no luck using groovy as the scripting language. > > Thanks > > Tim > > On Nov 30, 9:25 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > > > Hi, > > > > Great that its working. I am not sure I got the script you are > > executing, > > what are you trying to do? (just want to see if it can be optimized). > > > > On Tue, Nov 30, 2010 at 3:08 PM, Clinton Gormley \< > > [clin...@iannounce.co.uk](mailto:clin...@iannounce.co.uk)\>wrote: > > > > > On Tue, 2010-11-30 at 04:45 -0800, Timber wrote: > > > > > > > Thanks for pointing that out. The query is now working fine > > > > > You can reduce that further to this: > > > > > "query" : { > > > "custom\_score" : { > > > "query" : { > > > "bool" : { > > > "must\_not" : { > > > "term" : { > > > "group.vip" : "T" > > > } > > > }, > > > "must" : { > > > "term" : { > > > "category.pathIds" : "77705" > > > } > > > } > > > } > > > }, > > > "script" : > > > "parseFloat(eval(doc['catSequences'].value)[categoryId])", > > > "params" : { > > > "categoryId" : 77705 > > > }, > > > "lang" : "js" > > > } > > > } > > > > > clint ---
**Author:** ![Timber](https://avatars.discourse-cdn.com/v4/letter/t/5daacb/32.png) [@Timber](https://discuss.elastic.co/u/Timber)\ **Post date:** [December 1, 2010, 9:45am UTC](https://discuss.elastic.co/t/custom-score-requires-specific-attribute-ordering/3610/8 "2010-12-01T09:45:44Z")
Hi, The script parseFloat(eval(doc['catSequences'].value)[categoryId]) takes the value of catSequences for the product: ({76715:373.0,48124:141.0,76704:62.0,48122:325.0,76710:21.0}) and evals it so that we can index the object by the currently selected categoryId. This gives us a value for the score. 62.0 when categoryId is set to 76704 Life would be so much simpler if a product only existed in a single category or the display order was not determined by the category! Extracting the score value may be faster using string parsing rather than eval, I have not tested this yet. Still pretty much prototyping. We already use sorting on multiple fields, once the category entries are beyond the first few pages there is no position set so we sort the rest by best selling. The problem with groovy is I get the error [script\_lang not supported [groovy]] when I set "lang" : "groovy". It looks like the plugin has installed correctly, I have a lang-groovy directory in plugins which contains the two jars. Haven't spent too much time looking into this yet. On Dec 1, 12:43 am, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > I think I understand, not sure I understand the script though (maybe gmail > garbled it up). What was the problem with groovy? > > Also, you can sort on several fields, not just one, maybe that would help as > well? > > On Wed, Dec 1, 2010 at 12:33 AM, Timber [tim.berke...@gmail.com](mailto:tim.berke...@gmail.com) wrote: > > > Hi Shay, > > > I am trying too allow the products in each of our categories to be > > ordered in a category specific order. The first page of two of > > products in each category browsed are to be displayed in an order > > determined by our marketing team. When a search term is used we just > > use relevance. > > > The catSequences field is indexed as "catSequences": > > "({76715:373.0,48124:141.0,76704:62.0,48122:325.0,76710:21.0})" which > > puts each product in an ordered position in the category. This is > > obviously a sparse mapping. > > > On a side note I had no luck using groovy as the scripting language. > > > Thanks > > > Tim > > > On Nov 30, 9:25 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > > > > > Hi, > > > > Great that its working. I am not sure I got the script you are > > > executing, > > > what are you trying to do? (just want to see if it can be optimized). > > > > On Tue, Nov 30, 2010 at 3:08 PM, Clinton Gormley \< > > > [clin...@iannounce.co.uk](mailto:clin...@iannounce.co.uk)\>wrote: > > > > > On Tue, 2010-11-30 at 04:45 -0800, Timber wrote: > > > > > > > > > Thanks for pointing that out. The query is now working fine > > > > > You can reduce that further to this: > > > > > "query" : { > > > > "custom\_score" : { > > > > "query" : { > > > > "bool" : { > > > > "must\_not" : { > > > > "term" : { > > > > "group.vip" : "T" > > > > } > > > > }, > > > > "must" : { > > > > "term" : { > > > > "category.pathIds" : "77705" > > > > } > > > > } > > > > } > > > > }, > > > > "script" : > > > > "parseFloat(eval(doc['catSequences'].value)[categoryId])", > > > > "params" : { > > > > "categoryId" : 77705 > > > > }, > > > > "lang" : "js" > > > > } > > > > } > > > > > clint ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [December 2, 2010, 4:49pm UTC](https://discuss.elastic.co/t/custom-score-requires-specific-attribute-ordering/3610/9 "2010-12-02T16:49:38Z")
Got it. Strange regarding the groovy part not working, has a test for that. When you have time, it would be great if you can gist a test case. On Wed, Dec 1, 2010 at 11:45 AM, Timber [tim.berkeley@gmail.com](mailto:tim.berkeley@gmail.com) wrote: > Hi, > > The script parseFloat(eval(doc['catSequences'].value)[categoryId]) > > takes the value of catSequences for the product: > > ({76715:373.0,48124:141.0,76704:62.0,48122:325.0,76710:21.0}) > > and evals it so that we can index the object by the currently selected > categoryId. This gives us a value for the score. 62.0 when categoryId > is set to 76704 > > Life would be so much simpler if a product only existed in a single > category or the display order was not determined by the category! > > Extracting the score value may be faster using string parsing rather > than eval, I have not tested this yet. Still pretty much prototyping. > > We already use sorting on multiple fields, once the category entries > are beyond the first few pages there is no position set so we sort the > rest by best selling. > > The problem with groovy is I get the error [script\_lang not supported > [groovy]] when I set "lang" : "groovy". It looks like the plugin has > installed correctly, I have a lang-groovy directory in plugins which > contains the two jars. Haven't spent too much time looking into this > yet. > > On Dec 1, 12:43 am, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > > > I think I understand, not sure I understand the script though (maybe > > gmail > > garbled it up). What was the problem with groovy? > > > > Also, you can sort on several fields, not just one, maybe that would help > > as > > well? > > > > On Wed, Dec 1, 2010 at 12:33 AM, Timber [tim.berke...@gmail.com](mailto:tim.berke...@gmail.com) wrote: > > > > > Hi Shay, > > > > > I am trying too allow the products in each of our categories to be > > > ordered in a category specific order. The first page of two of > > > products in each category browsed are to be displayed in an order > > > determined by our marketing team. When a search term is used we just > > > use relevance. > > > > > The catSequences field is indexed as "catSequences": > > > "({76715:373.0,48124:141.0,76704:62.0,48122:325.0,76710:21.0})" which > > > puts each product in an ordered position in the category. This is > > > obviously a sparse mapping. > > > > > On a side note I had no luck using groovy as the scripting language. > > > > > Thanks > > > > > Tim > > > > > On Nov 30, 9:25 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > > > > > > > Hi, > > > > > > Great that its working. I am not sure I got the script you are > > > > executing, > > > > what are you trying to do? (just want to see if it can be optimized). > > > > > > On Tue, Nov 30, 2010 at 3:08 PM, Clinton Gormley \< > > > > [clin...@iannounce.co.uk](mailto:clin...@iannounce.co.uk)\>wrote: > > > > > > > On Tue, 2010-11-30 at 04:45 -0800, Timber wrote: > > > > > > > > > > > Thanks for pointing that out. The query is now working fine > > > > > > > You can reduce that further to this: > > > > > > > "query" : { > > > > > "custom\_score" : { > > > > > "query" : { > > > > > "bool" : { > > > > > "must\_not" : { > > > > > "term" : { > > > > > "group.vip" : "T" > > > > > } > > > > > }, > > > > > "must" : { > > > > > "term" : { > > > > > "category.pathIds" : "77705" > > > > > } > > > > > } > > > > > } > > > > > }, > > > > > "script" : > > > > > "parseFloat(eval(doc['catSequences'].value)[categoryId])", > > > > > "params" : { > > > > > "categoryId" : 77705 > > > > > }, > > > > > "lang" : "js" > > > > > } > > > > > } > > > > > > > clint ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:15am UTC](https://discuss.elastic.co/t/custom-score-requires-specific-attribute-ordering/3610/10 "2017-07-06T04:15:50Z")