# "Inject" external value for sorting

**URL:** <https://discuss.elastic.co/t/inject-external-value-for-sorting/7686>\
**Category:** Elasticsearch\
**Created:** [May 14, 2012, 2:34pm UTC](https://discuss.elastic.co/t/inject-external-value-for-sorting/7686 "2012-05-14T14:34:37Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Vadim\_Voituk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vadim_voituk/32/2841_2.png) [@Vadim\_Voituk](https://discuss.elastic.co/u/Vadim_Voituk)\
**Post date:** [May 14, 2012, 2:34pm UTC](https://discuss.elastic.co/t/inject-external-value-for-sorting/7686/1 "2012-05-14T14:34:37Z")

</div>

Hello,

What i'm trying ot implement is an ES-plugin, that "inject" some kind of  
external value (number) into document during search.  
Then this value will should be used in a sorting expression ("sort script").

My plugin register a native script, that is used in "script\_fields" to  
create additional "virtual" field.  
But the problem there, that "script\_fields" are calculated on a "returnable  
top" part of result-set.  
So such type of fields can't be used as part of sorting criteria.

Also "injecting" in sort a script (that is OK for my particular case) not  
works too, since doc(), source() and fields() return read-only collections.

Another one approach i'm thinking about is - to extend the mvel namespace  
with own, custom-made function, that will return this "external" value, and  
can be used in an sorting script like:

```
"script": " doc.score*2 + myCustomFunction(doc._id.value) "

```

But i can't find the proper way to extend mvel engine for now.

I've found a similar question but there are no solution published there:  
[https://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/custom$20scoring/elasticsearch/tukeqyZdy7I/W\_dZFy0b5ecJ](https://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/custom%2420scoring/elasticsearch/tukeqyZdy7I/W_dZFy0b5ecJ)

What is the right way to solve my problem?

Thanks in advance for any ideas, comments and suggestions.

---

<div class="post-metadata">

**Author:** ![narinder\_izap](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/narinder_izap/32/803_2.png) [@narinder\_izap](https://discuss.elastic.co/u/narinder_izap)\
**Post date:** [May 15, 2012, 4:12am UTC](https://discuss.elastic.co/t/inject-external-value-for-sorting/7686/2 "2012-05-15T04:12:25Z")

</div>

You can try with the cusom\_filters\_score at :

> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.

and you can assign the value to the document based on some specific field  
value. I hope its very good solution for your problem, if implemented  
properly.

On Monday, 14 May 2012 20:04:37 UTC+5:30, Vadim Voituk wrote:

> Hello,
> 
> What i'm trying ot implement is an ES-plugin, that "inject" some kind of  
> external value (number) into document during search.  
> Then this value will should be used in a sorting expression ("sort  
> script").
> 
> My plugin register a native script, that is used in "script\_fields" to  
> create additional "virtual" field.  
> But the problem there, that "script\_fields" are calculated on a  
> "returnable top" part of result-set.  
> So such type of fields can't be used as part of sorting criteria.
> 
> Also "injecting" in sort a script (that is OK for my particular case) not  
> works too, since doc(), source() and fields() return read-only collections.
> 
> Another one approach i'm thinking about is - to extend the mvel namespace  
> with own, custom-made function, that will return this "external" value, and  
> can be used in an sorting script like:
> 
> ```
> "script": " doc.score*2 + myCustomFunction(doc._id.value) "
> 
> ```
> 
> But i can't find the proper way to extend mvel engine for now.
> 
> I've found a similar question but there are no solution published there:
> 
> [Redirecting to Google Groups](https://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/custom$20scoring/elasticsearch/tukeqyZdy7I/W_dZFy0b5ecJ)
> 
> What is the right way to solve my problem?
> 
> Thanks in advance for any ideas, comments and suggestions.

---

<div class="post-metadata">

**Author:** ![Vadim\_Voituk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vadim_voituk/32/2841_2.png) [@Vadim\_Voituk](https://discuss.elastic.co/u/Vadim_Voituk)\
**Post date:** [May 15, 2012, 7:31am UTC](https://discuss.elastic.co/t/inject-external-value-for-sorting/7686/3 "2012-05-15T07:31:00Z")

</div>

Hello, thanks for your answer,  
but i don't really understand how to add custom calculated field to the  
document using custom\_filters\_scrore.

Could you please describe bit more about this?

On Tuesday, May 15, 2012 6:12:25 AM UTC+2, Narinder Kaur wrote:

> You can try with the cusom\_filters\_score at :
> 
> [http://www.elasticsearch.org/guide/reference/query-dsl/custom-filters-score-query.html](http://www.elasticsearch.org/guide/reference/query-dsl/custom-filters-score-query.html)
> 
> and you can assign the value to the document based on some specific field  
> value. I hope its very good solution for your problem, if implemented  
> properly.
> 
> On Monday, 14 May 2012 20:04:37 UTC+5:30, Vadim Voituk wrote:
> 
> > Hello,
> > 
> > What i'm trying ot implement is an ES-plugin, that "inject" some kind of  
> > external value (number) into document during search.  
> > Then this value will should be used in a sorting expression ("sort  
> > script").
> > 
> > My plugin register a native script, that is used in "script\_fields" to  
> > create additional "virtual" field.  
> > But the problem there, that "script\_fields" are calculated on a  
> > "returnable top" part of result-set.  
> > So such type of fields can't be used as part of sorting criteria.
> > 
> > Also "injecting" in sort a script (that is OK for my particular case) not  
> > works too, since doc(), source() and fields() return read-only collections.
> > 
> > Another one approach i'm thinking about is - to extend the mvel namespace  
> > with own, custom-made function, that will return this "external" value, and  
> > can be used in an sorting script like:
> > 
> > ```
> > "script": " doc.score*2 + myCustomFunction(doc._id.value) "
> > 
> > ```
> > 
> > But i can't find the proper way to extend mvel engine for now.
> > 
> > I've found a similar question but there are no solution published there:
> > 
> > [Redirecting to Google Groups](https://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/custom$20scoring/elasticsearch/tukeqyZdy7I/W_dZFy0b5ecJ)
> > 
> > What is the right way to solve my problem?
> > 
> > Thanks in advance for any ideas, comments and suggestions.

---

<div class="post-metadata">

**Author:** ![Klerisson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/klerisson/32/2878_2.png) [@Klerisson](https://discuss.elastic.co/u/Klerisson)\
**Post date:** [May 17, 2012, 1:10am UTC](https://discuss.elastic.co/t/inject-external-value-for-sorting/7686/4 "2012-05-17T01:10:55Z")

</div>

Just implemented a similar feature, but through a bad coding workaround.

I got the search response in JAVA application, than I performed the custom  
score calculation based on the highlighted fields, after that I stored the  
documents in a sorted list (PriorityQueue) and, finally, sent the response  
to the client.

The overall performance is not good at all, but still enough to my  
environment.

Regards,

--  
View this message in context: [http://elasticsearch-users.115913.n3.nabble.com/Inject-external-value-for-sorting-tp3985874p3998283.html](http://elasticsearch-users.115913.n3.nabble.com/Inject-external-value-for-sorting-tp3985874p3998283.html)  
Sent from the ElasticSearch Users mailing list archive at [Nabble.com](http://Nabble.com).

---

<div class="post-metadata">

**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\
**Post date:** [May 17, 2012, 8:29pm UTC](https://discuss.elastic.co/t/inject-external-value-for-sorting/7686/5 "2012-05-17T20:29:49Z")

</div>

Sorry, but I did not understand the question..., script\_fields are fields  
that only affect the top N hits you return, not the whole doc set. You  
probably want to have a script (can be native) that you then use with  
custom\_score query?

On Mon, May 14, 2012 at 5:34 PM, Vadim Voituk [vadim.voituk@gmail.com](mailto:vadim.voituk@gmail.com)wrote:

> Hello,
> 
> What i'm trying ot implement is an ES-plugin, that "inject" some kind of  
> external value (number) into document during search.  
> Then this value will should be used in a sorting expression ("sort  
> script").
> 
> My plugin register a native script, that is used in "script\_fields" to  
> create additional "virtual" field.  
> But the problem there, that "script\_fields" are calculated on a  
> "returnable top" part of result-set.  
> So such type of fields can't be used as part of sorting criteria.
> 
> Also "injecting" in sort a script (that is OK for my particular case) not  
> works too, since doc(), source() and fields() return read-only collections.
> 
> Another one approach i'm thinking about is - to extend the mvel namespace  
> with own, custom-made function, that will return this "external" value, and  
> can be used in an sorting script like:
> 
> ```
> "script": " doc.score*2 + myCustomFunction(doc._id.value) "
> 
> ```
> 
> But i can't find the proper way to extend mvel engine for now.
> 
> I've found a similar question but there are no solution published there:
> 
> [Redirecting to Google Groups](https://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/custom$20scoring/elasticsearch/tukeqyZdy7I/W_dZFy0b5ecJ)
> 
> What is the right way to solve my problem?
> 
> Thanks in advance for any ideas, comments and suggestions.

---

<div class="post-metadata">

**Author:** ![Vadim\_Voituk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vadim_voituk/32/2841_2.png) [@Vadim\_Voituk](https://discuss.elastic.co/u/Vadim_Voituk)\
**Post date:** [May 22, 2012, 8:35am UTC](https://discuss.elastic.co/t/inject-external-value-for-sorting/7686/6 "2012-05-22T08:35:59Z")

</div>

The problem with custom\_score is that it overwrites original score value  
calculated by ES.  
But i want my external "sorting value" to be available in a sorting formula  
together with original score and etc.

Something like that  
sort: {  
\_script: {  
type: number,  
script: doc.score_0.34123+ externalScore_0.1232  
}

Where float numbers are generated on client side (for example depending of  
user settings), right before sending query to ES, but the "externalScore" -  
it's my external value.

I've implemented this in a way like:

- I'm receiving the ranking formula from as an parameter to my native  
script
- Injecting externalScore value to mvel scope
- Passing it to MvelScriptEngine for further processing

From client-side it looks like:  
sort: {  
\_script: {  
language: native,  
type: number,  
script: my\_custom\_sorting\_script,  
params: {  
// ... some script specific params  
script: doc.score_0.34123+ externalScore_0.1232 // just a string  
param processed by mvel script engine  
}  
}

It works pretty fast, but i have a strong feeling that there are more  
elegant way to do this.  
Maybe some kind of plugin endpoint, i've missed?

On Thursday, May 17, 2012 10:29:49 PM UTC+2, kimchy wrote:

> Sorry, but I did not understand the question..., script\_fields are fields  
> that only affect the top N hits you return, not the whole doc set. You  
> probably want to have a script (can be native) that you then use with  
> custom\_score query?
> 
> On Mon, May 14, 2012 at 5:34 PM, Vadim Voituk wrote:
> 
> > Hello,
> > 
> > What i'm trying ot implement is an ES-plugin, that "inject" some kind of  
> > external value (number) into document during search.  
> > Then this value will should be used in a sorting expression ("sort  
> > script").
> > 
> > My plugin register a native script, that is used in "script\_fields" to  
> > create additional "virtual" field.  
> > But the problem there, that "script\_fields" are calculated on a  
> > "returnable top" part of result-set.  
> > So such type of fields can't be used as part of sorting criteria.
> > 
> > Also "injecting" in sort a script (that is OK for my particular case) not  
> > works too, since doc(), source() and fields() return read-only collections.
> > 
> > Another one approach i'm thinking about is - to extend the mvel namespace  
> > with own, custom-made function, that will return this "external" value, and  
> > can be used in an sorting script like:
> > 
> > ```
> > "script": " doc.score*2 + myCustomFunction(doc._id.value) "
> > 
> > ```
> > 
> > But i can't find the proper way to extend mvel engine for now.
> > 
> > I've found a similar question but there are no solution published there:
> > 
> > [Redirecting to Google Groups](https://groups.google.com/forum/?fromgroups#!searchin/elasticsearch/custom$20scoring/elasticsearch/tukeqyZdy7I/W_dZFy0b5ecJ)
> > 
> > What is the right way to solve my problem?
> > 
> > Thanks in advance for any ideas, comments and suggestions.

---

<div class="post-metadata">

**Author:** ![Vadim\_Voituk](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vadim_voituk/32/2841_2.png) [@Vadim\_Voituk](https://discuss.elastic.co/u/Vadim_Voituk)\
**Post date:** [May 22, 2012, 8:36am UTC](https://discuss.elastic.co/t/inject-external-value-for-sorting/7686/7 "2012-05-22T08:36:35Z")

</div>

As i understand - that's exactly how ES works inside 🙂

On Thursday, May 17, 2012 3:10:55 AM UTC+2, Klérisson Paixão wrote:

> Just implemented a similar feature, but through a bad coding workaround.
> 
> I got the search response in JAVA application, than I performed the custom  
> score calculation based on the highlighted fields, after that I stored the  
> documents in a sorted list (PriorityQueue) and, finally, sent the response  
> to the client.
> 
> The overall performance is not good at all, but still enough to my  
> environment.
> 
> Regards,
> 
> --  
> View this message in context:  
> [http://elasticsearch-users.115913.n3.nabble.com/Inject-external-value-for-sorting-tp3985874p3998283.html](http://elasticsearch-users.115913.n3.nabble.com/Inject-external-value-for-sorting-tp3985874p3998283.html)  
> Sent from the Elasticsearch Users mailing list archive at [Nabble.com](http://Nabble.com).

---

<div class="post-metadata">

**Author:** ![Klerisson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/klerisson/32/2878_2.png) [@Klerisson](https://discuss.elastic.co/u/Klerisson)\
**Post date:** [May 22, 2012, 3:19pm UTC](https://discuss.elastic.co/t/inject-external-value-for-sorting/7686/8 "2012-05-22T15:19:23Z")

</div>

In my case, the score wasn't been calculated, perhaps, because the query type. So, that's why I made that crappy trick. Anyway, I also did not figure out how to handle your situation. ☹

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 3:27am UTC](https://discuss.elastic.co/t/inject-external-value-for-sorting/7686/9 "2017-07-06T03:27:39Z")

</div>