# Script sorting

**URL:** <https://discuss.elastic.co/t/script-sorting/250338>\
**Category:** Elasticsearch\
**Created:** [September 29, 2020, 10:31am UTC](https://discuss.elastic.co/t/script-sorting/250338 "2020-09-29T10:31:38Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![flatmap101](https://avatars.discourse-cdn.com/v4/letter/f/ac8455/32.png) [@flatmap101](https://discuss.elastic.co/u/flatmap101)\
**Post date:** [September 29, 2020, 10:31am UTC](https://discuss.elastic.co/t/script-sorting/250338/1 "2020-09-29T10:31:38Z")

</div>

hi all!  
i am getting some very strange behavior while using scripted sorting.

the script itself looks approximately like that:

```auto
if (doc['fieldName1'].size() != 0 && doc['fieldName2'].size() != 0) { // 1 - might not be true, so a fallback scenario is added
  long field1 = doc['fieldName1'].value;  
  double field2 = doc['fieldName2'].value;
  if (field1 == 2) {
    return field1 * params.paramValueName / field2 
  } else {
    return field2 * params.paramValueName / field1 
  }
} else {
  if (doc['fallBackField'].size() != 0) { // 2 - always true, checked for emptiness out of safety reasons
    return doc['fallbackField'].value; 
  } else {
    return 1; // least possible case
  }
};

```

each time a script is run, it ends in an arm 2 where the fallback scenario returns the value of a fallback field.  
but when i query the index manually via curl by the known id, i can see that the values checked in the first if are always filled.

```auto
curl -X GET "localhost:9200/indexName_v46/_search?pretty" -H 'Content-Type: application/json' -d '
{ "query": { "bool": { "must" : [{ "match" : { "_id": "51c370c8-7a60-4b15-9466-0692e6b01e86" } }] } }, "_source": ["fieldName1", "fieldName2"] }'

```

what might be the issue i receive null values on each call and fall into a fallback case?

---

<div class="post-metadata">

**Author:** ![Hendrik\_Muhs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendrik_muhs/32/25802_2.png) [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)\
**Post date:** [September 29, 2020, 12:21pm UTC](https://discuss.elastic.co/t/script-sorting/250338/2 "2020-09-29T12:21:51Z")

</div>

In your script you are looking at [doc values](https://www.elastic.co/guide/en/elasticsearch/reference/current/doc-values.html) but the output of your curl command returns `_source`.

To use `_source` in your script you can use for example:

```auto
params._source['fieldName1']

```

Or you ensure that you have mappings defined for your fields, so that doc values exist.

Whether it is better to doc values or `_source` depends on what you want to do and therefore hard to answer without knowing your use case. Please have a look at the documentation link.

---

<div class="post-metadata">

**Author:** ![flatmap101](https://avatars.discourse-cdn.com/v4/letter/f/ac8455/32.png) [@flatmap101](https://discuss.elastic.co/u/flatmap101)\
**Post date:** [September 29, 2020, 8:53pm UTC](https://discuss.elastic.co/t/script-sorting/250338/3 "2020-09-29T20:53:01Z")

</div>

> [@Hendrik\_Muhs](#):
>
> but the output of your curl command returns `_source` .

the command was just a naive proof that those fields are not empty and therefore contain values, i thought that would just be a nice example that documents that are being sorted actually contain  
valid data, that is used for sorting itself. there is a non zero possibility that i just get it wrong :))

i will try to explain the whole scenario:

we have a set of documents that have prices (long numeric values), so basically "fieldName1" is "price", while "fieldName2" is "currency", a "fallbackField" is an outdated field that also contains a price, but is going to be deprecated soon, so is not really wanted to be relied on.

if price and currency are not null, an exchange rate ("params.paramValueName") should be applied to the price value. the set of documents should be therefore sorted against the price\*currency rate value, if neither price, nor currency rate are known, a value of a fallback field should be used as a sorted value.

now it should be a bit more clear :^))

---

<div class="post-metadata">

**Author:** ![Hendrik\_Muhs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/hendrik_muhs/32/25802_2.png) [@Hendrik\_Muhs](https://discuss.elastic.co/u/Hendrik_Muhs)\
**Post date:** [September 30, 2020, 6:49am UTC](https://discuss.elastic.co/t/script-sorting/250338/4 "2020-09-30T06:49:32Z")

</div>

The curl command does not return the _same_ values as your script. Again, you are comparing doc values and source. These are not the same, doc values are normalized, e.g. you could ingest a timestamp as long, but map it as date with timezone difference. You might also disable doc values for certain fields if you do not plan to search or aggregate on them.

To return the doc values with curl use:

```auto
curl -X GET "localhost:9200/indexName_v46/_search?pretty" -H 'Content-Type: application/json' -d '
{ "query": { "bool": { "must" : [{ "match" : { "_id": "51c370c8-7a60-4b15-9466-0692e6b01e86" } }] } }, "docvalue_fields": ["fieldName1", "fieldName2"] }'

```

Does that command return the desired values?

---

<div class="post-metadata">

**Author:** ![flatmap101](https://avatars.discourse-cdn.com/v4/letter/f/ac8455/32.png) [@flatmap101](https://discuss.elastic.co/u/flatmap101)\
**Post date:** [September 30, 2020, 10:19am UTC](https://discuss.elastic.co/t/script-sorting/250338/5 "2020-09-30T10:19:20Z")

</div>

yes, when querying with `docvalue_fields `, the desired values are actually in place.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 28, 2020, 10:19am UTC](https://discuss.elastic.co/t/script-sorting/250338/6 "2020-10-28T10:19:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.