# Performance: Sorting / Filtering by runtime fields

**URL:** https://discuss.elastic.co/t/performance-sorting-filtering-by-runtime-fields/320540
**Category:** Elasticsearch
**Tags:** painless
**Created:** [December 6, 2022, 8:08am UTC](https://discuss.elastic.co/t/performance-sorting-filtering-by-runtime-fields/320540 "2022-12-06T08:08:33Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![kennyprisjakt](https://avatars.discourse-cdn.com/v4/letter/k/e19b73/32.png) [@kennyprisjakt](https://discuss.elastic.co/u/kennyprisjakt)
#### Post date: [December 6, 2022, 8:08am UTC](https://discuss.elastic.co/t/performance-sorting-filtering-by-runtime-fields/320540/1 "2022-12-06T08:08:33Z")

</div>

Hey,

I'm looking into getting price conversions up to date in my indices.

- Price data: Updated index time
- Price conversions: Updated in query time (via runtime fields)
  - The data for the currency conversion changes often, so that's why I'm approaching this do be done via query time rather than index time.

The problem is that it seems to be too slow for acceptable usage.  
Indexing the prices with conversion already done will possibly too slow, as it's hundreds of millions of documents in my index.

So the question is if there is any recommendations to make my approach faster some how, or do i need to resort to a index-time solution instead (which would not always be up to date etc)?

Here is some technical background on my implementation, in case you want to try it out:

Add fake data to an existing index:

```auto
POST some-index-with-many-documents/_update_by_query?refresh
{
  "query": {
    "match_all": {}
  },
  "script": {
    "source": """
      Random rand = new Random();
      
      ctx._source.prices = 
        [
          'lowest':
          [
            'SEK': rand.nextInt(2000),
            'NOK': rand.nextInt(2000),
            'DKK': rand.nextInt(2000),
            'GBP': rand.nextInt(2000),
            'EUR': rand.nextInt(2000)
          ]
        ]
    """
  }
}

```

Setting the runtime field:

```auto
PUT some-index-with-many-documents/_mapping
{
  "runtime": {
    "lowestPrice": {
      "type": "double",
      "script": {
        "lang": "painless",
        "source":
        """
          String targetKey = params['targetKey'];
          HashMap conversionRates = params['conversionRates'];
          HashMap prices = params['_source']["prices"].get(targetKey);
          double lowestPrice = Double.MAX_VALUE;
          
          for(Map.Entry price : prices.entrySet()) {
            double convertedPrice = price.getValue() * conversionRates.get(price.getKey());
            if(lowestPrice < 0 || convertedPrice < lowestPrice) {
              lowestPrice = convertedPrice;
            }
          }
          
          emit(lowestPrice)
        """,
        "params": {
          "targetKey": "lowest",
          "conversionRates": {
            "DKK": 0.679828,
            "EUR": 0.091418,
            "GBP": 0.079018,
            "NOK": 0.93673,
            "SEK": 1
          }
        }
      }
    }
}

```

Sorting by price:

```auto
GET some-index-with-many-documents/_search
{
  "query": {
    "match_all": {}
  },
  "sort" : [
    { "lowestPrice" : {"order" : "asc"}}
    ],
  "fields": ["lowestPrice"]
}

```

Filtering by price:

```auto
GET development-discovery-flink-offer_product_se-v2022.11.15-19.44.00_price_test/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "match_all": {}
        }
      ],
      "filter": [
        {
          "range": {
            "lowestPrice": {
              "lte": "50"
            }
          }
        }
      ]
    }
  },
  "fields": ["lowestPrice"]
}

```

Thanks in advance,  
Kenny

---

<div class="post-metadata">

### Author: ![stu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stu/32/75063_2.png) [@stu](https://discuss.elastic.co/u/stu)
#### Post date: [December 7, 2022, 3:20pm UTC](https://discuss.elastic.co/t/performance-sorting-filtering-by-runtime-fields/320540/2 "2022-12-07T15:20:04Z")

</div>

That script looks fine. Pulling source is slow, so avoid it if possible.

If the set of prices is known ahead of time, you can access it via doc values (`doc['SEK'].value` or `$('SEK', 0)` after 8.1) which should be much faster.

---

<div class="post-metadata">

### Author: ![kennyprisjakt](https://avatars.discourse-cdn.com/v4/letter/k/e19b73/32.png) [@kennyprisjakt](https://discuss.elastic.co/u/kennyprisjakt)
#### Post date: [December 8, 2022, 9:58am UTC](https://discuss.elastic.co/t/performance-sorting-filtering-by-runtime-fields/320540/3 "2022-12-08T09:58:38Z")

</div>

Thanks for your input! I will try it out when i have time

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [January 5, 2023, 9:59am UTC](https://discuss.elastic.co/t/performance-sorting-filtering-by-runtime-fields/320540/4 "2023-01-05T09:59:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.