Elastic Scripting caching issue

Elastic Stack Elasticsearch
1

I am facing caching issue while using scripts.
So I am using below script in elasticsearch query.
But when it runs multiple times it gives circuit breaker exception( Too many dynamic script compilations within, max: [75/5m]) as my key keeps changing. So it caches everytime a new key is there. I tried to look for params help but I couldn't find any help where doc key is passed as param value. Can you help how can we achieve this?

"(doc.containsKey(key) && (doc[key] != null ||doc[key] == 0)) ? doc[key] : 1

2

In which context are you using this script?

3

Hi @dadoonet Can you please elaborate more on context which you are asking. Not aware of context here.
I am using default painless script in elastic cluster 6.4.2 if that is what you are looking for.

4

What is the exact and full request you are sending? Which API?

5

HI @dadoonet ,

I am sending the _search request on one of the index with body as :

{
  "from": 0,
  "size": 20,
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "is_in_stock": 1
          }
        }
      ]
    }
  },
  "aggs": {
    "min_price": {
      "aggs": {
        "min_price": {
          "nested": {
            "path": "store"
          },
          "aggs": {
            "min_price": {
              "min": {
                "script": "(doc.containsKey('store.warehouse_1162652_price') && (doc['store.warehouse_1162652_price'] != null || doc['store.warehouse_1162652_price'] == 0)) ? doc['store.warehouse_1162652_price'] : doc['store.price']"
              }
            }
          }
        }
      }
    }
  }
}

Here store.warehouse_1162652_price varies with every request. Like it is store.warehouse_1162652_price for this request. It can be store.warehouse_1162687_price for another request .So I am trying to use params for the script. Need to understand how can i write that.

6

I am also having this issue with scripted metric aggregations with stored scripts. The process taken to understand the issue is:

  1. a script code is updated and then pushed to the _scripts/ api overwriting the current stored script file.
  2. the scripted metric that references the stored script will about 50% of the time run an older version of the script or it will run the new script. Spamming the play button will show differences in responses about 50% of the time.

It feels like a caching issue of the old script. I can provide more details if needed.

GET index/_search
{
"aggs": {
"count_of_open": {
  "scripted_metric": {
    "init_script": {"id": "open_closed_init-map-test"},
    "map_script": {"id": "open_closed_map-script-map"},
    "combine_script":{"id": "open_closed_combine-script-map"},
    "reduce_script": {"id": "open_closed_reduction-map"}
  },
  "aggs": {
    "cloudType": {
      "terms": {
        "field": "cloudType.keyword",
        "size": 10
      }
    }
  }
}
},
"size": 0
}
7

Could you write something like this:

{
  "from": 0,
  "size": 20,
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "is_in_stock": 1
          }
        }
      ]
    }
  },
  "aggs": {
    "min_price": {
      "aggs": {
        "min_price": {
          "nested": {
            "path": "store"
          },
          "aggs": {
            "min_price": {
              "min": {
                "script": {
                  "source": "(doc.containsKey(params.field) && (doc[params.field] != null || doc[params.field] == 0)) ? doc[params.field] : doc['store.price']",
                  "params": {
                    "field": "store.warehouse_1162652_price"
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}
1 Like
8

Hi @dadoonet

This seems to be working. Thanks a lot for your help.
Cheers!

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.