"Too many dynamic script compilations" When using int literals

[neoform]

Too many dynamic script compilations within, max: [75/5m]; please use indexed, or scripts with parameters instead; this limit can be changed by the [script.max_compilations_rate]

The script I'm using has hardcoded integer literals, yet the documentation How to write scripts | Elasticsearch Guide [master] | Elastic refers to such literals as being "variables".

Is there a reason why a hardcoded number needs to be passed as a param and blocks the script from being cached?

[rjernst (Ryan Ernst)]

If your script is using the exact same value, always, then that should not be causing recompilation. A couple questions for you:

• Can you share your script?
• What parameters, if any, does it have?
• Are you dynamically generating the script?
• What type of script is it (eg filter script, scoring script, script field, etc)?

[neoform]

_score + (Math.log1p(doc['views'].value + 1) / 100) * (Math.log1p(100000 - doc['rank'].value) / 1000)

I had to replace the hardcoded number values (they never change), the script is a script_score and is passed along with the search query.

[rjernst (Ryan Ernst)]

You shouldn't need to replace those constants if they never changed. Is this a production instance, or something you can play with? I would be interested to see trace logging enabled for the ScriptService so we can see exactly what the system thinks is different between your script invocations (if it is always the same script, it should be cached). You can turn this on dynamically: https://www.elastic.co/guide/en/elasticsearch/reference/current/logging.html#configuring-logging-levels. I think the setting would be logger.org.elasticsearch.script.

[neoform]

This is a new production instance that has been causing these errors, we upgraded from 2.4 to 6.2 and were inundated with errors and had to immediately switch back to the old cluster.

I'll talk to our sysadmins to see what they say about adding more logging.

[neoform]

GET /testindex/_search
{
  "query": {
    "function_score": {
      "query": {
        "bool": {
          "filter": [
            {
              "term": {
                "privacy": "public"
              }
            },
            {
              "terms": {
                "status": [
                  "active",
                  "pendingactive"
                ]
              }
            }
          ],
          "must_not": [
            {
              "terms": {
                "user_id": [
                  13,
                  14
                ]
              }
            }
          ]
        }
      },
      "functions": [
        {
          "script_score": {
            "script": "_score + 1 + Math.log1p(doc['a'].value * doc['b'].value) * Math.log1p(doc['c'].value) * Math.log1p(doc['d'].value)"
          }
        }
      ]
    }
  },
  "sort": {
    "_score": "desc",
    "q": "desc"
  }
}

Result in logs:

[2018-03-21T16:37:24,519][TRACE][o.e.s.ScriptService      ] [ded6964] compiling script, type: [inline], lang: [painless], options: [{}]
[2018-03-21T16:37:24,519][TRACE][o.e.s.ScriptService      ] [ded6964] compiling lang: [painless] type: [inline] script: _score + 1 + Math.log1p(doc['a'].value * doc['b'].value) * Math.log1p(doc['c'].value) * Math.log1p(doc['d'].value)
[2018-03-21T16:49:46,767][TRACE][o.e.s.ScriptService      ] [ded6010] compiling lang: [painless] type: [inline] script: _score + 1 + Math.log1p(doc['a'].value * doc['b'].value) * Math.log1p(doc['c'].value) * Math.log1p(doc['d'].value)
[2018-03-21T16:49:46,767][TRACE][o.e.s.ScriptService      ] [ded6010] compiling lang: [painless] type: [inline] script: _score + 1 + Math.log1p(doc['a'].value * doc['b'].value) * Math.log1p(doc['c'].value) * Math.log1p(doc['d'].value)

Every time I run the query it shows to be recompiling.