Too many dynamic script compilations within max

[rvanegmond (Roel van Egmond)]

I know there are some posts about this but they didn't completely awnser my question.

I'm processing millions of data in terms of documents which need to be updated by a "where query" using two fields. The data in the query is coming from an external system. The posts I perform look like the one below, of course the values change.

{
   "query" : {
   "bool" : {
       "must" : [
         {
           "term" : {
           "path.raw" : {
           "value" : "d:/kibana-7.3.0/x-pack/legacy/plugins/siem/node_modules/lodash/fp/takeRight.js"
          }
        }
      },
      {
      "range" : {
        "file.modified.date" : {
          "gte" : "2019-07-24T16:58:46Z",
          "lte" : "2019-07-24T16:58:46Z"
          }
        }
      }
  ]
}
},
"script" : {
   "source" : "ctx._source.file.hash = '0A98E2919C1776463A3AD3768663987F'",
    "lang" : "painless"
}

}

First of all is this the correct way to do updates like this or are there better methods? I'm already using bulk when I know the actual _id the document to update.

Is it safe the change the max setting? And if not, then how should I perform something like this.

Thanks a lot!

[rvanegmond (Roel van Egmond)]

I managed to get around the exception by using params. The question remains is this the best and fastest way to do this kind of updates?

[spinscale (Alexander Reelsen)]

Hey,

using params will solve your script issue. Alternatively you could also use a pipeline like mentioned in the update by query docs (if you are using update by query, not a hundred percent sure).

--Alex