Script difference between 1.7 and 6.8

Elastic Stack Elasticsearch
1

Hello,

Does anyone can help on how the following search can be translated to be compatible with ES 6.8?

GET access-*/_search
{
    "query": {
        "range": {
            "request.time": {
                "gte": "2019-08-21 17:52",
                "lt": "2019-08-21 17:53",
                "format": "yyyy-MM-dd HH:mm"
            }
        }
    },
    "aggs": {
        "group": {
            "terms": {
                "script": "def matcher = doc['request.uri'].value =~ /domain[=\\/]([^\\/=&]+)/; return doc['request.method'].value + '--*' + doc['response.code'].value + '--*' + doc['geoip.country_code2'].value + '--*' + doc['request.uri'].value.split('\\\\?')[0].replaceAll(/\\/(request|domain)\\/[^&\\?\\/]+/, '') + '--*' + (matcher.getCount() > 0 ? matcher[0][1] : '')",
                "size": 10000
            },
            "aggs": {
                "totalResponseTime": {
                    "sum": {
                        "field": "response.time"
                    }
                }
            }
        }
    }
}

This is returning the following error:

#! Deprecation: returning default values for missing document values is deprecated. Set system property '-Des.scripting.exception_for_missing_value=true' to make behaviour compatible with future major versions!
{
  "took" : 17,
  "timed_out" : false,
  "_shards" : {
    "total" : 42,
    "successful" : 39,
    "skipped" : 0,
    "failed" : 3,
    "failures" : [
      {
        "shard" : 0,
        "index" : "access-2019.08.21",
        "node" : "Fy19yOx7SyeH2PKlJCQpPg",
        "reason" : {
          "type" : "script_exception",
          "reason" : "runtime error",
          "script_stack" : [
            """return doc['request.method'].value + '--*' + doc['response.code'].value + '--*' + doc['geoip.country_code2'].value + '--*' + doc['request.uri'].value.split('\\?')[0].replaceAll(/\/(request|domain)\/[^&\?\/]+/, '') + '--*' + (matcher.getCount() > 0 ? matcher[0][1] : '')""",
            "                                                                                                                                                     ^---- HERE"
          ],
          "script" : """def matcher = doc['request.uri'].value =~ /domain[=\/]([^\/=&]+)/; return doc['request.method'].value + '--*' + doc['response.code'].value + '--*' + doc['geoip.country_code2'].value + '--*' + doc['request.uri'].value.split('\\?')[0].replaceAll(/\/(request|domain)\/[^&\?\/]+/, '') + '--*' + (matcher.getCount() > 0 ? matcher[0][1] : '')""",
          "lang" : "painless",
          "caused_by" : {
            "type" : "illegal_argument_exception",
            "reason" : "dynamic method [java.lang.String, split/1] not found"
          }
        }
      }
    ]
  },
  "hits" : {
    "total" : 0,
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "group" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [ ]
    }
  }
}

After some research we found out 3 major changes here:

  • Split -> should use like this /\\?/.split(doc['request.uri'].value)[0]
  • getCount() -> maybe is not needed anymore
  • ReplaceAll -> similarly to split we should use the pattern and then to the replacement

It ended up like this:

GET c3532-access-*/_search
{
    "query": {
        "range": {
            "request.time": {
                "gte": "2019-08-21 17:52",
                "lt": "2019-08-21 17:53",
                "format": "yyyy-MM-dd HH:mm"
            }
        }
    },
    "aggs": {
        "group": {
            "terms": {
                "script": "def matches = doc['request.uri'].value =~ /domain[=\\/]([^\\/=&]+)/;  return doc['request.method'].value + '--*' + doc['response.code'].value + '--*' + doc['geoip.country_code2'].value + '--*' + /\\/(request|domain)\\/[^&\\?\\/]+/.matcher(/\\\\?/.split(doc['request.uri'].value)[0]).replaceAll('') + '--*' + (matches ? matches[0][1] : '')",
                "size": 10000
            },
            "aggs": {
                "totalResponseTime": {
                    "sum": {
                        "field": "response.time"
                    }
                }
            }
        }
    }
}

But now we get the following:

#! Deprecation: returning default values for missing document values is deprecated. Set system property '-Des.scripting.exception_for_missing_value=true' to make behaviour compatible with future major versions!
{
  "took" : 15,
  "timed_out" : false,
  "_shards" : {
    "total" : 42,
    "successful" : 39,
    "skipped" : 0,
    "failed" : 3,
    "failures" : [
      {
        "shard" : 0,
        "index" : "access-2019.08.21",
        "node" : "Fy19yOx7SyeH2PKlJCQpPg",
        "reason" : {
          "type" : "script_exception",
          "reason" : "runtime error",
          "script_stack" : [
            """return doc['request.method'].value + '--*' + doc['response.code'].value + '--*' + doc['geoip.country_code2'].value + '--*' + /\/(request|domain)\/[^&\?\/]+/.matcher(/\\?/.split(doc['request.uri'].value)[0]).replaceAll('') + '--*' + (matches ? matches[0][1] : '')""",
            "                                                                                                                                                                                                                                         ^---- HERE"
          ],
          "script" : """def matches = doc['request.uri'].value =~ /domain[=\/]([^\/=&]+)/;  return doc['request.method'].value + '--*' + doc['response.code'].value + '--*' + doc['geoip.country_code2'].value + '--*' + /\/(request|domain)\/[^&\?\/]+/.matcher(/\\?/.split(doc['request.uri'].value)[0]).replaceAll('') + '--*' + (matches ? matches[0][1] : '')""",
          "lang" : "painless",
          "caused_by" : {
            "type" : "illegal_argument_exception",
            "reason" : "Attempting to address a non-array-like type [java.lang.Boolean] as an array."
          }
        }
      }
    ]
  },
  "hits" : {
    "total" : 0,
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "group" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [ ]
    }
  }
}

And we're stuck on this.

Any help is welcome.

Thanks.

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.