Groovy script to iterate through nested object for scoring throws error

Elastic Stack Elasticsearch
1

I have a nested object mygroup in my document as follows:

{
  "name": "john",
  "mygroup": [
    {
      "start": 200,
      "end": 490
    },
    {
      "start": 800,
      "end": 900
    },
    {
      "start": 22,
      "end": 49
    }
  ]
}

One of my script_score in my function_score is as follows:

"script_score": {
  "params": {
    "gboost": -1350,
    "desired": 30,
    "delta": 10
  },
  "script": "doc['mygroup'].values.any{ if(it -> it.end >= desired &&  it.start <= desired) { return (it.end - delta) * 2} };" 
}

Here my goal is to find an object that matches my condition where start is >= desired AND end <= desired (There is always ONLY one object that will suffice this condition, that's guaranteed in my data storage), once I find that object then I would like to return (it.end - delta) * 2 for score calculation

My above script "doc['mygroup'].values.any{ if(it -> it.end >= desired && it.start <= desired) { return (it.end - delta) * 2} };" is not working and it keeps throwing error. I am using ES 2.3 and groovy for scripting language.

2

Can you post the error?

3

Sure @nik9000 , this is what I get:

{
  "took": 5,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 4,
    "failed": 1,
    "failures": [
      {
        "shard": 0,
        "index": "listings_v1",
        "node": "ce3vfaL765et7UXTD4ewQ",
        "reason": {
          "type": "query_parsing_exception",
          "reason": "script_score the script could not be loaded",
          "index": "listings_v1",
          "line": 15,
          "col": 17,
          "caused_by": {
            "type": "script_exception",
            "reason": "scripts of type [inline], operation [search] and lang [groovy] are disabled"
          }
        }
      }
    ]
  },
  "hits": {
    "total": 0,
    "max_score": null,
    "hits": []
  }
}

FYI my elasticsearch.yml has this setting:

script.inline: true
script.indexed: true

I am using ES2.3. Please note my all other groovy queries work fine in script_score, only this one fails

4

It is weird that sometimes I get different error message saying
"reason": "missing_method_exception: No signature of method: java.util.HashMap.call() is applicable for argument types: (e59f5ff9a4ea5a82513a5a8e06fcc8269b487334$_run_closure1$_closure2) values: [e59f5ff9a4ea5a82513a5a8e06fcc8269b487334$_run_closure1$_closure2@6be9b69c]\nPossible solutions: wait(), any(), wait(long), any(groovy.lang.Closure), each(groovy.lang.Closure), max(groovy.lang.Closure)"

5

By the way "script": "doc['mygroup'].values.... < i think I cannot use doc here. It should be "script": "_source['mygroup'].values per ElasticSearch: access document nested value in groovy script ? Please let me knwo what do you think @nik9000

6

So _source and doc should both be available, though I think to get the values for doc you'd need doc['mygroup.start'] and doc['mygroup.end']. The reason is that doc processes things column wise whereas _source uses the source document. I expect doc to be much faster if processing a lot of data.

scripts of type [inline], operation [search] and lang [groovy] are disabled

That makes me suspect at least one of your Elasticsearch nodes doesn't have inline scripts enabled. Those errors should appear in the logs - maybe you can look and find if it is just one node?

7

Since I need to loop through mygroup object first to find the object that matches my condition, so
"script": "doc['mygroup.start'].values.any{.......... will not work, right?

Regardin So _source and doc should both be available
doc it doesn't work, it only works with _source

8

It should work with doc, just differently. More like
[doc['mygroup.start'], doc['mygroup.end']].transpose().any { [start, end] -> end >= desired && start <= desired) return (start - delta) * 2

9

Thanks for your suggestion @nik9000

There was a minor typo in your snippet
[doc['mygroup.start'], doc['mygroup.end']].transpose().any { [start, end] -> end >= desired && start <= desired) return (start - delta) * 2

changed to

[doc['mygroup.start'], doc['mygroup.end']].transpose().any { [start, end] -> end >= desired && start <= desired} return (start - delta) * 2

then the error is

"reason": { "type": "query_parsing_exception", "reason": "script_score the script could not be loaded", "index": "listings_v1", "line": 15, "col": 17, "caused_by": { "type": "script_exception", "reason": "[doc['mygroup.start'], doc['mygroup.end']].transpose().any { [start, end] -> end >= desired && start <= desired} return (start - delta) * 2", "caused_by": { "type": "script_exception", "reason": "failed to compile groovy script", "caused_by": { "type": "multiple_compilation_errors_exception", "reason": "multiple_compilation_errors_exception: startup failed:\n7b2778accce04239f1cc125c4f6467f97c82295f: 1: expecting '}', found '->' @ line 1, column 89.\n ranspose().any { [start, end] -> end\n ^\n\n1 error\n" } } } }

10

Sorry! I'm not really a groovy expert and I just wrote it without trying it against anything. I expect there is a syntax error when I destructure the list around [start, end]. I thought it worked like that but I think it should instead be (start, end). Something like that ought to work for you though.

11

Thanks @nik9000 for your persistency to unblock me ...really appreciated. See thsi is what I get as error: