Groovy script in Elastic Stack 5.0 error: action request validation exception

[maziyar (Maziyar PANAHI)]

Hi,

After upgrading my cluster from 2.4 to 5.0 I am having trouble to get my script files working.

Here is what my script look like in /etc/elasticsearch/scripts/update_keywords.groovy:

if (!ctx._source.containsKey("tweet_keywords")) { ctx._source.tweet_keywords = myfield; ctx._source.tweet_keywords+=keyword; } else { ctx._source.tweet_keywords+=keyword; }

I use this elasticsearch.js latest npm for bulk updates which it worked in 2.4:

  var updateOps = []; 
  updateOps.push({
                "update" : {
                    _index: indexName,
                    _type: 'tweet',
                    _id: hit._id
                }
            }
        );
        updateOps.push({
                "script_file": "update_keywords",
                "params" : {
                    keyword: [{
                        "id": keywordsIndexName[index],
                        "count":1
                    }],
                    "myfield": []
                },
                "upsert" : {}
            }
        );
        es_client.bulk({
            body: updateOps
        }, function (err, resp) {
            if(!err){
                updateOps=[];
            }else{
                console.log(err, resp);
                updateOps=[];
            }
        });

I even added this to each node elasticsearch.yml and restarted them all:
> script.stored: true

   script.file: true

Now I am receiving this error.

{ [Error: [action_request_validation_exception] Validation Failed: 1: script or doc is missing;2: script or doc is missing;3: script or doc is missing;]
status: 400,
displayName: 'BadRequest',
message: '[action_request_validation_exception] Validation Failed: 1: script or doc is missing;2: script or doc is missing;3: script or doc is missing;',
path: '/_bulk',
query: {},
body: '{"update":{"_index":"climate-2016.10","_type":"tweet","_id":"782787933824843776"}}\n{"script_file":"update_keywords","lang":"groovy","params":{"keyword":[{"id":"above-ground biomass","count":1}],"myfield":},"upsert":{}}\n{"update":{"_index":"climate-2016.10","_type":"tweet","_id":"788112034026184704"}}\n{"script_file":"update_keywords","lang":"groovy","params":{"keyword":[{"id":"above-ground biomass","count":1}],"myfield":},"upsert":{}}\n{"update":{"_index":"climate-2016.10","_type":"tweet","_id":"786204189533712387"}}\n{"script_file":"update_keywords","lang":"groovy","params":{"keyword":[{"id":"above-ground biomass","count":1}],"myfield":},"upsert":{}}\n',
statusCode: 400,
response: '{"error":{"root_cause":[{"type":"action_request_validation_exception","reason":"Validation Failed: 1: script or doc is missing;2: script or doc is missing;3: script or doc is missing;"}],"type":"action_request_validation_exception","reason":"Validation Failed: 1: script or doc is missing;2: script or doc is missing;3: script or doc is missing;"},"status":400}',
toString: [Function],
toJSON: [Function] } { error:
{ root_cause: [ [Object] ],
type: 'action_request_validation_exception',
reason: 'Validation Failed: 1: script or doc is missing;2: script or doc is missing;3: script or doc is missing;' },
status: 400 }

If I chang the script field to this it won't show any error but it doesn't update anything neither:

      updateOps.push({
                "script":{"file":"update_keywords"},"lang":"groovy",                    
                "params" : {
                    keyword: [{
                        "id": keywordsIndexName[index],
                        "count":1
                    }],
                    "myfield": []
                },
                "upsert" : {}
            }
        );

Is there anything else I should do in order to get this working like 2.4?

Many thanks

[maziyar (Maziyar PANAHI)]

After spending the entire day on this, apparently something has changed in elasticsearch.js new version that needs the script field be formatted as follow:

              "script":{
                    "lang":"groovy",
                    "file":"update_keywords",
                    "params" : {
                        "keyword": [{
                            "id": keywordsIndexName[index],
                            "count":1
                        }],
                        "myfield": []
                    }
                },

Params need to be in side script field and language must be mentioned. I wish the error would have been more clear on this. I really thought there was something wrong with my script or elasticsearch configs.

I will also upgrade groovy to painless soon before it is completely gone in next version.

Best,
Maziyar