Scripted field conditionals failing (5 of 15 shards failed)

[caub (Cyril Auburtin)]

I get this error "n of N shards failed" in the bottom right notification in Discover tab, when I add a scripted field such as:

_msg    painless    return doc['err_msg'].value || doc['message'].value;      String

I tried many variants like:

Both fields exist, the error still happen with return doc['err_msg'].value || doc['err_stack'].value for example (but anyway, that script should still work for non-existing fields)

There are many similar topics
https://discuss.elastic.co/search?q=scripted%20field%20conditional%20category%3A7 but I couldn't find one that worked in my case

I don't see specific errors in kibana or elasticsearch logs

Kibana and ES version 6.4.0

[Joe_Fleming]

but anyway, that script should still work for non-existing fields

I'm probably not as well versed in Painless as you are, but are you sure that's true? It's the different between getting a string value and not getting any value, and given that Painless is written in Java, and is strictly typed, I would expect missing fields, and even document that are missing data in both fields, to cause errors.

I don't see specific errors in kibana or elasticsearch logs

I don't think you'll get errors in the server logs, but you should get some kind of error both in the response. You should be able to expand the error in Kibana to see it, and it might give you some more details about why those shards failed.

[caub (Cyril Auburtin)]

Ok thanks @Joe_Fleming, you were right on checking for the value, I found another thread explaining it

Now the issue is that my scripted fields have empty values, even though the doc fields are not empty

Scripted field 1: _msg painless String

if (doc.err_msg.value != null)
  return doc.err_msg.value + '--' + doc.err_stack.value;
return '';

Scripted field 2: _req painless String

if (doc['req.path'].value != null)
  return doc['req.method'].value +' '+ doc['req.path'].value +' '+ doc['res.status'].value;
return '';

^ You can see above doc['req.path'], doc['req.method'] are not empty

We would thinkdoc['req.path'].value != null is evaluated to false, but if I change the script to just:

return doc['req.method'].value +' '+ doc['req.path'].value +' '+ doc['res.status'].value;

it's still the same

---

note I took care to refresh the kibana filebeat- index after editing those fields

note: on the 5 of 15 shards failed error
the error with shards I has before, is probably due to err_stack being a Text, and not a Keyword

I had force those types preconfigured in my filebeat.yml config:

setup.template.append_fields:
  - name: err_msg
    type: keyword
  - name: err_stack
    type: text

But I removed this, it's not necessary as kibana will take string type by default, which works

[caub (Cyril Auburtin)]

Oh in doc['req.method'].value + ' ' + doc['req.path'].value + ' ' + doc['res.status'].value res.status is an integer type, so it failed silently for this reason

I need to cast it to string

continuing here: Cast Long to String