Script field and regexp

[apoc]

Hello,

I installed the elastic stack 5.2.2.
In a script field, I would like to test another field with a regexp :
return (doc['processContext.componentName'].value ==~ /IN/);

I get an error Courier Fetch: 1 of 5 shards failed. in kibana. Nothing in elasticsearch or kibana logs

The script.painless.regex.enabled parameter is set in elasticsearch.yml

Any clue ?

Thank you for your help

[ppisljar (Peter Pisljar)]

can you try with something like if (doc['....'].value ==~ /IN/) return 'match'; else return ''; ?

[apoc]

Same error

[mfillion (Mathieu)]

I try other things :

Test 1 (Kibana script field):

return [
"doc['processContext.componentName'] = " + doc['processContext.componentName'],
"doc['processContext.componentName'].value = " + doc['processContext.componentName'].value,
"doc['processContext.componentName'].length = " + doc['processContext.componentName'].length,
""TOTO" ==~ /IN/ = " + ("TOTO" ==~ /IN/),
""IN_TOTO" ==~ /IN/ = " + ("IN_TOTO" ==~ /IN.*/)
];

Works with this result :
doc['processContext.componentName'] = [IN_Ret], doc['processContext.componentName'].value = IN_Ret, doc['processContext.componentName'].length = 1, "TOTO" ==~ /IN/ = false, "IN_TOTO" ==~ /IN/ = true

Test 2 (Kibana script field):

return [
"doc['processContext.componentName'] = " + doc['processContext.componentName'],
"doc['processContext.componentName'].value = " + doc['processContext.componentName'].value,
"doc['processContext.componentName'].length = " + doc['processContext.componentName'].length,
""TOTO" ==~ /IN/ = " + ("TOTO" ==~ /IN/),
""IN_TOTO" ==~ /IN/ = " + ("IN_TOTO" ==~ /IN./),
"doc['processContext.componentName'].value ==~ /IN/ = " + (**doc['processContext.componentName'].value ==~ /IN./**)
];

Don't work, same Kibana error "Courier Fetch: 1 of 5 shards failed". Nothing in elasticsearch or kibana logs

Test 3 (Kibana Dev Console):

GET test-/_search
{
"query": {
"match_all": {}
},
"script_fields" : {
"test1" : {
"script" : {
"lang": "painless",
"inline": "return **doc['processContext.componentName'].value ==~ /IN./**"
} } }}

Works with this result :
{
"took": 5,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 998,
"max_score": 1,
"hits": [
{
"_index": "test-2017-03",
"_type": "step",
"_id": "AVsVAHiaaDmnhEi3LNvC",
"_score": 1,
"fields": {
"test1": [
false
] } }, ....

Test 4 (Add the script fields in a saved search):

Trying to add the script fields directly in the kibanaSavedObjectMeta.searchSourceJSON of a saved search

Don't work when we try to open this search in kibana. Same Kibana error "Courier Fetch: 1 of 5 shards failed". Nothing in elasticsearch or kibana logs

Thank you for your help,

[mfillion (Mathieu)]

I've created an issue : https://github.com/elastic/kibana/issues/11016

[weltenwort (Felix Stürmer)]

Hi @mfillion,

what is the type of the field processContext.componentName in the mapping? Is the type consistent across all indices matching the index pattern used?

[mfillion (Mathieu)]

Hi, the field is mapped like this :

"componentName": {
"type": "keyword",
"ignore_above": 5000
}

Yes, the type is consistent across all indices.

[weltenwort (Felix Stürmer)]

Is processContext of type object (default if no type is set) or nested?

Also, does the full processContext.componentName field exist in all documents? The presence of ignore_above indicates this might not be the case. Otherwise the script might have to contain a guard against that.

[mfillion (Mathieu)]

processContext is of type object.

The processContext.componentName is in all documents. I've tried to put a guard based on a doc.contain... but the error was the same.

If you watch my case 3, it works in dev console. I don't understand why here and not in the script field...

Thanks for your help,

[weltenwort (Felix Stürmer)]

Thank you for trying that out. This is a curious problem indeed. Maybe you can inspect the network traffic using the browser developer tools, take the query that the Discover app sends and execute it in the dev console? That way we could try to reduce the query to a minimal failing example.