Error related to scripted fields

Elastic Stack Kibana
1

Hi ......all
here is my code for scripted field
when i save this script in kibana it throws compiler error
is my script is correct......?

Thanks in advance

filter : {
    "nested": {
        "path": "ConfigRules",
        "filter": {
            "script": {
                "script": "def m = /(.*)/.matcher(doc['ConfigRules.ConfigRuleArn.keyword'].value);
if ( m.matches() ) {
   return m.group(1)
} else {
   return "no match"
}",
            }
        }
    }
}
2

Hi,

I am not very sure about what is happening here. So, I asked a Kibana developer.

Can you please do this and let us know what happens? All you need to put in the scripted field is the part under script key:

def m = /(.*)/.matcher(doc['ConfigRules.ConfigRuleArn.keyword'].value);
if ( m.matches() ) {
return m.group(1)
} else {
return "no match"
}

We also have a couple of blogposts which might help: https://www.elastic.co/blog/using-painless-kibana-scripted-fields and an webinar on painless: https://www.elastic.co/content-pack

You will have to register to watch the webinar.

Thanks,
Bhavya

3

Hi....Thank you for your response
when i use below code for scripted field

def m = /(.*)/.matcher(doc['ConfigRules.ConfigRuleArn.keyword'].value);
if ( m.matches() ) {
return m.group(1)
} else {
return "no match"
}

it throws this "Courier Fetch: 3 of 5 shards failed." and
scripted field is created but it holds no value it says "This field is present in your elasticsearch mapping but not in any documents in the search results. You may still be able to visualize or search on it." and i have not get results in discovery phase too
the field"ConfigRules.ConfigRuleArn.keyword" is nested
but when i make use of root fields(top level fields) like message its working fine

4

i am tried to modify the code like this

def m = /(.*)/.matcher(doc['[ConfigRules][ConfigRuleArn][keyword]'].value);
if ( m.matches() ) {
return m.group(1)
} else {
return "no match"
}

and second one is like this

def m = /(.*)/.matcher(doc['[ConfigRules][ConfigRuleArn.keyword]'].value);
if ( m.matches() ) {
return m.group(1)
} else {
return "no match"
}

both these tries are not worked these are also throws same warning which i mentioned above

5

Hi,

I am lost here :). Can you give us some more details on what you are trying to do here?
Also can you add null check before matching and see what happens?

Thanks,
Bhavya

6

Also, I would try to start out with the most simple scripted field first to make sure you're getting the data you expect before going on to a more complicated script since they can be challenging to debug.

So for example, if I just want to make sure I can use a field in a script I might start out with something like this;

doc['geo.srcdest'].value

And then go to Discover and add that field to the doc table view. If there are any docs that don't have this field it will fail. Then you know you need that check first.

The other thing to consider is if you can accomplish your needs without using regular expressions (just because that's something else you have to enable in your Elasticsearch config).
Like it says here; https://www.elastic.co/blog/using-painless-kibana-scripted-fields
"Note: Whenever possible, avoid using regex expressions to extract substrings, as indexOf() operations are less resource-intensive and less error-prone. "

Lee

7

HI ......ALL
today i tried with null check
here are the codes that i tried
Test-1:

def path = doc['ConfigRules.ConfigRuleArn.keyword'].value;
if (path != null) {
def m = /(.*)/.matcher(doc['ConfigRules.ConfigRuleArn.keyword'].value);
if ( m.matches() ) {
return m.group(1);
} else {
return "no match";
}
}

Test-2:

def path = doc['ConfigRules.ConfigRuleArn.keyword'].value;
if (path != null) {
doc['ConfigRules.ConfigRuleArn.keyword'].value
}

Test-3:

if (!doc['ConfigRules.ConfigRuleArn.keyword'].empty) {
  return doc['ConfigRules.ConfigRuleArn.keyword'].value ;
}
return null;

These test are failed
NOTE:
using the doc keyword, will cause the terms for that field to be loaded to memory (cached), which will result in faster execution, but more memory consumption. Also, the doc[...] notation only allows for simple valued fields (can’t return a json object from it) and make sense only on non-analyzed or single term based fields.

when i checked in kibana field "ConfigRules.ConfigRuleArn.keyword" is under "analyzed"
so am i doing mistake here....?

8

Yes, I'm pretty sure that you can only use fields that are both searchable and aggregatable and not analyzed in scripted fields. And keywords usually have those correct attributes.

If you really need that field and/or some regular expression match of it I think you'll need to go back and look at how the data is being loaded, if there's a mapping for the index, and/or if the default mapping has been changed.

And if you do have to change mapping and possibly reload data, you might also be able to parse it on the ingest side so that you don't have to use a scripted field in Kibana.

What are you using to load the data? And what version of Elasticsearch and Kibana are you on?

Regards,
Lee

1 Like
9

Thank you for making clear about scripted fields
i am using
logstash 5.3.0
kibana 5.3.0
elasticsearch 5.3.0
and i make use of logstash grok to create those fields and its worked
Thank you

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.