# Error related to scripted fields

**URL:** https://discuss.elastic.co/t/error-related-to-scripted-fields/84164
**Category:** Kibana
**Created:** [May 1, 2017, 4:15pm UTC](https://discuss.elastic.co/t/error-related-to-scripted-fields/84164 "2017-05-01T16:15:01Z")
**Posts on this page:** 10
**Page:** 1

<div class="post-metadata">

### Author: ![kabali12345](https://avatars.discourse-cdn.com/v4/letter/k/6a8cbe/32.png) [@kabali12345](https://discuss.elastic.co/u/kabali12345)
#### Post date: [May 1, 2017, 4:15pm UTC](https://discuss.elastic.co/t/error-related-to-scripted-fields/84164/1 "2017-05-01T16:15:01Z")

</div>

Hi ......all  
here is my code for scripted field  
when i save this script in kibana it throws compiler error  
is my script is correct......?

Thanks in advance

```auto
filter : {
    "nested": {
        "path": "ConfigRules",
        "filter": {
            "script": {
                "script": "def m = /(.*)/.matcher(doc['ConfigRules.ConfigRuleArn.keyword'].value);
if ( m.matches() ) {
   return m.group(1)
} else {
   return "no match"
}",
            }
        }
    }
}

```

---

<div class="post-metadata">

### Author: ![bhavyarm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bhavyarm/32/22392_2.png) [@bhavyarm](https://discuss.elastic.co/u/bhavyarm)
#### Post date: [May 1, 2017, 4:51pm UTC](https://discuss.elastic.co/t/error-related-to-scripted-fields/84164/2 "2017-05-01T16:51:17Z")

</div>

Hi,

I am not very sure about what is happening here. So, I asked a Kibana developer.

Can you please do this and let us know what happens? All you need to put in the scripted field is the part under script key:

def m = /(.\*)/.matcher(doc['ConfigRules.ConfigRuleArn.keyword'].value);  
if ( m.matches() ) {  
return m.group(1)  
} else {  
return "no match"  
}

We also have a couple of blogposts which might help: [https://www.elastic.co/blog/using-painless-kibana-scripted-fields](https://www.elastic.co/blog/using-painless-kibana-scripted-fields) and an webinar on painless: [https://www.elastic.co/content-pack](https://www.elastic.co/content-pack)

You will have to register to watch the webinar.

Thanks,  
Bhavya

---

<div class="post-metadata">

### Author: ![kabali12345](https://avatars.discourse-cdn.com/v4/letter/k/6a8cbe/32.png) [@kabali12345](https://discuss.elastic.co/u/kabali12345)
#### Post date: [May 2, 2017, 8:34am UTC](https://discuss.elastic.co/t/error-related-to-scripted-fields/84164/3 "2017-05-02T08:34:27Z")

</div>

Hi....Thank you for your response  
when i use below code for scripted field

```auto
def m = /(.*)/.matcher(doc['ConfigRules.ConfigRuleArn.keyword'].value);
if ( m.matches() ) {
return m.group(1)
} else {
return "no match"
}

```

it throws this "Courier Fetch: 3 of 5 shards failed." and  
scripted field is created but it holds no value it says "This field is present in your elasticsearch mapping but not in any documents in the search results. You may still be able to visualize or search on it." and i have not get results in discovery phase too  
the field"ConfigRules.ConfigRuleArn.keyword" is nested  
but when i make use of root fields(top level fields) like message its working fine

---

<div class="post-metadata">

### Author: ![kabali12345](https://avatars.discourse-cdn.com/v4/letter/k/6a8cbe/32.png) [@kabali12345](https://discuss.elastic.co/u/kabali12345)
#### Post date: [May 2, 2017, 8:44am UTC](https://discuss.elastic.co/t/error-related-to-scripted-fields/84164/4 "2017-05-02T08:44:30Z")

</div>

i am tried to modify the code like this

```auto
def m = /(.*)/.matcher(doc['[ConfigRules][ConfigRuleArn][keyword]'].value);
if ( m.matches() ) {
return m.group(1)
} else {
return "no match"
}

```

and second one is like this

```auto
def m = /(.*)/.matcher(doc['[ConfigRules][ConfigRuleArn.keyword]'].value);
if ( m.matches() ) {
return m.group(1)
} else {
return "no match"
}

```

both these tries are not worked these are also throws same warning which i mentioned above

---

<div class="post-metadata">

### Author: ![bhavyarm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/bhavyarm/32/22392_2.png) [@bhavyarm](https://discuss.elastic.co/u/bhavyarm)
#### Post date: [May 2, 2017, 2:56pm UTC](https://discuss.elastic.co/t/error-related-to-scripted-fields/84164/5 "2017-05-02T14:56:55Z")

</div>

Hi,

I am lost here :). Can you give us some more details on what you are trying to do here?  
Also can you add null check before matching and see what happens?

> <https://github.com/elastic/kibana/issues/11016#issuecomment-291524557>

Thanks,  
Bhavya

---

<div class="post-metadata">

### Author: ![LeeDr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leedr/32/9289_2.png) [@LeeDr](https://discuss.elastic.co/u/LeeDr)
#### Post date: [May 2, 2017, 3:22pm UTC](https://discuss.elastic.co/t/error-related-to-scripted-fields/84164/6 "2017-05-02T15:22:56Z")

</div>

Also, I would try to start out with the most simple scripted field first to make sure you're getting the data you expect before going on to a more complicated script since they can be challenging to debug.

So for example, if I just want to make sure I can use a field in a script I might start out with something like this;

`doc['geo.srcdest'].value`

And then go to Discover and add that field to the doc table view. If there are any docs that don't have this field it will fail. Then you know you need that check first.

The other thing to consider is if you can accomplish your needs without using regular expressions (just because that's something else you have to enable in your Elasticsearch config).  
Like it says here; [https://www.elastic.co/blog/using-painless-kibana-scripted-fields](https://www.elastic.co/blog/using-painless-kibana-scripted-fields)  
"Note: Whenever possible, avoid using regex expressions to extract substrings, as indexOf() operations are less resource-intensive and less error-prone. "

Lee

---

<div class="post-metadata">

### Author: ![kabali12345](https://avatars.discourse-cdn.com/v4/letter/k/6a8cbe/32.png) [@kabali12345](https://discuss.elastic.co/u/kabali12345)
#### Post date: [May 3, 2017, 8:47am UTC](https://discuss.elastic.co/t/error-related-to-scripted-fields/84164/7 "2017-05-03T08:47:21Z")

</div>

HI ......ALL  
today i tried with null check  
here are the codes that i tried  
Test-1:

```auto
def path = doc['ConfigRules.ConfigRuleArn.keyword'].value;
if (path != null) {
def m = /(.*)/.matcher(doc['ConfigRules.ConfigRuleArn.keyword'].value);
if ( m.matches() ) {
return m.group(1);
} else {
return "no match";
}
}

```

Test-2:

```auto
def path = doc['ConfigRules.ConfigRuleArn.keyword'].value;
if (path != null) {
doc['ConfigRules.ConfigRuleArn.keyword'].value
} 

```

Test-3:

```auto
if (!doc['ConfigRules.ConfigRuleArn.keyword'].empty) {
  return doc['ConfigRules.ConfigRuleArn.keyword'].value ;
}
return null;

```

These test are failed  
NOTE:  
using the doc keyword, will cause the terms for that field to be loaded to memory (cached), which will result in faster execution, but more memory consumption. Also, **the doc[...] notation only allows for simple valued fields (can’t return a json object from it) and make sense only on non-analyzed or single term based fields**.

when i checked in kibana field " **ConfigRules.ConfigRuleArn.keyword**" is under " **analyzed**"  
so am i doing mistake here....?

---

<div class="post-metadata">

### Author: ![LeeDr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leedr/32/9289_2.png) [@LeeDr](https://discuss.elastic.co/u/LeeDr)
#### Post date: [May 3, 2017, 1:35pm UTC](https://discuss.elastic.co/t/error-related-to-scripted-fields/84164/8 "2017-05-03T13:35:17Z")

</div>

Yes, I'm pretty sure that you can only use fields that are both `searchable` and `aggregatable` and not `analyzed` in scripted fields. And keywords _usually_ have those correct attributes.

If you really need that field and/or some regular expression match of it I think you'll need to go back and look at how the data is being loaded, if there's a mapping for the index, and/or if the default mapping has been changed.

And if you do have to change mapping and possibly reload data, you might also be able to parse it on the ingest side so that you don't have to use a scripted field in Kibana.

What are you using to load the data? And what version of Elasticsearch and Kibana are you on?

Regards,  
Lee

---

<div class="post-metadata">

### Author: ![kabali12345](https://avatars.discourse-cdn.com/v4/letter/k/6a8cbe/32.png) [@kabali12345](https://discuss.elastic.co/u/kabali12345)
#### Post date: [May 3, 2017, 1:42pm UTC](https://discuss.elastic.co/t/error-related-to-scripted-fields/84164/9 "2017-05-03T13:42:07Z")

</div>

Thank you for making clear about scripted fields  
i am using  
logstash 5.3.0  
kibana 5.3.0  
elasticsearch 5.3.0  
and i make use of logstash grok to create those fields and its worked  
Thank you

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 31, 2017, 1:45pm UTC](https://discuss.elastic.co/t/error-related-to-scripted-fields/84164/10 "2017-05-31T13:45:55Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
