### Author:  [@rjernst](https://discuss.elastic.co/u/rjernst)
#### Post date: [March 11, 2018, 3:44pm UTC](https://discuss.elastic.co/t/painless-scripting-compile-errors/123412/2 "2018-03-11T15:44:12Z")
Accessing fields within a scripted field is done with the special `doc` variable, but you need to continue to use it throughout your script. Also, I don't remember the "in" keyword, but you can use a for-each loop with `:`. Finally, you need to use type names, even if that type name is the dynamic `def`. For example:
```auto
for (String name : doc['parsed.FIRST_NAME']) {
if ('Steve' == name){
return name;
}
}
return 'something else';
```
---
### Author:  [@Errors](https://discuss.elastic.co/u/Errors)
#### Post date: [March 15, 2018, 9:59pm UTC](https://discuss.elastic.co/t/painless-scripting-compile-errors/123412/3 "2018-03-15T21:59:27Z")
Thanks for that, I've rebuilt my code a bit 🙂 ! Unfortunately I have another issue now. In my index I've pushed data from two different datasets (into one index, as you can't do comparisons across indexes using elasticsearch). The fields from the dataset that are pushed in first become 'missing fields' and when I'm scripting, I'm unable to access them even though I know they're there.
When I create an index that just contains data from one dataset, I can access the FIRST\_NAME field like so:
```
for (String name: params['_source']['FIRST_NAME']){
if (name != null) {
return name
}
}
```
however once I add in the additional data, I get the following error:
`Courier Fetch: 1 of 5 shards failed.`
```
Error: Request to Elasticsearch failed: {"error":{"root_cause":[{"type":"script_exception","reason":"runtime error","script_stack":["for (String name: params['_source']['FIRST_NAME']){\n "," ^---- HERE"],"script":"for (String name: params['_source']['FIRST_NAME']){\n if (name != null) {\n return name\n }\n \n}","lang":"painless"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"xmlxpathcopy","node":"UGEXdG44R0WjDyGbzrbZTg","reason":{"type":"script_exception","reason":"runtime error","script_stack":["for (String name: params['_source']['FIRST_NAME']){\n "," ^---- HERE"],"script":"for (String name: params['_source']['FIRST_NAME']){\n if (name != null) {\n return name\n }\n \n}","lang":"painless","caused_by":{"type":"null_pointer_exception","reason":null}}}]},"status":500}
at http://localhost:5601/bundles/kibana.bundle.js?v=16108:29:9038
at Function.Promise.try (http://localhost:5601/bundles/commons.bundle.js?v=16108:86:22184)
at http://localhost:5601/bundles/commons.bundle.js?v=16108:86:21568
at Array.map ()
at Function.Promise.map (http://localhost:5601/bundles/commons.bundle.js?v=16108:86:21523)
at callResponseHandlers (http://localhost:5601/bundles/kibana.bundle.js?v=16108:29:8654)
at http://localhost:5601/bundles/kibana.bundle.js?v=16108:28:29176
at processQueue (http://localhost:5601/bundles/commons.bundle.js?v=16108:39:9912)
at http://localhost:5601/bundles/commons.bundle.js?v=16108:39:10805
at Scope.$digest (http://localhost:5601/bundles/commons.bundle.js?v=16108:39:21741)
```
Is there a way to access fields that claim to be 'missing' in elasticsearch?
As an extra, when I check the mapping of the index containing both datasets, I can see the field 'FIRST\_NAME'.
---
### Author:  [@rjernst](https://discuss.elastic.co/u/rjernst)
#### Post date: [March 17, 2018, 5:04am UTC](https://discuss.elastic.co/t/painless-scripting-compile-errors/123412/4 "2018-03-17T05:04:37Z")
First, I highly recommend you do not use `_source` within a script. Depending on the type of script used (for example, a scoring script), this can be _extremly_ costly in I/O. With that said, whether you access doc values with `doc` or original values with `_source`, you need to check if the field actually exists for the document the script is currently operating on. The field existing in mappings only means elasticsearch knows how to parse the field when indexing (among other things). But if a single document doesn't contain that field (FIRST\_NAME in your case), then `doc['FIRST_NAME'].value` will return null, as would `params._source['FIRST_NAME']`.
---
### Author:  [@Errors](https://discuss.elastic.co/u/Errors)
#### Post date: [March 19, 2018, 10:12am UTC](https://discuss.elastic.co/t/painless-scripting-compile-errors/123412/5 "2018-03-19T10:12:50Z")
Okay I'll change that now 🙂
As for the field existing, I can search for, say, values contained within the 'FIRST\_NAME' like 'Steve', within the specified index (using the discover tab on Kibana) and I can find them. Not all documents contain this parameter, only about 1/10th of them do however they are there within the index - they just aren't being captured by the script if that makes sense.
---
### Author:  [@rjernst](https://discuss.elastic.co/u/rjernst)
#### Post date: [March 19, 2018, 11:48pm UTC](https://discuss.elastic.co/t/painless-scripting-compile-errors/123412/6 "2018-03-19T23:48:32Z")
Being able to search in a field only means the mapping exists for the field. If a particular document does not have a _value_ for the field, you cannot use `doc['FIRST_NAME'].value` as a blanket statement. You need to qualify this by checking if `FIRST_NAME` exists in `doc` like:
```auto
if (doc.containsKey('FIRST_NAME')) {
return doc['FIRST_NAME'].value;
}
return 'some default value';
```
---
### Author:  [@Tibo](https://discuss.elastic.co/u/Tibo)
#### Post date: [March 21, 2018, 1:25pm UTC](https://discuss.elastic.co/t/painless-scripting-compile-errors/123412/7 "2018-03-21T13:25:01Z")
I have come across the same error trying to use painless scripting on string data. Thanks for the help so far.
That's what my script currently looks like :
> {
> "script": {
> "lang": "painless",
> "inline": "if (doc.containsKey('request')){return doc['request'];} else return -1;"
> }
> }
And I still get the error : `{"type":"index_out_of_bounds_exception","reason":null}}}]},"status":500}`
When I replace the line by `"if (doc.containsKey('request')){return 'test';} else return -1;"` on the same data sample, it works and returns 100% 'test'.
How can I safely access `doc['request'].value` without this `index_out_of_bounds_exception` ?
---
### Author:  [@rjernst](https://discuss.elastic.co/u/rjernst)
#### Post date: [March 21, 2018, 4:12pm UTC](https://discuss.elastic.co/t/painless-scripting-compile-errors/123412/8 "2018-03-21T16:12:34Z")
The code you pasted above returns `doc['request']`, not `doc['request'].value`. Have you tried that latter? Also, do you see any errors in your elasticsearch log?
---
### Author:  [@Tibo](https://discuss.elastic.co/u/Tibo)
#### Post date: [March 21, 2018, 4:42pm UTC](https://discuss.elastic.co/t/painless-scripting-compile-errors/123412/9 "2018-03-21T16:42:21Z")
Yes I have tried `doc['request'].value`, the above code is just a simplification in order to keep only the error part.
I do see an error in my elasticsearch log, and a big one (~200 lines).
The following error happened with the code from my previous post :
> [2018-03-21T17:25:52,565][DEBUG][o.e.a.s.TransportSearchAction] [Mona01] [prodapacheflorence-2018.02.28][0], node[r8Zt9kl\_QzGACqU2DBs9Ow], [R], s[ST ARTED], a[id=GmLJXFqtRSqCwb-CV1Mixw]: Failed to execute [SearchRequest{searchType=QUERY\_THEN\_FETCH, indices=[prodapacheflorence-2018.02.27, prodapac heflorence-2018.02.28], indicesOptions=IndicesOptions[id=39, ignore\_unavailable=true, allow\_no\_indices=true, expand\_wildcards\_open=true, expand\_wild cards\_closed=false, allow\_alisases\_to\_multiple\_indices=true, forbid\_closed\_indices=true], types=, routing='null', preference='1521643103178', requ estCache=null, scroll=null, source={
> ...
> "aggregations" : {
> "5" : {
> "terms" : {
> "field" : "request",
> "script" : {
> "inline" : "if (doc.containsKey('request')){return doc['request'];} else return -1;",
> "lang" : "painless"
> },
> "size" : 5,
> "min\_doc\_count" : 1,
> "shard\_min\_doc\_count" : 0,
> "show\_term\_doc\_count\_error" : false,
> "order" : [
> {
> "\_count" : "desc"
> },
> {
> "\_term" : "asc"
> ...
> org.elasticsearch.transport.RemoteTransportException: [Mona01][10.224.4.13:9300][indices:data/read/search[phase/query]]
> Caused by: org.elasticsearch.search.query.QueryPhaseExecutionException: Query Failed [Failed to execute main query]
> at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:414) ~[elasticsearch-5.5.1.jar:5.5.1]
> at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:108) ~[elasticsearch-5.5.1.jar:5.5.1]
> ...
> Caused by: org.elasticsearch.script.ScriptException: runtime error
> at org.elasticsearch.painless.PainlessScript.convertToScriptException(PainlessScript.java:101) ~[?:?]
> ...
> Caused by: java.lang.IndexOutOfBoundsException
> at java.nio.Buffer.checkIndex(Buffer.java:546) ~[?:1.8.0\_161]
> at java.nio.DirectByteBuffer.getShort(DirectByteBuffer.java:594) ~[?:1.8.0\_161]
> ...
I have some difficulties to understand how it can get through the condition `doc.containsKey('request')` and still have null error when evaluating `doc['request']`
---
### Author:  [@rjernst](https://discuss.elastic.co/u/rjernst)
#### Post date: [March 21, 2018, 6:13pm UTC](https://discuss.elastic.co/t/painless-scripting-compile-errors/123412/10 "2018-03-21T18:13:50Z")
Can you please paste a link to the _entire_ error log message (all stack frames)? Do not remove any lines.
---
### Author:  [@rjernst](https://discuss.elastic.co/u/rjernst)
#### Post date: [March 21, 2018, 6:15pm UTC](https://discuss.elastic.co/t/painless-scripting-compile-errors/123412/11 "2018-03-21T18:15:19Z")
Also, if you are pasting here instead of in a gist or something like that, please put it in triple backticks (this will make it monospaced). Eg:
```auto
Caused by: whatever.Exception
```
---
### Author:  [@Errors](https://discuss.elastic.co/u/Errors)
#### Post date: [March 22, 2018, 12:37am UTC](https://discuss.elastic.co/t/painless-scripting-compile-errors/123412/12 "2018-03-22T00:37:26Z")
Thanks, your suggestion to me worked and I can now access the variable and use them 🙂
I have a smaller question now, with painless is it possible to use the 'in' operator?
In my case I've made two arrays (take values from the fields present in elasticsearch) called name1 and name2 and I'm trying to check if any values from name1 are present in name2
I've tried the following:
```
for (i:name1){
for(j:name2){
if (name1[i] =~ name2) {
return 'Contact ' + name1[i] + ' found';
}
else {
return 'no';
}
}
}
```
Which produces this:
```
Error: Request to Elasticsearch failed: {"error":{"root_cause":[{"type":"script_exception","reason":"compile error","script_stack":["... return 'nay';\n}\n\nfor (i:name1){\n for(j:name2) ..."," ^---- HERE"],"script":"def name1 = [];\ndef name2 = [];\n\nif (doc['FIRST_NAME'].value != null){\n name1.add(doc['FIRST_NAME'].value);\n}\nif (doc['fullname'].value != null){\n name2.add(doc['fullname'].value);\n}\n\nelse {\n return 'nay';\n}\n\nfor (i:name1){\n for(j:name2){\n if (name1[i] =~ name2) {\n return 'Contact ' + name1[i] + ' found';\n }\n else {\n return 'no';\n }\n }\n}","lang":"painless"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"xmlxpathcopy","node":"UGEXdG44R0WjDyGbzrbZTg","reason":{"type":"script_exception","reason":"compile error","script_stack":["... return 'nay';\n}\n\nfor (i:name1){\n for(j:name2) ..."," ^---- HERE"],"script":"def name1 = [];\ndef name2 = [];\n\nif (doc['FIRST_NAME'].value != null){\n name1.add(doc['FIRST_NAME'].value);\n}\nif (doc['fullname'].value != null){\n name2.add(doc['fullname'].value);\n}\n\nelse {\n return 'nay';\n}\n\nfor (i:name1){\n for(j:name2){\n if (name1[i] =~ name2) {\n return 'Contact ' + name1[i] + ' found';\n }\n else {\n return 'no';\n }\n }\n}","lang":"painless","caused_by":{"type":"illegal_argument_exception","reason":"invalid sequence of tokens near [':'].","caused_by":{"type":"no_viable_alt_exception","reason":null}}}}]},"status":500}
at http://localhost:5601/bundles/kibana.bundle.js?v=16108:29:9038
at Function.Promise.try (http://localhost:5601/bundles/commons.bundle.js?v=16108:86:22184)
at http://localhost:5601/bundles/commons.bundle.js?v=16108:86:21568
at Array.map ()
at Function.Promise.map (http://localhost:5601/bundles/commons.bundle.js?v=16108:86:21523)
at callResponseHandlers (http://localhost:5601/bundles/kibana.bundle.js?v=16108:29:8654)
at http://localhost:5601/bundles/kibana.bundle.js?v=16108:28:29176
at processQueue (http://localhost:5601/bundles/commons.bundle.js?v=16108:39:9912)
at http://localhost:5601/bundles/commons.bundle.js?v=16108:39:10805
at Scope.$digest (http://localhost:5601/bundles/commons.bundle.js?v=16108:39:21741)
```
I've also tried this:
```
for (int i = 0; i < name1.length; ++i){
if (name1[i] in name2) {
return 'Contact ' + name1[i] + ' found';
}
}
```
which returns:
```
Error: Request to Elasticsearch failed: {"error":{"root_cause":[{"type":"script_exception","reason":"compile error","script_stack":["... ; ++i){\n if (name1[i] in name2) {\n retur ..."," ^---- HERE"],"script":"def name1 = [];\ndef name2 = [];\n\nif (doc['FIRST_NAME'].value != null){\n name1.add(doc['FIRST_NAME'].value);\n}\nif (doc['fullname'].value != null){\n name2.add(doc['fullname'].value);\n}\n\nelse {\n return 'nay';\n}\n\nfor (int i = 0; i < name1.length; ++i){\n if (name1[i] in name2) {\n return 'Contact ' + name1[i] + ' found';\n }\n}","lang":"painless"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"xmlxpathcopy","node":"UGEXdG44R0WjDyGbzrbZTg","reason":{"type":"script_exception","reason":"compile error","script_stack":["... ; ++i){\n if (name1[i] in name2) {\n retur ..."," ^---- HERE"],"script":"def name1 = [];\ndef name2 = [];\n\nif (doc['FIRST_NAME'].value != null){\n name1.add(doc['FIRST_NAME'].value);\n}\nif (doc['fullname'].value != null){\n name2.add(doc['fullname'].value);\n}\n\nelse {\n return 'nay';\n}\n\nfor (int i = 0; i < name1.length; ++i){\n if (name1[i] in name2) {\n return 'Contact ' + name1[i] + ' found';\n }\n}","lang":"painless","caused_by":{"type":"illegal_argument_exception","reason":"unexpected token ['in'] was expecting one of [')']."}}}]},"status":500}
at http://localhost:5601/bundles/kibana.bundle.js?v=16108:29:9038
at Function.Promise.try (http://localhost:5601/bundles/commons.bundle.js?v=16108:86:22184)
at http://localhost:5601/bundles/commons.bundle.js?v=16108:86:21568
at Array.map ()
at Function.Promise.map (http://localhost:5601/bundles/commons.bundle.js?v=16108:86:21523)
at callResponseHandlers (http://localhost:5601/bundles/kibana.bundle.js?v=16108:29:8654)
at http://localhost:5601/bundles/kibana.bundle.js?v=16108:28:29176
at processQueue (http://localhost:5601/bundles/commons.bundle.js?v=16108:39:9912)
at http://localhost:5601/bundles/commons.bundle.js?v=16108:39:10805
at Scope.$digest (http://localhost:5601/bundles/commons.bundle.js?v=16108:39:21741)
```
I've had a look online and it seems like this is a valid operator, which is leading me to believe my syntax is wrong
---
### Author:  [@rjernst](https://discuss.elastic.co/u/rjernst)
#### Post date: [March 22, 2018, 4:14am UTC](https://discuss.elastic.co/t/painless-scripting-compile-errors/123412/13 "2018-03-22T04:14:41Z")
I think there are 2 features being used not quite correctly in your examples. First, a for-each loop must have a typed lefthand side (even if that type is `def`) and the value is set to each value of the collection, not the index of the value. Second, there is no `in` operator, but Java's standard Collection interface provides a `contains` method for this purpose. I think your example would look something like this:
```auto
for (String x : name1){
if (name2.contains(x)) {
return 'Contact ' + x + ' found';
} else {
return 'no';
}
}
```
Something to keep in mind: the runtime of this loop will be quadratic on the number of names. If these are anything more than trivial lists (for example, more than a handful of elements), you should look at using `Set`s to do efficient intersection.
---
### Author:  [@Tibo](https://discuss.elastic.co/u/Tibo)
#### Post date: [March 22, 2018, 8:42am UTC](https://discuss.elastic.co/t/painless-scripting-compile-errors/123412/14 "2018-03-22T08:42:58Z")
Ok, here it is : [Full log message](https://gist.github.com/Tibo669/b5e90f8ebf2251a770fe85d8966101ae)
Hope it helps
---
### Author:  [@rjernst](https://discuss.elastic.co/u/rjernst)
#### Post date: [March 22, 2018, 5:01pm UTC](https://discuss.elastic.co/t/painless-scripting-compile-errors/123412/15 "2018-03-22T17:01:23Z")
The type of script you are using (on a terms aggregation) has a slightly different syntax. The `doc` variable is not available. Instead, use the special `_value` variable, which will have the value of that field for the document the script is being run on. Also, a terms aggregation should return the type of data that the term contains, so returning `-1` won't work. Instead, return some String value.
---
### Author:  [@Tibo](https://discuss.elastic.co/u/Tibo)
#### Post date: [March 23, 2018, 8:57am UTC](https://discuss.elastic.co/t/painless-scripting-compile-errors/123412/16 "2018-03-23T08:57:22Z")
Thanks, that's exactly what I wanted.
If it can help anyone else, that's the final script :
> "inline": "if (\_value.length() \> 12) {return \_value.substring(0,12);} else return \_value;"
Maybe that `_value` is worth being mentionned in the documentation.
Thanks
---
### Author:  [@rjernst](https://discuss.elastic.co/u/rjernst)
#### Post date: [March 23, 2018, 4:33pm UTC](https://discuss.elastic.co/t/painless-scripting-compile-errors/123412/17 "2018-03-23T16:33:04Z")
It has an [example in the docs](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#_value_script_8), albeit with little to no context.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 20, 2018, 4:33pm UTC](https://discuss.elastic.co/t/painless-scripting-compile-errors/123412/18 "2018-04-20T16:33:23Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.