# CompileException - out of heap space - tried to access class ScriptDocValues$Strings$1 from class ASMAccessorImpl

**URL:** <https://discuss.elastic.co/t/compileexception-out-of-heap-space-tried-to-access-class-scriptdocvalues-strings-1-from-class-asmaccessorimpl/14180>\
**Category:** Elasticsearch\
**Created:** [October 31, 2013, 12:30am UTC](https://discuss.elastic.co/t/compileexception-out-of-heap-space-tried-to-access-class-scriptdocvalues-strings-1-from-class-asmaccessorimpl/14180 "2013-10-31T00:30:11Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![benmccann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benmccann/32/420_2.png) [@benmccann](https://discuss.elastic.co/u/benmccann)\
**Post date:** [October 31, 2013, 12:30am UTC](https://discuss.elastic.co/t/compileexception-out-of-heap-space-tried-to-access-class-scriptdocvalues-strings-1-from-class-asmaccessorimpl/14180/1 "2013-10-31T00:30:11Z")

</div>

I'm trying to boost my query to prefer documents with a longer array for a  
certain field, but it's failing to execute.

Any ideas what I might be doing wrong? Here's my code:

```
// Create query
BoolQueryBuilder query = QueryBuilders.boolQuery();
query.should(QueryBuilders.matchQuery("myField1", searchFieldValue1));
query.should(QueryBuilders.matchQuery("myField2", searchFieldValue2));

// Boost score based on array length
ScriptScoreFunctionBuilder script = new ScriptScoreFunctionBuilder();
script.script("_score * doc['myArray.myObjField'].values.length");

// Create final scoring function
FunctionScoreQueryBuilder function = 

```

QueryBuilders.functionScoreQuery(query, script);

This gives me the following errors:

org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to  
execute phase [query], all shards failed; shardFailures  
{[Bfi-HZIRQuWIZ6\_ApL7sag][myIndex][3]:  
QueryPhaseExecutionException[[myIndex][3]: query[filtered(function score  
(myField1:searchFieldValue1,function=script[\_score \*  
doc['myArray.myObjField'].values.length], params  
[null]))-\>cache(\_type:person)],from[0],size[10]: Query Failed [Failed to  
execute main query]]; nested: CompileException[[Error:  
java.lang.OutOfMemoryError: Java heap space]  
[Near : {... \_score \* doc['myArray.my ....}]  
^  
[Line: 1, Column: 1]]; nested:  
ElasticSearchException[java.lang.OutOfMemoryError: Java heap space];  
nested: ExecutionError[java.lang.OutOfMemoryError: Java heap space];  
nested: OutOfMemoryError[Java heap space];  
}{[Bfi-HZIRQuWIZ6\_ApL7sag][myIndex][4]: ElasticSearchException[tried to  
access class org.elasticsearch.index.fielddata.ScriptDocValues$Strings$1  
from class ASMAccessorImpl\_4033620681383178478830]; nested:  
IllegalAccessError[tried to access class  
org.elasticsearch.index.fielddata.ScriptDocValues$Strings$1 from class  
ASMAccessorImpl\_4033620681383178478830];  
}{[Bfi-HZIRQuWIZ6\_ApL7sag][myIndex][1]:  
QueryPhaseExecutionException[[myIndex][1]: query[filtered(function score  
(myField1:searchFieldValue1,function=script[\_score \*  
doc['myArray.myObjField'].values.length], params  
[null]))-\>cache(\_type:person)],from[0],size[10]: Query Failed [Failed to  
execute main query]]; nested: CompileException[[Error:  
java.lang.OutOfMemoryError: Java heap space]  
[Near : {... \_score \* doc['myArray.my ....}]  
^  
[Line: 1, Column: 1]]; nested:  
ElasticSearchException[java.lang.OutOfMemoryError: Java heap space];  
nested: ExecutionError[java.lang.OutOfMemoryError: Java heap space];  
nested: OutOfMemoryError[Java heap space];  
}{[Bfi-HZIRQuWIZ6\_ApL7sag][myIndex][2]:  
QueryPhaseExecutionException[[myIndex][2]: query[filtered(function score  
(myField1:searchFieldValue1,function=script[\_score \*  
doc['myArray.myObjField'].values.length], params  
[null]))-\>cache(\_type:person)],from[0],size[10]: Query Failed [Failed to  
execute main query]]; nested: CompileException[[Error:  
java.lang.OutOfMemoryError: Java heap space]  
[Near : {... \_score \* doc['myArray.my ....}]  
^  
[Line: 1, Column: 1]]; nested:  
ElasticSearchException[java.lang.OutOfMemoryError: Java heap space];  
nested: ExecutionError[java.lang.OutOfMemoryError: Java heap space];  
nested: OutOfMemoryError[Java heap space];  
}{[Bfi-HZIRQuWIZ6\_ApL7sag][myIndex][0]: ElasticSearchException[tried to  
access class org.elasticsearch.index.fielddata.ScriptDocValues$Strings$1  
from class ASMAccessorImpl\_3774454791383178471610]; nested:  
IllegalAccessError[tried to access class  
org.elasticsearch.index.fielddata.ScriptDocValues$Strings$1 from class  
ASMAccessorImpl\_3774454791383178471610]; }

Thanks,  
Ben

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![benmccann](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benmccann/32/420_2.png) [@benmccann](https://discuss.elastic.co/u/benmccann)\
**Post date:** [October 31, 2013, 1:39am UTC](https://discuss.elastic.co/t/compileexception-out-of-heap-space-tried-to-access-class-scriptdocvalues-strings-1-from-class-asmaccessorimpl/14180/2 "2013-10-31T01:39:44Z")

</div>

This bug is the only thing I can find that looks at all related. Using  
.size() instead of .length doesn't solve the problem for me though

> <https://github.com/elastic/elasticsearch/issues/3094>
>
> Hi,
> 
> We have a 2-node ES cluster running 0.90.0 release in our QA environment. …When we started using a script filter in our query, we got the following stack trace after a few queries to ES. After looking into the issue a bit, the issue seemed to be caused by the use of MVEL in compiled or accelerated mode. Further research seems to indicate that disabling JIT for MVEL worked around the problem (via -Dmvel2.disable.jit=true).
> 
> Disabling MVEL JIT would also mean a performance concern for us. What's root cause of this issue? How should we address it?
> 
> Our QA environment.
> 
> \<b\>OS\</b\>
> Ubuntu 12.10 (GNU/Linux 3.5.0-17-generic x86\_64)
> 
> \<b\>Java version, vendor\</b\>
> java version "1.7.0\_21"
> OpenJDK Runtime Environment (IcedTea 2.3.9) (7u21-2.3.9-0ubuntu0.12.10.1)
> OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)
> 
> \<b\>Script\</b\>
> 
> \<pre\>
> if(doc\['targeting.keys'\] == null || doc\['targeting.keys'\].values.length == 0)
> return true;
> foreach (kv : doc\['targeting.keys'\].values) {
> var found = 0;
> foreach (qkey : qkeys) {
> if (kv == qkey) {
> found = 1;
> }
> }
> if(found == 0){
> return false;
> }
> }
> return true;
> \</pre\>
> 
> 
> \<b\>Stack Trace\</b\>
> 
> \<pre\>
> \[2013-05-23 21:04:55,650\]\[DEBUG\]\[action.search.type \] \[Omen\] \[indexedbuyitem\]\[0\], node\[-3mMO7JjTKKhIk92AJt\_xQ\], \[R\], s\[STARTED\]: Failed to execute \[org.elasticsearch.action.search.SearchRequest@1d5e469d\]
> 
> java.lang.IllegalAccessError: org/elasticsearch/index/fielddata/ScriptDocValues$Strings$1
> at ASMAccessorImpl\_1471145441369339522430.getValue(Unknown Source)
> at org.elasticsearch.common.mvel2.optimizers.dynamic.DynamicGetAccessor.getValue(DynamicGetAccessor.java:73)
> at org.elasticsearch.common.mvel2.ast.ASTNode.getReducedValueAccelerated(ASTNode.java:108)
> at org.elasticsearch.common.mvel2.ast.BinaryOperation.getReducedValueAccelerated(BinaryOperation.java:108)
> at org.elasticsearch.common.mvel2.ast.Or.getReducedValueAccelerated(Or.java:34)
> at org.elasticsearch.common.mvel2.ast.Or.getReducedValueAccelerated(Or.java:34)
> at org.elasticsearch.common.mvel2.MVELRuntime.execute(MVELRuntime.java:85)
> at org.elasticsearch.common.mvel2.compiler.CompiledExpression.getDirectValue(CompiledExpression.java:123)
> at org.elasticsearch.common.mvel2.compiler.CompiledExpression.getValue(CompiledExpression.java:119)
> at org.elasticsearch.script.mvel.MvelScriptEngineService$MvelSearchScript.run(MvelScriptEngineService.java:192)
> at org.elasticsearch.index.query.ScriptFilterParser$ScriptFilter$ScriptDocSet.matchDoc(ScriptFilterParser.java:184)
> at org.elasticsearch.common.lucene.docset.MatchDocIdSet.get(MatchDocIdSet.java:67)
> at org.apache.lucene.search.FilteredDocIdSet$1.get(FilteredDocIdSet.java:65)
> at org.elasticsearch.common.lucene.docset.AndDocIdSet$AndBits.get(AndDocIdSet.java:106)
> at org.elasticsearch.common.lucene.docset.AndDocIdSet$AndBits.get(AndDocIdSet.java:106)
> at org.elasticsearch.common.lucene.docset.BitsDocIdSetIterator$FilteredIterator.match(BitsDocIdSetIterator.java:59)
> at org.apache.lucene.search.FilteredDocIdSetIterator.nextDoc(FilteredDocIdSetIterator.java:60)
> at org.apache.lucene.search.FilteredDocIdSetIterator.nextDoc(FilteredDocIdSetIterator.java:59)
> at org.apache.lucene.search.ConstantScoreQuery$ConstantScorer.nextDoc(ConstantScoreQuery.java:185)
> at org.elasticsearch.common.lucene.search.function.FiltersFunctionScoreQuery$CustomBoostFactorScorer.nextDoc(FiltersFunctionScoreQuery.java:283)
> at org.apache.lucene.search.Scorer.score(Scorer.java:63)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:605)
> at org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:156)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:572)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:524)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:501)
> at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:345)
> at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:127)
> at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:239)
> at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:141)
> at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
> at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:205)
> at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.onFirstPhaseResult(TransportSearchTypeAction.java:281)
> at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$3.onFailure(TransportSearchTypeAction.java:213)
> at org.elasticsearch.search.action.SearchServiceTransportAction$2.handleException(SearchServiceTransportAction.j
> \</pre\>

I'm running Java 7 on Ubuntu 13.04:  
$ java -version  
java version "1.7.0\_25"  
OpenJDK Runtime Environment (IcedTea 2.3.10) (7u25-2.3.10-1ubuntu0.13.04.2)  
OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)

On Wednesday, October 30, 2013 5:30:11 PM UTC-7, Ben McCann wrote:

> I'm trying to boost my query to prefer documents with a longer array for a  
> certain field, but it's failing to execute.
> 
> Any ideas what I might be doing wrong? Here's my code:
> 
> ```
> // Create query
> BoolQueryBuilder query = QueryBuilders.boolQuery();
> query.should(QueryBuilders.matchQuery("myField1", searchFieldValue1));
> query.should(QueryBuilders.matchQuery("myField2", searchFieldValue2));
> 
> // Boost score based on array length
> ScriptScoreFunctionBuilder script = new ScriptScoreFunctionBuilder();
> script.script("_score * doc['myArray.myObjField'].values.length");
> 
> // Create final scoring function
> FunctionScoreQueryBuilder function = 
> 
> ```
> 
> QueryBuilders.functionScoreQuery(query, script);
> 
> This gives me the following errors:
> 
> org.elasticsearch.action.search.SearchPhaseExecutionException: Failed to  
> execute phase [query], all shards failed; shardFailures  
> {[Bfi-HZIRQuWIZ6\_ApL7sag][myIndex][3]:  
> QueryPhaseExecutionException[[myIndex][3]: query[filtered(function score  
> (myField1:searchFieldValue1,function=script[\_score \*  
> doc['myArray.myObjField'].values.length], params  
> [null]))-\>cache(\_type:person)],from[0],size[10]: Query Failed [Failed to  
> execute main query]]; nested: CompileException[[Error:  
> java.lang.OutOfMemoryError: Java heap space]  
> [Near : {... \_score \* doc['myArray.my ....}]  
> ^  
> [Line: 1, Column: 1]]; nested:  
> ElasticSearchException[java.lang.OutOfMemoryError: Java heap space];  
> nested: ExecutionError[java.lang.OutOfMemoryError: Java heap space];  
> nested: OutOfMemoryError[Java heap space];  
> }{[Bfi-HZIRQuWIZ6\_ApL7sag][myIndex][4]: ElasticSearchException[tried to  
> access class org.elasticsearch.index.fielddata.ScriptDocValues$Strings$1  
> from class ASMAccessorImpl\_4033620681383178478830]; nested:  
> IllegalAccessError[tried to access class  
> org.elasticsearch.index.fielddata.ScriptDocValues$Strings$1 from class  
> ASMAccessorImpl\_4033620681383178478830];  
> }{[Bfi-HZIRQuWIZ6\_ApL7sag][myIndex][1]:  
> QueryPhaseExecutionException[[myIndex][1]: query[filtered(function score  
> (myField1:searchFieldValue1,function=script[\_score \*  
> doc['myArray.myObjField'].values.length], params  
> [null]))-\>cache(\_type:person)],from[0],size[10]: Query Failed [Failed to  
> execute main query]]; nested: CompileException[[Error:  
> java.lang.OutOfMemoryError: Java heap space]  
> [Near : {... \_score \* doc['myArray.my ....}]  
> ^  
> [Line: 1, Column: 1]]; nested:  
> ElasticSearchException[java.lang.OutOfMemoryError: Java heap space];  
> nested: ExecutionError[java.lang.OutOfMemoryError: Java heap space];  
> nested: OutOfMemoryError[Java heap space];  
> }{[Bfi-HZIRQuWIZ6\_ApL7sag][myIndex][2]:  
> QueryPhaseExecutionException[[myIndex][2]: query[filtered(function score  
> (myField1:searchFieldValue1,function=script[\_score \*  
> doc['myArray.myObjField'].values.length], params  
> [null]))-\>cache(\_type:person)],from[0],size[10]: Query Failed [Failed to  
> execute main query]]; nested: CompileException[[Error:  
> java.lang.OutOfMemoryError: Java heap space]  
> [Near : {... \_score \* doc['myArray.my ....}]  
> ^  
> [Line: 1, Column: 1]]; nested:  
> ElasticSearchException[java.lang.OutOfMemoryError: Java heap space];  
> nested: ExecutionError[java.lang.OutOfMemoryError: Java heap space];  
> nested: OutOfMemoryError[Java heap space];  
> }{[Bfi-HZIRQuWIZ6\_ApL7sag][myIndex][0]: ElasticSearchException[tried to  
> access class org.elasticsearch.index.fielddata.ScriptDocValues$Strings$1  
> from class ASMAccessorImpl\_3774454791383178471610]; nested:  
> IllegalAccessError[tried to access class  
> org.elasticsearch.index.fielddata.ScriptDocValues$Strings$1 from class  
> ASMAccessorImpl\_3774454791383178471610]; }
> 
> Thanks,  
> Ben

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [October 31, 2013, 10:44am UTC](https://discuss.elastic.co/t/compileexception-out-of-heap-space-tried-to-access-class-scriptdocvalues-strings-1-from-class-asmaccessorimpl/14180/3 "2013-10-31T10:44:27Z")

</div>

Hey Ben,

you ran out of heap/memory. The simplest solution is always to add more  
memory to your nodes - ok, simplest solution from an application point of  
view 🙂

Obviously it would be great, if we can find out, why the OOM exception  
actually happened. Question is, did the exception happen, because you  
executed/compiled a script or did it already happen before. You should  
check your logs for that.

If it happened before, you should also find out why (facetting on an  
analyzed field, sorting on data which does not fit in memory etc). So you  
will have to query nodes stats/infos APIs to find more about your running  
system.

Another thing is, that you might want to try with an oracle JDK and see if  
you get the same problems.

--Alex

On Thu, Oct 31, 2013 at 2:39 AM, Ben McCann [benjamin.j.mccann@gmail.com](mailto:benjamin.j.mccann@gmail.com)wrote:

> This bug is the only thing I can find that looks at all related. Using  
> .size() instead of .length doesn't solve the problem for me though  
> [IllegalAccessError when using an MVEL script filter in the query · Issue #3094 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/3094)
> 
> I'm running Java 7 on Ubuntu 13.04:  
> $ java -version  
> java version "1.7.0\_25"  
> OpenJDK Runtime Environment (IcedTea 2.3.10) (7u25-2.3.10-1ubuntu0.13.04.2)  
> OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)
> 
> On Wednesday, October 30, 2013 5:30:11 PM UTC-7, Ben McCann wrote:
> 
> > I'm trying to boost my query to prefer documents with a longer array for  
> > a certain field, but it's failing to execute.
> > 
> > Any ideas what I might be doing wrong? Here's my code:
> > 
> > ```
> > // Create query
> > BoolQueryBuilder query = QueryBuilders.boolQuery();
> > query.should(QueryBuilders.**matchQuery("myField1",
> > 
> > ```
> > 
> > searchFieldValue1));  
> > query.should(QueryBuilders.\*\*matchQuery("myField2",  
> > searchFieldValue2));
> > 
> > ```
> > // Boost score based on array length
> > ScriptScoreFunctionBuilder script = new ScriptScoreFunctionBuilder();
> > script.script("_score * doc['myArray.myObjField'].**values.length");
> > 
> > // Create final scoring function
> > FunctionScoreQueryBuilder function = QueryBuilders.**functionScoreQuery(query,
> > 
> > ```
> > 
> > script);
> > 
> > This gives me the following errors:
> > 
> > org.elasticsearch.action.\*\*search.\*\*SearchPhaseExecutionException:  
> > Failed to execute phase [query], all shards failed; shardFailures  
> > {[Bfi-HZIRQuWIZ6\_ApL7sag][\*\*myIndex][3]: QueryPhaseExecutionException[[\*\*myIndex][3]:  
> > query[filtered(function score (myField1:searchFieldValue1,\*\*function=script[\_score
> > 
> > - doc['myArray.myObjField'].**values.length], params  
> > [null]))-\>cache(\_type:person)]**,from[0],size[10]: Query Failed [Failed  
> > to execute main query]]; nested: CompileException[[Error:  
> > java.lang.OutOfMemoryError: Java heap space]  
> > [Near : {... _score \* doc['myArray.my ....}]  
> > ^  
> > [Line: 1, Column: 1]]; nested: ElasticSearchException[java.\*\*lang.OutOfMemoryError:  
> > Java heap space]; nested: ExecutionError[java.lang.\*\*OutOfMemoryError:  
> > Java heap space]; nested: OutOfMemoryError[Java heap space];  
> > }{[Bfi-HZIRQuWIZ6\_ApL7sag][\*\*myIndex][4]: ElasticSearchException[tried  
> > to access class org.elasticsearch.index.\*\*fielddata.ScriptDocValues$\*\*Strings$1  
> > from class ASMAccessorImpl_**4033620681383178478830]; nested:  
> > IllegalAccessError[tried to access class org.elasticsearch.index.**  
> > fielddata.ScriptDocValues$\*\*Strings$1 from class ASMAccessorImpl\_\*\*4033620681383178478830];  
> > }{[Bfi-HZIRQuWIZ6\_ApL7sag][\*_myIndex][1]: QueryPhaseExecutionException[[_  
> > _myIndex][1]: query[filtered(function score (myField1:searchFieldValue1,_  
> > \*function=script[\_score \* doc['myArray.myObjField'].**values.length],  
> > params [null]))-\>cache(\_type:person)]**,from[0],size[10]: Query Failed  
> > [Failed to execute main query]]; nested: CompileException[[Error:  
> > java.lang.OutOfMemoryError: Java heap space]  
> > [Near : {... \_score \* doc['myArray.my ....}]  
> > ^  
> > [Line: 1, Column: 1]]; nested: ElasticSearchException[java.\*\*lang.OutOfMemoryError:  
> > Java heap space]; nested: ExecutionError[java.lang.\*\*OutOfMemoryError:  
> > Java heap space]; nested: OutOfMemoryError[Java heap space];  
> > }{[Bfi-HZIRQuWIZ6\_ApL7sag][\*_myIndex][2]: QueryPhaseExecutionException[[_  
> > _myIndex][2]: query[filtered(function score (myField1:searchFieldValue1,_  
> > \*function=script[_score \* doc['myArray.myObjField'].**values.length],  
> > params [null]))-\>cache(\_type:person)]**,from[0],size[10]: Query Failed  
> > [Failed to execute main query]]; nested: CompileException[[Error:  
> > java.lang.OutOfMemoryError: Java heap space]  
> > [Near : {... score \* doc['myArray.my ....}]  
> > ^  
> > [Line: 1, Column: 1]]; nested: ElasticSearchException[java.\*\*lang.OutOfMemoryError:  
> > Java heap space]; nested: ExecutionError[java.lang.\*\*OutOfMemoryError:  
> > Java heap space]; nested: OutOfMemoryError[Java heap space];  
> > }{[Bfi-HZIRQuWIZ6\_ApL7sag][\*\*myIndex][0]: ElasticSearchException[tried  
> > to access class org.elasticsearch.index.\*\*fielddata.ScriptDocValues$\*\*Strings$1  
> > from class ASMAccessorImpl**3774454791383178471610]; nested:  
> > IllegalAccessError[tried to access class org.elasticsearch.index.**  
> > fielddata.ScriptDocValues$\*\*Strings$1 from class ASMAccessorImpl_\*\*3774454791383178471610];  
> > }
> > 
> > Thanks,  
> > Ben
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Ben\_McCann\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ben_mccann_2/32/1097_2.png) [@Ben\_McCann\_2](https://discuss.elastic.co/u/Ben_McCann_2)\
**Post date:** [October 31, 2013, 7:16pm UTC](https://discuss.elastic.co/t/compileexception-out-of-heap-space-tried-to-access-class-scriptdocvalues-strings-1-from-class-asmaccessorimpl/14180/4 "2013-10-31T19:16:33Z")

</div>

Hi Alex,

I appreciate the response. It sounds like you're suggesting that if I get  
an OOM exception once, than all future requests will fail stating there's  
been an OOM exception? That's not the case for me. I can run other queries  
and only this one fails, so it seems pretty clear the problem is with this  
query specifically.

I've attached the log from my elasticsearch server, which has some more  
information about the problem. I'm really stuck on this, so I'm hoping your  
expertise will get me pointed in the right direction 🙂

Thanks,  
Ben

On Thu, Oct 31, 2013 at 3:44 AM, Alexander Reelsen [alr@spinscale.de](mailto:alr@spinscale.de) wrote:

> Hey Ben,
> 
> you ran out of heap/memory. The simplest solution is always to add more  
> memory to your nodes - ok, simplest solution from an application point of  
> view 🙂
> 
> Obviously it would be great, if we can find out, why the OOM exception  
> actually happened. Question is, did the exception happen, because you  
> executed/compiled a script or did it already happen before. You should  
> check your logs for that.
> 
> If it happened before, you should also find out why (facetting on an  
> analyzed field, sorting on data which does not fit in memory etc). So you  
> will have to query nodes stats/infos APIs to find more about your running  
> system.
> 
> Another thing is, that you might want to try with an oracle JDK and see if  
> you get the same problems.
> 
> --Alex
> 
> On Thu, Oct 31, 2013 at 2:39 AM, Ben McCann [benjamin.j.mccann@gmail.com](mailto:benjamin.j.mccann@gmail.com)wrote:
> 
> > This bug is the only thing I can find that looks at all related. Using  
> > .size() instead of .length doesn't solve the problem for me though  
> > [IllegalAccessError when using an MVEL script filter in the query · Issue #3094 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/3094)
> > 
> > I'm running Java 7 on Ubuntu 13.04:  
> > $ java -version  
> > java version "1.7.0\_25"  
> > OpenJDK Runtime Environment (IcedTea 2.3.10)  
> > (7u25-2.3.10-1ubuntu0.13.04.2)  
> > OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)
> > 
> > On Wednesday, October 30, 2013 5:30:11 PM UTC-7, Ben McCann wrote:
> > 
> > > I'm trying to boost my query to prefer documents with a longer array for  
> > > a certain field, but it's failing to execute.
> > > 
> > > Any ideas what I might be doing wrong? Here's my code:
> > > 
> > > ```
> > > // Create query
> > > BoolQueryBuilder query = QueryBuilders.boolQuery();
> > > query.should(QueryBuilders.**matchQuery("myField1",
> > > 
> > > ```
> > > 
> > > searchFieldValue1));  
> > > query.should(QueryBuilders.\*\*matchQuery("myField2",  
> > > searchFieldValue2));
> > > 
> > > ```
> > > // Boost score based on array length
> > > ScriptScoreFunctionBuilder script = new ScriptScoreFunctionBuilder();
> > > script.script("_score * doc['myArray.myObjField'].**values.length");
> > > 
> > > // Create final scoring function
> > > FunctionScoreQueryBuilder function = QueryBuilders.**functionScoreQuery(query,
> > > 
> > > ```
> > > 
> > > script);
> > > 
> > > This gives me the following errors:
> > > 
> > > org.elasticsearch.action.\*\*search.\*\*SearchPhaseExecutionException:  
> > > Failed to execute phase [query], all shards failed; shardFailures  
> > > {[Bfi-HZIRQuWIZ6\_ApL7sag][\*_myIndex][3]: QueryPhaseExecutionException[[_  
> > > \*myIndex][3]: query[filtered(function score (myField1:searchFieldValue1,  
> > > \*\*function=script[_score \* doc['myArray.myObjField'].**values.length],  
> > > params [null]))-\>cache(\_type:person)]**,from[0],size[10]: Query Failed  
> > > [Failed to execute main query]]; nested: CompileException[[Error:  
> > > java.lang.OutOfMemoryError: Java heap space]  
> > > [Near : {... score \* doc['myArray.my ....}]  
> > > ^  
> > > [Line: 1, Column: 1]]; nested: ElasticSearchException[java.\*\*lang.OutOfMemoryError:  
> > > Java heap space]; nested: ExecutionError[java.lang.\*\*OutOfMemoryError:  
> > > Java heap space]; nested: OutOfMemoryError[Java heap space];  
> > > }{[Bfi-HZIRQuWIZ6\_ApL7sag][\*\*myIndex][4]: ElasticSearchException[tried  
> > > to access class org.elasticsearch.index.\*\*fielddata.ScriptDocValues$\*\*Strings$1  
> > > from class ASMAccessorImpl**4033620681383178478830]; nested:  
> > > IllegalAccessError[tried to access class org.elasticsearch.index.**  
> > > fielddata.ScriptDocValues$\*\*Strings$1 from class ASMAccessorImpl_\*\*4033620681383178478830];  
> > > }{[Bfi-HZIRQuWIZ6\_ApL7sag][\*\*myIndex][1]: QueryPhaseExecutionException[[  
> > > \*\*myIndex][1]: query[filtered(function score  
> > > (myField1:searchFieldValue1,\*\*function=script[\_score \*  
> > > doc['myArray.myObjField'].**values.length], params  
> > > [null]))-\>cache(\_type:person)]**,from[0],size[10]: Query Failed [Failed  
> > > to execute main query]]; nested: CompileException[[Error:  
> > > java.lang.OutOfMemoryError: Java heap space]  
> > > [Near : {... \_score \* doc['myArray.my ....}]  
> > > ^  
> > > [Line: 1, Column: 1]]; nested: ElasticSearchException[java.\*\*lang.OutOfMemoryError:  
> > > Java heap space]; nested: ExecutionError[java.lang.\*\*OutOfMemoryError:  
> > > Java heap space]; nested: OutOfMemoryError[Java heap space];  
> > > }{[Bfi-HZIRQuWIZ6\_ApL7sag][\*\*myIndex][2]: QueryPhaseExecutionException[[  
> > > \*\*myIndex][2]: query[filtered(function score  
> > > (myField1:searchFieldValue1,\*\*function=script[_score \*  
> > > doc['myArray.myObjField'].**values.length], params  
> > > [null]))-\>cache(\_type:person)]**,from[0],size[10]: Query Failed [Failed  
> > > to execute main query]]; nested: CompileException[[Error:  
> > > java.lang.OutOfMemoryError: Java heap space]  
> > > [Near : {... score \* doc['myArray.my ....}]  
> > > ^  
> > > [Line: 1, Column: 1]]; nested: ElasticSearchException[java.\*\*lang.OutOfMemoryError:  
> > > Java heap space]; nested: ExecutionError[java.lang.\*\*OutOfMemoryError:  
> > > Java heap space]; nested: OutOfMemoryError[Java heap space];  
> > > }{[Bfi-HZIRQuWIZ6\_ApL7sag][\*\*myIndex][0]: ElasticSearchException[tried  
> > > to access class org.elasticsearch.index.\*\*fielddata.ScriptDocValues$\*\*Strings$1  
> > > from class ASMAccessorImpl**3774454791383178471610]; nested:  
> > > IllegalAccessError[tried to access class org.elasticsearch.index.**  
> > > fielddata.ScriptDocValues$\*\*Strings$1 from class ASMAccessorImpl_\*\*3774454791383178471610];  
> > > }
> > > 
> > > Thanks,  
> > > Ben
> > > 
> > > --  
> > > You received this message because you are subscribed to the Google Groups  
> > > "elasticsearch" group.  
> > > To unsubscribe from this group and stop receiving emails from it, send an  
> > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > 
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> 
> --  
> You received this message because you are subscribed to a topic in the  
> Google Groups "elasticsearch" group.  
> To unsubscribe from this topic, visit  
> [https://groups.google.com/d/topic/elasticsearch/SyRV8SS6dDg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/SyRV8SS6dDg/unsubscribe).  
> To unsubscribe from this group and all its topics, send an email to  
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--

> **[Ben McCann on about.me](https://about.me/benmccann)**
>
> I am a Founder, Entrepreneur, and Investor in Mountain View, California. Visit my website.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Ben\_McCann\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ben_mccann_2/32/1097_2.png) [@Ben\_McCann\_2](https://discuss.elastic.co/u/Ben_McCann_2)\
**Post date:** [November 1, 2013, 12:31am UTC](https://discuss.elastic.co/t/compileexception-out-of-heap-space-tried-to-access-class-scriptdocvalues-strings-1-from-class-asmaccessorimpl/14180/5 "2013-11-01T00:31:51Z")

</div>

I just installed Oracle JDK and am having the same problem. Let me know if  
there's anything else I can do to help debug this.

Thanks,  
Ben

On Thu, Oct 31, 2013 at 12:16 PM, Ben McCann [ben@benmccann.com](mailto:ben@benmccann.com) wrote:

> Hi Alex,
> 
> I appreciate the response. It sounds like you're suggesting that if I get  
> an OOM exception once, than all future requests will fail stating there's  
> been an OOM exception? That's not the case for me. I can run other queries  
> and only this one fails, so it seems pretty clear the problem is with this  
> query specifically.
> 
> I've attached the log from my elasticsearch server, which has some more  
> information about the problem. I'm really stuck on this, so I'm hoping your  
> expertise will get me pointed in the right direction 🙂
> 
> Thanks,  
> Ben
> 
> On Thu, Oct 31, 2013 at 3:44 AM, Alexander Reelsen [alr@spinscale.de](mailto:alr@spinscale.de)wrote:
> 
> > Hey Ben,
> > 
> > you ran out of heap/memory. The simplest solution is always to add more  
> > memory to your nodes - ok, simplest solution from an application point of  
> > view 🙂
> > 
> > Obviously it would be great, if we can find out, why the OOM exception  
> > actually happened. Question is, did the exception happen, because you  
> > executed/compiled a script or did it already happen before. You should  
> > check your logs for that.
> > 
> > If it happened before, you should also find out why (facetting on an  
> > analyzed field, sorting on data which does not fit in memory etc). So you  
> > will have to query nodes stats/infos APIs to find more about your running  
> > system.
> > 
> > Another thing is, that you might want to try with an oracle JDK and see  
> > if you get the same problems.
> > 
> > --Alex
> > 
> > On Thu, Oct 31, 2013 at 2:39 AM, Ben McCann [benjamin.j.mccann@gmail.com](mailto:benjamin.j.mccann@gmail.com)wrote:
> > 
> > > This bug is the only thing I can find that looks at all related. Using  
> > > .size() instead of .length doesn't solve the problem for me though  
> > > [IllegalAccessError when using an MVEL script filter in the query · Issue #3094 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/3094)
> > > 
> > > I'm running Java 7 on Ubuntu 13.04:  
> > > $ java -version  
> > > java version "1.7.0\_25"  
> > > OpenJDK Runtime Environment (IcedTea 2.3.10)  
> > > (7u25-2.3.10-1ubuntu0.13.04.2)  
> > > OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)
> > > 
> > > On Wednesday, October 30, 2013 5:30:11 PM UTC-7, Ben McCann wrote:
> > > 
> > > > I'm trying to boost my query to prefer documents with a longer array  
> > > > for a certain field, but it's failing to execute.
> > > > 
> > > > Any ideas what I might be doing wrong? Here's my code:
> > > > 
> > > > ```
> > > > // Create query
> > > > BoolQueryBuilder query = QueryBuilders.boolQuery();
> > > > query.should(QueryBuilders.**matchQuery("myField1",
> > > > 
> > > > ```
> > > > 
> > > > searchFieldValue1));  
> > > > query.should(QueryBuilders.\*\*matchQuery("myField2",  
> > > > searchFieldValue2));
> > > > 
> > > > ```
> > > > // Boost score based on array length
> > > > ScriptScoreFunctionBuilder script = new
> > > > 
> > > > ```
> > > > 
> > > > ScriptScoreFunctionBuilder();  
> > > > script.script("\_score \* doc['myArray.myObjField'].\*\*  
> > > > values.length");
> > > > 
> > > > ```
> > > > // Create final scoring function
> > > > FunctionScoreQueryBuilder function = QueryBuilders.**functionScoreQuery(query,
> > > > 
> > > > ```
> > > > 
> > > > script);
> > > > 
> > > > This gives me the following errors:
> > > > 
> > > > org.elasticsearch.action.\*\*search.\*\*SearchPhaseExecutionException:  
> > > > Failed to execute phase [query], all shards failed; shardFailures  
> > > > {[Bfi-HZIRQuWIZ6\_ApL7sag][\*\*myIndex][3]: QueryPhaseExecutionException[[  
> > > > \*\*myIndex][3]: query[filtered(function score  
> > > > (myField1:searchFieldValue1,\*\*function=script[_score \*  
> > > > doc['myArray.myObjField'].**values.length], params  
> > > > [null]))-\>cache(\_type:person)]**,from[0],size[10]: Query Failed  
> > > > [Failed to execute main query]]; nested: CompileException[[Error:  
> > > > java.lang.OutOfMemoryError: Java heap space]  
> > > > [Near : {... score \* doc['myArray.my ....}]  
> > > > ^  
> > > > [Line: 1, Column: 1]]; nested: ElasticSearchException[java.\*\*lang.OutOfMemoryError:  
> > > > Java heap space]; nested: ExecutionError[java.lang.\*\*OutOfMemoryError:  
> > > > Java heap space]; nested: OutOfMemoryError[Java heap space];  
> > > > }{[Bfi-HZIRQuWIZ6\_ApL7sag][\*\*myIndex][4]: ElasticSearchException[tried  
> > > > to access class org.elasticsearch.index.\*\*fielddata.ScriptDocValues$\*\*Strings$1  
> > > > from class ASMAccessorImpl**4033620681383178478830]; nested:  
> > > > IllegalAccessError[tried to access class org.elasticsearch.index.**  
> > > > fielddata.ScriptDocValues$\*\*Strings$1 from class ASMAccessorImpl_\*\*4033620681383178478830];  
> > > > }{[Bfi-HZIRQuWIZ6\_ApL7sag][\*\*myIndex][1]:  
> > > > QueryPhaseExecutionException[[\*\*myIndex][1]: query[filtered(function  
> > > > score (myField1:searchFieldValue1,\*\*function=script[\_score \*  
> > > > doc['myArray.myObjField'].**values.length], params  
> > > > [null]))-\>cache(\_type:person)]**,from[0],size[10]: Query Failed  
> > > > [Failed to execute main query]]; nested: CompileException[[Error:  
> > > > java.lang.OutOfMemoryError: Java heap space]  
> > > > [Near : {... \_score \* doc['myArray.my ....}]  
> > > > ^  
> > > > [Line: 1, Column: 1]]; nested: ElasticSearchException[java.\*\*lang.OutOfMemoryError:  
> > > > Java heap space]; nested: ExecutionError[java.lang.\*\*OutOfMemoryError:  
> > > > Java heap space]; nested: OutOfMemoryError[Java heap space];  
> > > > }{[Bfi-HZIRQuWIZ6\_ApL7sag][\*\*myIndex][2]:  
> > > > QueryPhaseExecutionException[[\*\*myIndex][2]: query[filtered(function  
> > > > score (myField1:searchFieldValue1,\*\*function=script[_score \*  
> > > > doc['myArray.myObjField'].**values.length], params  
> > > > [null]))-\>cache(\_type:person)]**,from[0],size[10]: Query Failed  
> > > > [Failed to execute main query]]; nested: CompileException[[Error:  
> > > > java.lang.OutOfMemoryError: Java heap space]  
> > > > [Near : {... score \* doc['myArray.my ....}]  
> > > > ^  
> > > > [Line: 1, Column: 1]]; nested: ElasticSearchException[java.\*\*lang.OutOfMemoryError:  
> > > > Java heap space]; nested: ExecutionError[java.lang.\*\*OutOfMemoryError:  
> > > > Java heap space]; nested: OutOfMemoryError[Java heap space];  
> > > > }{[Bfi-HZIRQuWIZ6\_ApL7sag][\*\*myIndex][0]: ElasticSearchException[tried  
> > > > to access class org.elasticsearch.index.\*\*fielddata.ScriptDocValues$\*\*Strings$1  
> > > > from class ASMAccessorImpl**3774454791383178471610]; nested:  
> > > > IllegalAccessError[tried to access class org.elasticsearch.index.**  
> > > > fielddata.ScriptDocValues$\*\*Strings$1 from class ASMAccessorImpl_\*\*3774454791383178471610];  
> > > > }
> > > > 
> > > > Thanks,  
> > > > Ben
> > > > 
> > > > --  
> > > > You received this message because you are subscribed to the Google  
> > > > Groups "elasticsearch" group.  
> > > > To unsubscribe from this group and stop receiving emails from it, send  
> > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > > 
> > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > 
> > --  
> > You received this message because you are subscribed to a topic in the  
> > Google Groups "elasticsearch" group.  
> > To unsubscribe from this topic, visit  
> > [https://groups.google.com/d/topic/elasticsearch/SyRV8SS6dDg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/SyRV8SS6dDg/unsubscribe).  
> > To unsubscribe from this group and all its topics, send an email to  
> > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> 
> --  
> [Ben McCann - Mountain View, California, Connectifier, Carnegie Mellon University | about.me](http://about.me/benmccann)

--

> **[Ben McCann on about.me](https://about.me/benmccann)**
>
> I am a Founder, Entrepreneur, and Investor in Mountain View, California. Visit my website.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\
**Post date:** [November 1, 2013, 2:39pm UTC](https://discuss.elastic.co/t/compileexception-out-of-heap-space-tried-to-access-class-scriptdocvalues-strings-1-from-class-asmaccessorimpl/14180/6 "2013-11-01T14:39:40Z")

</div>

Hey,

only quick checking your log, it seems the exception stems from

at  
org.elasticsearch.index.fielddata.AbstractIndexFieldData.load(AbstractIndexFieldData.java:60)

which means, it did occur while you tried to load data into memory which  
was too big.

Can you provide some more information what field you are loading? How this  
field looks like? What is the mapping of this field?  
If you facet on this field (without executing a script), do you go OOM as  
well? What is the expected cardinality of the field you are using in the  
script?

As a (slow as hell, so not for production) workaround you could try to use  
\_source.field instead of doc['field'] - which should prevent the OOM (but  
requires a disk seek for each found document).

--Alex

On Fri, Nov 1, 2013 at 1:31 AM, Ben McCann [ben@benmccann.com](mailto:ben@benmccann.com) wrote:

> I just installed Oracle JDK and am having the same problem. Let me know if  
> there's anything else I can do to help debug this.
> 
> Thanks,  
> Ben
> 
> On Thu, Oct 31, 2013 at 12:16 PM, Ben McCann [ben@benmccann.com](mailto:ben@benmccann.com) wrote:
> 
> > Hi Alex,
> > 
> > I appreciate the response. It sounds like you're suggesting that if I get  
> > an OOM exception once, than all future requests will fail stating there's  
> > been an OOM exception? That's not the case for me. I can run other queries  
> > and only this one fails, so it seems pretty clear the problem is with this  
> > query specifically.
> > 
> > I've attached the log from my elasticsearch server, which has some more  
> > information about the problem. I'm really stuck on this, so I'm hoping your  
> > expertise will get me pointed in the right direction 🙂
> > 
> > Thanks,  
> > Ben
> > 
> > On Thu, Oct 31, 2013 at 3:44 AM, Alexander Reelsen [alr@spinscale.de](mailto:alr@spinscale.de)wrote:
> > 
> > > Hey Ben,
> > > 
> > > you ran out of heap/memory. The simplest solution is always to add more  
> > > memory to your nodes - ok, simplest solution from an application point of  
> > > view 🙂
> > > 
> > > Obviously it would be great, if we can find out, why the OOM exception  
> > > actually happened. Question is, did the exception happen, because you  
> > > executed/compiled a script or did it already happen before. You should  
> > > check your logs for that.
> > > 
> > > If it happened before, you should also find out why (facetting on an  
> > > analyzed field, sorting on data which does not fit in memory etc). So you  
> > > will have to query nodes stats/infos APIs to find more about your running  
> > > system.
> > > 
> > > Another thing is, that you might want to try with an oracle JDK and see  
> > > if you get the same problems.
> > > 
> > > --Alex
> > > 
> > > On Thu, Oct 31, 2013 at 2:39 AM, Ben McCann \<[benjamin.j.mccann@gmail.com](mailto:benjamin.j.mccann@gmail.com)
> > > 
> > > > wrote:
> > > 
> > > > This bug is the only thing I can find that looks at all related. Using  
> > > > .size() instead of .length doesn't solve the problem for me though  
> > > > [IllegalAccessError when using an MVEL script filter in the query · Issue #3094 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/3094)
> > > > 
> > > > I'm running Java 7 on Ubuntu 13.04:  
> > > > $ java -version  
> > > > java version "1.7.0\_25"  
> > > > OpenJDK Runtime Environment (IcedTea 2.3.10)  
> > > > (7u25-2.3.10-1ubuntu0.13.04.2)  
> > > > OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)
> > > > 
> > > > On Wednesday, October 30, 2013 5:30:11 PM UTC-7, Ben McCann wrote:
> > > > 
> > > > > I'm trying to boost my query to prefer documents with a longer array  
> > > > > for a certain field, but it's failing to execute.
> > > > > 
> > > > > Any ideas what I might be doing wrong? Here's my code:
> > > > > 
> > > > > ```
> > > > > // Create query
> > > > > BoolQueryBuilder query = QueryBuilders.boolQuery();
> > > > > query.should(QueryBuilders.**matchQuery("myField1",
> > > > > 
> > > > > ```
> > > > > 
> > > > > searchFieldValue1));  
> > > > > query.should(QueryBuilders.\*\*matchQuery("myField2",  
> > > > > searchFieldValue2));
> > > > > 
> > > > > ```
> > > > > // Boost score based on array length
> > > > > ScriptScoreFunctionBuilder script = new
> > > > > 
> > > > > ```
> > > > > 
> > > > > ScriptScoreFunctionBuilder();  
> > > > > script.script("\_score \* doc['myArray.myObjField'].\*\*  
> > > > > values.length");
> > > > > 
> > > > > ```
> > > > > // Create final scoring function
> > > > > FunctionScoreQueryBuilder function = QueryBuilders.**functionScoreQuery(query,
> > > > > 
> > > > > ```
> > > > > 
> > > > > script);
> > > > > 
> > > > > This gives me the following errors:
> > > > > 
> > > > > org.elasticsearch.action.\*\*search.\*\*SearchPhaseExecutionException:  
> > > > > Failed to execute phase [query], all shards failed; shardFailures  
> > > > > {[Bfi-HZIRQuWIZ6\_ApL7sag][\*\*myIndex][3]:  
> > > > > QueryPhaseExecutionException[[\*\*myIndex][3]: query[filtered(function  
> > > > > score (myField1:searchFieldValue1,\*\*function=script[_score \*  
> > > > > doc['myArray.myObjField'].**values.length], params  
> > > > > [null]))-\>cache(\_type:person)]**,from[0],size[10]: Query Failed  
> > > > > [Failed to execute main query]]; nested: CompileException[[Error:  
> > > > > java.lang.OutOfMemoryError: Java heap space]  
> > > > > [Near : {... score \* doc['myArray.my ....}]  
> > > > > ^  
> > > > > [Line: 1, Column: 1]]; nested: ElasticSearchException[java.\*\*lang.OutOfMemoryError:  
> > > > > Java heap space]; nested: ExecutionError[java.lang.\*\*OutOfMemoryError:  
> > > > > Java heap space]; nested: OutOfMemoryError[Java heap space];  
> > > > > }{[Bfi-HZIRQuWIZ6\_ApL7sag][\*myIndex][4]:  
> > > > > ElasticSearchException[tried to access class org.elasticsearch.index.  
> > > > > \*fielddata.ScriptDocValues$\*\*Strings$1 from class ASMAccessorImpl\*\*4033620681383178478830];  
> > > > > nested: IllegalAccessError[tried to access class org.elasticsearch.index.  
> > > > > \*\*fielddata.ScriptDocValues$\*\*Strings$1 from class ASMAccessorImpl_\*\*4033620681383178478830];  
> > > > > }{[Bfi-HZIRQuWIZ6\_ApL7sag][\*\*myIndex][1]:  
> > > > > QueryPhaseExecutionException[[\*\*myIndex][1]: query[filtered(function  
> > > > > score (myField1:searchFieldValue1,\*\*function=script[\_score \*  
> > > > > doc['myArray.myObjField'].**values.length], params  
> > > > > [null]))-\>cache(\_type:person)]**,from[0],size[10]: Query Failed  
> > > > > [Failed to execute main query]]; nested: CompileException[[Error:  
> > > > > java.lang.OutOfMemoryError: Java heap space]  
> > > > > [Near : {... \_score \* doc['myArray.my ....}]  
> > > > > ^  
> > > > > [Line: 1, Column: 1]]; nested: ElasticSearchException[java.\*\*lang.OutOfMemoryError:  
> > > > > Java heap space]; nested: ExecutionError[java.lang.\*\*OutOfMemoryError:  
> > > > > Java heap space]; nested: OutOfMemoryError[Java heap space];  
> > > > > }{[Bfi-HZIRQuWIZ6\_ApL7sag][\*\*myIndex][2]:  
> > > > > QueryPhaseExecutionException[[\*\*myIndex][2]: query[filtered(function  
> > > > > score (myField1:searchFieldValue1,\*\*function=script[_score \*  
> > > > > doc['myArray.myObjField'].**values.length], params  
> > > > > [null]))-\>cache(\_type:person)]**,from[0],size[10]: Query Failed  
> > > > > [Failed to execute main query]]; nested: CompileException[[Error:  
> > > > > java.lang.OutOfMemoryError: Java heap space]  
> > > > > [Near : {... score \* doc['myArray.my ....}]  
> > > > > ^  
> > > > > [Line: 1, Column: 1]]; nested: ElasticSearchException[java.\*\*lang.OutOfMemoryError:  
> > > > > Java heap space]; nested: ExecutionError[java.lang.\*\*OutOfMemoryError:  
> > > > > Java heap space]; nested: OutOfMemoryError[Java heap space];  
> > > > > }{[Bfi-HZIRQuWIZ6\_ApL7sag][\*myIndex][0]:  
> > > > > ElasticSearchException[tried to access class org.elasticsearch.index.  
> > > > > \*fielddata.ScriptDocValues$\*\*Strings$1 from class ASMAccessorImpl\*\*3774454791383178471610];  
> > > > > nested: IllegalAccessError[tried to access class org.elasticsearch.index.  
> > > > > \*\*fielddata.ScriptDocValues$\*\*Strings$1 from class ASMAccessorImpl_\*\*3774454791383178471610];  
> > > > > }
> > > > > 
> > > > > Thanks,  
> > > > > Ben
> > > > > 
> > > > > --  
> > > > > You received this message because you are subscribed to the Google  
> > > > > Groups "elasticsearch" group.  
> > > > > To unsubscribe from this group and stop receiving emails from it, send  
> > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > > > 
> > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > 
> > > --  
> > > You received this message because you are subscribed to a topic in the  
> > > Google Groups "elasticsearch" group.  
> > > To unsubscribe from this topic, visit  
> > > [https://groups.google.com/d/topic/elasticsearch/SyRV8SS6dDg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/SyRV8SS6dDg/unsubscribe).  
> > > To unsubscribe from this group and all its topics, send an email to  
> > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > 
> > --  
> > [Ben McCann - Mountain View, California, Connectifier, Carnegie Mellon University | about.me](http://about.me/benmccann)
> 
> --  
> [Ben McCann - Mountain View, California, Connectifier, Carnegie Mellon University | about.me](http://about.me/benmccann)
> 
> --  
> You received this message because you are subscribed to the Google Groups  
> "elasticsearch" group.  
> To unsubscribe from this group and stop receiving emails from it, send an  
> email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Ben\_McCann\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ben_mccann_2/32/1097_2.png) [@Ben\_McCann\_2](https://discuss.elastic.co/u/Ben_McCann_2)\
**Post date:** [November 1, 2013, 6:33pm UTC](https://discuss.elastic.co/t/compileexception-out-of-heap-space-tried-to-access-class-scriptdocvalues-strings-1-from-class-asmaccessorimpl/14180/7 "2013-11-01T18:33:02Z")

</div>

Hey Alex,

We have hundreds of millions of documents. Each documents has an array of  
objects. I'm trying to boost the score based on the length of that array.  
To do so I'm looking at doc['arrayField.objectField'].values.length since  
it seems I can't directly look at the array length, but have to look at an  
indexed field within that array. I'd guess that arrayField has about three  
entries on average and objectField has about 50 chars for each entry.

I'd agree that org.elasticsearch.index.fielddata.  
AbstractIndexFieldData.load(AbstractIndexFieldData.java:60) seems to be the  
culprit. What data is it trying to load? Is it trying to load the whole  
index for the field into memory? How come I have this problem when looking  
at the length, but not directly searching against the field? Is it that  
searching against the field you can look at the inverted index, but to look  
at the array length you have to go the other direction starting with the  
document instead?

I'd guess there are two solutions. I can either add more memory to the  
machine or index the field length? Any recommendations on which might  
be preferable? Can I make a call to elasticsearch to figure out how big the  
index is or do I have to estimate it based off what I know about the data?  
Can I have elasticsearch store a computed index based on  
doc['arrayField.objectField'].values.length  
or do I have to update every source document?

Thanks,  
Ben

On Fri, Nov 1, 2013 at 7:39 AM, Alexander Reelsen [alr@spinscale.de](mailto:alr@spinscale.de) wrote:

> Hey,
> 
> only quick checking your log, it seems the exception stems from
> 
> at  
> org.elasticsearch.index.fielddata.AbstractIndexFieldData.load(AbstractIndexFieldData.java:60)
> 
> which means, it did occur while you tried to load data into memory which  
> was too big.
> 
> Can you provide some more information what field you are loading? How this  
> field looks like? What is the mapping of this field?  
> If you facet on this field (without executing a script), do you go OOM as  
> well? What is the expected cardinality of the field you are using in the  
> script?
> 
> As a (slow as hell, so not for production) workaround you could try to use  
> \_source.field instead of doc['field'] - which should prevent the OOM (but  
> requires a disk seek for each found document).
> 
> --Alex
> 
> On Fri, Nov 1, 2013 at 1:31 AM, Ben McCann [ben@benmccann.com](mailto:ben@benmccann.com) wrote:
> 
> > I just installed Oracle JDK and am having the same problem. Let me know  
> > if there's anything else I can do to help debug this.
> > 
> > Thanks,  
> > Ben
> > 
> > On Thu, Oct 31, 2013 at 12:16 PM, Ben McCann [ben@benmccann.com](mailto:ben@benmccann.com) wrote:
> > 
> > > Hi Alex,
> > > 
> > > I appreciate the response. It sounds like you're suggesting that if I  
> > > get an OOM exception once, than all future requests will fail stating  
> > > there's been an OOM exception? That's not the case for me. I can run other  
> > > queries and only this one fails, so it seems pretty clear the problem is  
> > > with this query specifically.
> > > 
> > > I've attached the log from my elasticsearch server, which has some more  
> > > information about the problem. I'm really stuck on this, so I'm hoping your  
> > > expertise will get me pointed in the right direction 🙂
> > > 
> > > Thanks,  
> > > Ben
> > > 
> > > On Thu, Oct 31, 2013 at 3:44 AM, Alexander Reelsen [alr@spinscale.de](mailto:alr@spinscale.de)wrote:
> > > 
> > > > Hey Ben,
> > > > 
> > > > you ran out of heap/memory. The simplest solution is always to add more  
> > > > memory to your nodes - ok, simplest solution from an application point of  
> > > > view 🙂
> > > > 
> > > > Obviously it would be great, if we can find out, why the OOM exception  
> > > > actually happened. Question is, did the exception happen, because you  
> > > > executed/compiled a script or did it already happen before. You should  
> > > > check your logs for that.
> > > > 
> > > > If it happened before, you should also find out why (facetting on an  
> > > > analyzed field, sorting on data which does not fit in memory etc). So you  
> > > > will have to query nodes stats/infos APIs to find more about your running  
> > > > system.
> > > > 
> > > > Another thing is, that you might want to try with an oracle JDK and see  
> > > > if you get the same problems.
> > > > 
> > > > --Alex
> > > > 
> > > > On Thu, Oct 31, 2013 at 2:39 AM, Ben McCann \<  
> > > > [benjamin.j.mccann@gmail.com](mailto:benjamin.j.mccann@gmail.com)\> wrote:
> > > > 
> > > > > This bug is the only thing I can find that looks at all related. Using  
> > > > > .size() instead of .length doesn't solve the problem for me though  
> > > > > [IllegalAccessError when using an MVEL script filter in the query · Issue #3094 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/3094)
> > > > > 
> > > > > I'm running Java 7 on Ubuntu 13.04:  
> > > > > $ java -version  
> > > > > java version "1.7.0\_25"  
> > > > > OpenJDK Runtime Environment (IcedTea 2.3.10)  
> > > > > (7u25-2.3.10-1ubuntu0.13.04.2)  
> > > > > OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)
> > > > > 
> > > > > On Wednesday, October 30, 2013 5:30:11 PM UTC-7, Ben McCann wrote:
> > > > > 
> > > > > > I'm trying to boost my query to prefer documents with a longer array  
> > > > > > for a certain field, but it's failing to execute.
> > > > > > 
> > > > > > Any ideas what I might be doing wrong? Here's my code:
> > > > > > 
> > > > > > ```
> > > > > > // Create query
> > > > > > BoolQueryBuilder query = QueryBuilders.boolQuery();
> > > > > > query.should(QueryBuilders.**matchQuery("myField1",
> > > > > > 
> > > > > > ```
> > > > > > 
> > > > > > searchFieldValue1));  
> > > > > > query.should(QueryBuilders.\*\*matchQuery("myField2",  
> > > > > > searchFieldValue2));
> > > > > > 
> > > > > > ```
> > > > > > // Boost score based on array length
> > > > > > ScriptScoreFunctionBuilder script = new
> > > > > > 
> > > > > > ```
> > > > > > 
> > > > > > ScriptScoreFunctionBuilder();  
> > > > > > script.script("\_score \* doc['myArray.myObjField'].\*\*  
> > > > > > values.length");
> > > > > > 
> > > > > > ```
> > > > > > // Create final scoring function
> > > > > > FunctionScoreQueryBuilder function = QueryBuilders.**functionScoreQuery(query,
> > > > > > 
> > > > > > ```
> > > > > > 
> > > > > > script);
> > > > > > 
> > > > > > This gives me the following errors:
> > > > > > 
> > > > > > org.elasticsearch.action.\*\*search.\*\*SearchPhaseExecutionException:  
> > > > > > Failed to execute phase [query], all shards failed; shardFailures  
> > > > > > {[Bfi-HZIRQuWIZ6\_ApL7sag][\*\*myIndex][3]:  
> > > > > > QueryPhaseExecutionException[[\*\*myIndex][3]: query[filtered(function  
> > > > > > score (myField1:searchFieldValue1,\*\*function=script[_score \*  
> > > > > > doc['myArray.myObjField'].**values.length], params  
> > > > > > [null]))-\>cache(\_type:person)]**,from[0],size[10]: Query Failed  
> > > > > > [Failed to execute main query]]; nested: CompileException[[Error:  
> > > > > > java.lang.OutOfMemoryError: Java heap space]  
> > > > > > [Near : {... score \* doc['myArray.my ....}]  
> > > > > > ^  
> > > > > > [Line: 1, Column: 1]]; nested: ElasticSearchException[java.\*\*lang.OutOfMemoryError:  
> > > > > > Java heap space]; nested: ExecutionError[java.lang.\*\*OutOfMemoryError:  
> > > > > > Java heap space]; nested: OutOfMemoryError[Java heap space];  
> > > > > > }{[Bfi-HZIRQuWIZ6\_ApL7sag][\*\*myIndex][4]:  
> > > > > > ElasticSearchException[tried to access class org.elasticsearch.index.  
> > > > > > \*\*fielddata.ScriptDocValues$\*\*Strings$1 from class ASMAccessorImpl\*\*4033620681383178478830];  
> > > > > > nested: IllegalAccessError[tried to access class org.elasticsearch.index.  
> > > > > > \*\*fielddata.ScriptDocValues$\*\*Strings$1 from class ASMAccessorImpl_\*\*4033620681383178478830];  
> > > > > > }{[Bfi-HZIRQuWIZ6\_ApL7sag][\*\*myIndex][1]:  
> > > > > > QueryPhaseExecutionException[[\*\*myIndex][1]: query[filtered(function  
> > > > > > score (myField1:searchFieldValue1,\*\*function=script[\_score \*  
> > > > > > doc['myArray.myObjField'].**values.length], params  
> > > > > > [null]))-\>cache(\_type:person)]**,from[0],size[10]: Query Failed  
> > > > > > [Failed to execute main query]]; nested: CompileException[[Error:  
> > > > > > java.lang.OutOfMemoryError: Java heap space]  
> > > > > > [Near : {... \_score \* doc['myArray.my ....}]  
> > > > > > ^  
> > > > > > [Line: 1, Column: 1]]; nested: ElasticSearchException[java.\*\*lang.OutOfMemoryError:  
> > > > > > Java heap space]; nested: ExecutionError[java.lang.\*\*OutOfMemoryError:  
> > > > > > Java heap space]; nested: OutOfMemoryError[Java heap space];  
> > > > > > }{[Bfi-HZIRQuWIZ6\_ApL7sag][\*\*myIndex][2]:  
> > > > > > QueryPhaseExecutionException[[\*\*myIndex][2]: query[filtered(function  
> > > > > > score (myField1:searchFieldValue1,\*\*function=script[_score \*  
> > > > > > doc['myArray.myObjField'].**values.length], params  
> > > > > > [null]))-\>cache(\_type:person)]**,from[0],size[10]: Query Failed  
> > > > > > [Failed to execute main query]]; nested: CompileException[[Error:  
> > > > > > java.lang.OutOfMemoryError: Java heap space]  
> > > > > > [Near : {... score \* doc['myArray.my ....}]  
> > > > > > ^  
> > > > > > [Line: 1, Column: 1]]; nested: ElasticSearchException[java.\*\*lang.OutOfMemoryError:  
> > > > > > Java heap space]; nested: ExecutionError[java.lang.\*\*OutOfMemoryError:  
> > > > > > Java heap space]; nested: OutOfMemoryError[Java heap space];  
> > > > > > }{[Bfi-HZIRQuWIZ6\_ApL7sag][\*\*myIndex][0]:  
> > > > > > ElasticSearchException[tried to access class org.elasticsearch.index.  
> > > > > > \*\*fielddata.ScriptDocValues$\*\*Strings$1 from class ASMAccessorImpl\*\*3774454791383178471610];  
> > > > > > nested: IllegalAccessError[tried to access class org.elasticsearch.index.  
> > > > > > \*\*fielddata.ScriptDocValues$\*\*Strings$1 from class ASMAccessorImpl_\*\*3774454791383178471610];  
> > > > > > }
> > > > > > 
> > > > > > Thanks,  
> > > > > > Ben
> > > > > > 
> > > > > > --  
> > > > > > You received this message because you are subscribed to the Google  
> > > > > > Groups "elasticsearch" group.  
> > > > > > To unsubscribe from this group and stop receiving emails from it, send  
> > > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).
> > > > > 
> > > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > > 
> > > > --  
> > > > You received this message because you are subscribed to a topic in the  
> > > > Google Groups "elasticsearch" group.  
> > > > To unsubscribe from this topic, visit  
> > > > [https://groups.google.com/d/topic/elasticsearch/SyRV8SS6dDg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/SyRV8SS6dDg/unsubscribe)  
> > > > .  
> > > > To unsubscribe from this group and all its topics, send an email to  
> > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> > > 
> > > --  
> > > [Ben McCann - Mountain View, California, Connectifier, Carnegie Mellon University | about.me](http://about.me/benmccann)
> > 
> > --  
> > [Ben McCann - Mountain View, California, Connectifier, Carnegie Mellon University | about.me](http://about.me/benmccann)
> > 
> > --  
> > You received this message because you are subscribed to the Google Groups  
> > "elasticsearch" group.  
> > To unsubscribe from this group and stop receiving emails from it, send an  
> > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).
> 
> --  
> You received this message because you are subscribed to a topic in the  
> Google Groups "elasticsearch" group.  
> To unsubscribe from this topic, visit  
> [https://groups.google.com/d/topic/elasticsearch/SyRV8SS6dDg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/SyRV8SS6dDg/unsubscribe).  
> To unsubscribe from this group and all its topics, send an email to  
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--

> **[Ben McCann on about.me](https://about.me/benmccann)**
>
> I am a Founder, Entrepreneur, and Investor in Mountain View, California. Visit my website.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [November 2, 2013, 9:31am UTC](https://discuss.elastic.co/t/compileexception-out-of-heap-space-tried-to-access-class-scriptdocvalues-strings-1-from-class-asmaccessorimpl/14180/8 "2013-11-02T09:31:35Z")

</div>

Yes, for doc access, scripts load all field values into the heap.

I'd not plug endless RAM into machines just because of scripted access to  
arrays.

Just index the length of the array alongside with the array and you're fine  
by accessing this integer field.

Jörg

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Ben\_McCann\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ben_mccann_2/32/1097_2.png) [@Ben\_McCann\_2](https://discuss.elastic.co/u/Ben_McCann_2)\
**Post date:** [November 2, 2013, 11:57am UTC](https://discuss.elastic.co/t/compileexception-out-of-heap-space-tried-to-access-class-scriptdocvalues-strings-1-from-class-asmaccessorimpl/14180/9 "2013-11-02T11:57:55Z")

</div>

Thanks Jörg. Indexing the array length certainly seems the best option. I  
have other advanced scoring I'd like to do, so I want to understand this  
well before I run into other trouble. Which operations exactly cause all  
values for a field to be loaded into the heap? Can I use a function score  
query[http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html)at  
all without this happening or is it just this one particular operation  
of my script that is troublesome?

Thanks,  
Ben

On Sat, Nov 2, 2013 at 2:31 AM, [joergprante@gmail.com](mailto:joergprante@gmail.com) \<[joergprante@gmail.com](mailto:joergprante@gmail.com)

> wrote:

> Yes, for doc access, scripts load all field values into the heap.
> 
> I'd not plug endless RAM into machines just because of scripted access to  
> arrays.
> 
> Just index the length of the array alongside with the array and you're  
> fine by accessing this integer field.
> 
> Jörg
> 
> --  
> You received this message because you are subscribed to a topic in the  
> Google Groups "elasticsearch" group.  
> To unsubscribe from this topic, visit  
> [https://groups.google.com/d/topic/elasticsearch/SyRV8SS6dDg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/SyRV8SS6dDg/unsubscribe).  
> To unsubscribe from this group and all its topics, send an email to  
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--

> **[Ben McCann on about.me](https://about.me/benmccann)**
>
> I am a Founder, Entrepreneur, and Investor in Mountain View, California. Visit my website.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [November 2, 2013, 1:10pm UTC](https://discuss.elastic.co/t/compileexception-out-of-heap-space-tried-to-access-class-scriptdocvalues-strings-1-from-class-asmaccessorimpl/14180/10 "2013-11-02T13:10:33Z")

</div>

I think it is the DocLookup class. In SearchLookup, a synthetic "doc"  
variable is handled by DocLookup. From AbstractSearchScript you can see how  
doc() is called and an index reader (AtomicReaderContext) is executed to  
load all values for a field if you access the field.

[https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/search/lookup/DocLookup.java#L102](https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/search/lookup/DocLookup.java#L102)

It does not matter if you use function score query or not. The crucial  
thing is how to obtain field values for a "simple" evaluation like array  
length from an inverted index which is not so easy as it would be in a  
direct index.

Jörg

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Ben\_McCann\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ben_mccann_2/32/1097_2.png) [@Ben\_McCann\_2](https://discuss.elastic.co/u/Ben_McCann_2)\
**Post date:** [November 2, 2013, 4:35pm UTC](https://discuss.elastic.co/t/compileexception-out-of-heap-space-tried-to-access-class-scriptdocvalues-strings-1-from-class-asmaccessorimpl/14180/11 "2013-11-02T16:35:37Z")

</div>

So it sounds like if I use doc['my\_field\_name'].value then that would work,  
but if I do anything other than access just the field value then it will  
load all the values into the cache? Or should I use \_fields.my\_field\_name  
or \_source.my\_field\_name? I've seen examples with each, but don't  
understand the differences or implications.

Thanks so much for the help!

-Ben

On Sat, Nov 2, 2013 at 6:10 AM, [joergprante@gmail.com](mailto:joergprante@gmail.com) \<[joergprante@gmail.com](mailto:joergprante@gmail.com)

> wrote:

> I think it is the DocLookup class. In SearchLookup, a synthetic "doc"  
> variable is handled by DocLookup. From AbstractSearchScript you can see how  
> doc() is called and an index reader (AtomicReaderContext) is executed to  
> load all values for a field if you access the field.
> 
> [https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/search/lookup/DocLookup.java#L102](https://github.com/elasticsearch/elasticsearch/blob/master/src/main/java/org/elasticsearch/search/lookup/DocLookup.java#L102)
> 
> It does not matter if you use function score query or not. The crucial  
> thing is how to obtain field values for a "simple" evaluation like array  
> length from an inverted index which is not so easy as it would be in a  
> direct index.
> 
> Jörg
> 
> --  
> You received this message because you are subscribed to a topic in the  
> Google Groups "elasticsearch" group.  
> To unsubscribe from this topic, visit  
> [https://groups.google.com/d/topic/elasticsearch/SyRV8SS6dDg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/SyRV8SS6dDg/unsubscribe).  
> To unsubscribe from this group and all its topics, send an email to  
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--

> **[Ben McCann on about.me](https://about.me/benmccann)**
>
> I am a Founder, Entrepreneur, and Investor in Mountain View, California. Visit my website.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [November 2, 2013, 5:11pm UTC](https://discuss.elastic.co/t/compileexception-out-of-heap-space-tried-to-access-class-scriptdocvalues-strings-1-from-class-asmaccessorimpl/14180/12 "2013-11-02T17:11:47Z")

</div>

The implications are as follows:

\_doc (or doc) access by a script loads all the field values of an indexed  
field into a cache

\_field access by a script accesses a stored field and loads the values of a  
field, but not into a cache

\_source access by a script loads the whole doc source per call into memory  
and does not cache

Therefore the conclusion is

\_doc is the fastest but requires most memory, \_field is for a stored field,  
much slower than \_doc, requires not much memory because of not caching, and  
\_source is the slowest, it loads whole source per doc and requires only a  
little memory because of not caching.

The info can be found at

[http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-script-fields.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-script-fields.html)

[http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-scripting.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-scripting.html)

Jörg

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Ben\_McCann\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ben_mccann_2/32/1097_2.png) [@Ben\_McCann\_2](https://discuss.elastic.co/u/Ben_McCann_2)\
**Post date:** [November 2, 2013, 7:03pm UTC](https://discuss.elastic.co/t/compileexception-out-of-heap-space-tried-to-access-class-scriptdocvalues-strings-1-from-class-asmaccessorimpl/14180/13 "2013-11-02T19:03:11Z")

</div>

Thanks! This is super helpful! Last question, I swear... Is the only time  
when you would use \_source instead of \_field when the field is not stored?  
In my case for boosting by array length, could I store it as a script  
field[http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-script-fields.html](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-script-fields.html)  
and  
then access in my function score query as \_field to solve the heap overflow  
problem I was facing?

Thanks again so much for the awesome help.

-Ben

On Sat, Nov 2, 2013 at 10:11 AM, [joergprante@gmail.com](mailto:joergprante@gmail.com) \<  
[joergprante@gmail.com](mailto:joergprante@gmail.com)\> wrote:

> The implications are as follows:
> 
> \_doc (or doc) access by a script loads all the field values of an indexed  
> field into a cache
> 
> \_field access by a script accesses a stored field and loads the values of  
> a field, but not into a cache
> 
> \_source access by a script loads the whole doc source per call into memory  
> and does not cache
> 
> Therefore the conclusion is
> 
> \_doc is the fastest but requires most memory, \_field is for a stored  
> field, much slower than \_doc, requires not much memory because of not  
> caching, and \_source is the slowest, it loads whole source per doc and  
> requires only a little memory because of not caching.
> 
> The info can be found at
> 
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-script-fields.html)
> 
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-scripting.html)
> 
> Jörg
> 
> --  
> You received this message because you are subscribed to a topic in the  
> Google Groups "elasticsearch" group.  
> To unsubscribe from this topic, visit  
> [https://groups.google.com/d/topic/elasticsearch/SyRV8SS6dDg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/SyRV8SS6dDg/unsubscribe).  
> To unsubscribe from this group and all its topics, send an email to  
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--

> **[Ben McCann on about.me](https://about.me/benmccann)**
>
> I am a Founder, Entrepreneur, and Investor in Mountain View, California. Visit my website.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\
**Post date:** [November 2, 2013, 8:24pm UTC](https://discuss.elastic.co/t/compileexception-out-of-heap-space-tried-to-access-class-scriptdocvalues-strings-1-from-class-asmaccessorimpl/14180/14 "2013-11-02T20:24:28Z")

</div>

Unfortunately, script\_fields cannot be used for boosting. If so, the values  
would also have to be loaded into memory. I don't know another way to  
compute an array length of field values in an inverted index. The "JSON  
array" is just a "model" for repeatable fields (multi-valued fields) in ES,  
they do not exist as an array in the index, so, functions like array length  
of field values are becoming very high resource demanding complex functions.

Jörg

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![Ben\_McCann\_2](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ben_mccann_2/32/1097_2.png) [@Ben\_McCann\_2](https://discuss.elastic.co/u/Ben_McCann_2)\
**Post date:** [November 3, 2013, 3:12am UTC](https://discuss.elastic.co/t/compileexception-out-of-heap-space-tried-to-access-class-scriptdocvalues-strings-1-from-class-asmaccessorimpl/14180/15 "2013-11-03T03:12:28Z")

</div>

Thanks!

I filed a pull request to update the \_source docs with some of the advice  
you've given:

> <https://github.com/elastic/elasticsearch/pull/4060>

I also filed a feature request to allow for storing and indexing a computed  
field:

> <https://github.com/elastic/elasticsearch/issues/4059>
>
> I'd like to be able to store a computed field. Right now the only option is to q…uery against it is to compute it and explicitly store it in my source document. E.g. I'm syncing my docs from mongodb with a river and would like to boost using the length of an array in the document, but I have to update all my docs in mongodb with the array length in order to do that. I would rather be able to have elasticsearch compute and store the field so that I do not have to update my docs in mongodb.
> 
> I've seen others asking for this as well:
> https://groups.google.com/forum/?fromgroups=#!searchin/elasticsearch/computed/elasticsearch/ro187eatzIo/lTrl4sFSJqoJ

Appreciate all the help!

-Ben

On Sat, Nov 2, 2013 at 1:24 PM, [joergprante@gmail.com](mailto:joergprante@gmail.com) \<[joergprante@gmail.com](mailto:joergprante@gmail.com)

> wrote:

> Unfortunately, script\_fields cannot be used for boosting. If so, the  
> values would also have to be loaded into memory. I don't know another way  
> to compute an array length of field values in an inverted index. The "JSON  
> array" is just a "model" for repeatable fields (multi-valued fields) in ES,  
> they do not exist as an array in the index, so, functions like array length  
> of field values are becoming very high resource demanding complex functions.
> 
> Jörg
> 
> --  
> You received this message because you are subscribed to a topic in the  
> Google Groups "elasticsearch" group.  
> To unsubscribe from this topic, visit  
> [https://groups.google.com/d/topic/elasticsearch/SyRV8SS6dDg/unsubscribe](https://groups.google.com/d/topic/elasticsearch/SyRV8SS6dDg/unsubscribe).  
> To unsubscribe from this group and all its topics, send an email to  
> [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
> For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

--

> **[Ben McCann on about.me](https://about.me/benmccann)**
>
> I am a Founder, Entrepreneur, and Investor in Mountain View, California. Visit my website.

--  
You received this message because you are subscribed to the Google Groups "elasticsearch" group.  
To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com).  
For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 2:09am UTC](https://discuss.elastic.co/t/compileexception-out-of-heap-space-tried-to-access-class-scriptdocvalues-strings-1-from-class-asmaccessorimpl/14180/16 "2017-07-06T02:09:10Z")

</div>