I only have 81,000 docs, 26mb size, 1 index and 2 shards. The index metadata mapping fields are just flat listed, no nested fields.
fielddata memory_size_in_bytes: 2179736
The following query keep causing circuit breaker request time OOM exception:
org.elasticsearch.transport.RemoteTransportException: [DataTwo][inet[/127.0.0.1:9302]][indices:data/read/search[phase/query]] Caused by: org.elasticsearch.common.breaker.CircuitBreakingException: [REQUEST] Data too large, data for [<reused_arrays>] would be larger than limit of [304113254/290mb] at org.elasticsearch.common.breaker.ChildMemoryCircuitBreaker.circuitBreak(ChildMemoryCircuitBreaker.java:97) at org.elasticsearch.common.breaker.ChildMemoryCircuitBreaker.addEstimateBytesAndMaybeBreak(ChildMemoryCircuitBreaker.java:148) at org.elasticsearch.common.util.BigArrays.adjustBreaker(BigArrays.java:398) at org.elasticsearch.common.util.BigArrays.validate(BigArrays.java:435) at org.elasticsearch.common.util.BigArrays.newByteArray(BigArrays.java:460) at org.elasticsearch.common.util.BigArrays.newByteArray(BigArrays.java:468) at org.elasticsearch.search.aggregations.metrics.cardinality.HyperLogLogPlusPlus.<init>(HyperLogLogPlusPlus.java:170) ... [2016-01-28 17:17:25,334][WARN ][transport.netty ] [NoData] Message not fully read (response) for [16190] handler org.elasticsearch.search.action.SearchServiceTransportAction$6@1f8eb399, error [false], resetting
Can anyone help me take a look to this query builder and help me optimize it? Can someone also tell me why the collection model "breath_first" doesn't make difference to "depth_first".
`
final FilterBuilder filterBuilder = termFilter("month", getMostRecentMonthToQuery(indexId));
final String revenueScript = getRevenueScript(revenueModifier);
final String sortString = convertTermSortToAggregationSort(sortColumn);
final boolean isAscending = sortOrder.equalsIgnoreCase("asc");
final SubAggCollectionMode aggCollectionMode = SubAggCollectionMode.parse("breadth_first");
final SearchRequestBuilder builder = elasticSearchClient.prepareSearch(indexId)
.setTypes(TYPE_NAME)
.setQuery(QueryBuilders.filteredQuery(QueryBuilders.matchAllQuery(), filterBuilder))
.addAggregation(AggregationBuilders.terms("carCount")
.field(CAR_RAW)
.size(pageSize)
.order(Terms.Order.aggregation(sortString, isAscending))
.collectMode(aggCollectionMode)
.subAggregation(AggregationBuilders.terms("partProviderCount")
.field(PART_PROVIDER_RAW)
.size(0)
)
.subAggregation(AggregationBuilders.cardinality("partCount")
.field(PART_NUMBER_RAW)
.precisionThreshold(40000)
)
.subAggregation(AggregationBuilders.avg(IGPART_AGG_NAME)
.field(IGPART_TERM_NAME)
)
.subAggregation(AggregationBuilders.avg(CGPART_AGG_NAME)
.field(CGPART_TERM_NAME)
)
.subAggregation(AggregationBuilders.avg(OGPART_AGG_NAME)
.field(OGPART_TERM_NAME)
)
.subAggregation(AggregationBuilders.avg(CPPART_AGG_NAME)
.field(CPPART_TERM_NAME)
)
.subAggregation(AggregationBuilders.sum(IGPART_PRICE_AGG_NAME)
.field(IGPART_TERM_NAME)
.script(revenueScript)
)
.subAggregation(AggregationBuilders.sum(CGPART_PRICE_AGG_NAME)
.field(CGPART_TERM_NAME)
.script(revenueScript)
)
.subAggregation(AggregationBuilders.sum(OGPART_PRICE_AGG_NAME)
.field(OGPART_TERM_NAME)
.script(revenueScript)
)
.subAggregation(AggregationBuilders.sum(CPPART_PRICE_AGG_NAME)
.field(CPPART_TERM_NAME)
.script(revenueScript)
)
);
return builder;
`