# Failed shards with top\_hits **URL:** \ **Category:** Elasticsearch\ **Created:** [January 21, 2018, 10:35pm UTC](https://discuss.elastic.co/t/failed-shards-with-top-hits/116407 "2018-01-21T22:35:05Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![mihalikv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mihalikv/32/26793_2.png) [@mihalikv](https://discuss.elastic.co/u/mihalikv)\ **Post date:** [January 21, 2018, 10:35pm UTC](https://discuss.elastic.co/t/failed-shards-with-top-hits/116407/1 "2018-01-21T22:35:05Z")
Hi, i'am using **ES 5.5.0** in one index with default 5 shards. When i run query with following aggregation: "attributes":{ "global":{ ``` }, "aggs":{ "attributes_products":{ "filter":{ "bool":{ "must":[ some filters ] } }, "aggs":{ "attributes":{ "nested":{ "path":"attributes" }, "aggs":{ "code":{ "terms":{ "field":"attributes.code", "size":40 }, "aggs":{ "translations":{ "nested":{ "path":"attributes.translated_fields" }, "aggs":{ "sk":{ "nested":{ "path":"attributes.translated_fields.sk" }, "aggs":{ "value":{ "terms":{ "field":"attributes.translated_fields.sk.value" "size":40 }, "aggs":{ "source":{ "aggs":{ "top_attributes":{ "top_hits":{ "size":1 } }, "products":{ "reverse_nested":{ }, "aggs":{ "cardinality":{ "cardinality":{ "field":"parent_id" } } } } }, "reverse_nested":{ "path":"attributes" } } } } } } } } } } } } } } } } ``` with some filters on index with about 5000 items in it. I end up with 3 failed shards and very inaccurate cardinality value. If i delete top\_hits aggregation all 5 shards are success. Is it problem of deep nested aggs, problem with just one index or problem with **top\_hits**? ---
**Author:** ![s1monw](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/s1monw/32/3637_2.png) [@s1monw](https://discuss.elastic.co/u/s1monw)\ **Post date:** [January 26, 2018, 12:55pm UTC](https://discuss.elastic.co/t/failed-shards-with-top-hits/116407/2 "2018-01-26T12:55:41Z")
can you show what causes the shard failure. ---
**Author:** ![mihalikv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mihalikv/32/26793_2.png) [@mihalikv](https://discuss.elastic.co/u/mihalikv)\ **Post date:** [January 26, 2018, 1:38pm UTC](https://discuss.elastic.co/t/failed-shards-with-top-hits/116407/3 "2018-01-26T13:38:28Z")
reason is null and type is null\_pointer\_exception ---
**Author:** ![s1monw](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/s1monw/32/3637_2.png) [@s1monw](https://discuss.elastic.co/u/s1monw)\ **Post date:** [January 26, 2018, 1:39pm UTC](https://discuss.elastic.co/t/failed-shards-with-top-hits/116407/4 "2018-01-26T13:39:33Z")
> [@mihalikv](#): > > reason is null and type is null\_pointer\_exception do you have the stacktrace for it? I need to know where it happens ---
**Author:** ![mihalikv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mihalikv/32/26793_2.png) [@mihalikv](https://discuss.elastic.co/u/mihalikv)\ **Post date:** [January 26, 2018, 2:42pm UTC](https://discuss.elastic.co/t/failed-shards-with-top-hits/116407/5 "2018-01-26T14:42:15Z")
I am trying to get some trace but `?error_trace=true` not working and i don't find another solution. Result is still ``` { "took": 42, "timed_out": false, "_shards": { "total": 5, "successful": 1, "failed": 4, "failures": [ { "shard": 0, "index": "shop", "node": "EWaVA2IoSyi0adHgxABZ1Q", "reason": { "type": "null_pointer_exception", "reason": null } } ] }, "hits": "aggregations" } ``` How can i get stacktrace ? I am running query in Kibana. ---
**Author:** ![mihalikv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mihalikv/32/26793_2.png) [@mihalikv](https://discuss.elastic.co/u/mihalikv)\ **Post date:** [January 26, 2018, 3:02pm UTC](https://discuss.elastic.co/t/failed-shards-with-top-hits/116407/6 "2018-01-26T15:02:38Z")
ok here it is ``` org.elasticsearch.transport.RemoteTransportException: [EWaVA2I][127.0.0.1:9300][indices:data/read/search[phase/query]] Caused by: java.lang.NullPointerException at org.elasticsearch.search.aggregations.bucket.BestBucketsDeferringCollector.prepareSelectedBuckets(BestBucketsDeferringCollector.java:160) ~[elasticsearch-5.5.0.jar:5.5.0] at org.elasticsearch.search.aggregations.bucket.DeferringBucketCollector.replay(DeferringBucketCollector.java:44) ~[elasticsearch-5.5.0.jar:5.5.0] at org.elasticsearch.search.aggregations.AggregatorBase.runDeferredCollections(AggregatorBase.java:206) ~[elasticsearch-5.5.0.jar:5.5.0] at org.elasticsearch.search.aggregations.bucket.terms.LongTermsAggregator.buildAggregation(LongTermsAggregator.java:156) ~[elasticsearch-5.5.0.jar:5.5.0] at org.elasticsearch.search.aggregations.AggregatorFactory$MultiBucketAggregatorWrapper.buildAggregation(AggregatorFactory.java:147) ~[elasticsearch-5.5.0.jar:5.5.0] at org.elasticsearch.search.aggregations.bucket.BucketsAggregator.bucketAggregations(BucketsAggregator.java:116) ~[elasticsearch-5.5.0.jar:5.5.0] at org.elasticsearch.search.aggregations.bucket.nested.NestedAggregator.buildAggregation(NestedAggregator.java:97) ~[elasticsearch-5.5.0.jar:5.5.0] at org.elasticsearch.search.aggregations.AggregatorFactory$MultiBucketAggregatorWrapper.buildAggregation(AggregatorFactory.java:147) ~[elasticsearch-5.5.0.jar:5.5.0] at org.elasticsearch.search.aggregations.bucket.BucketsAggregator.bucketAggregations(BucketsAggregator.java:116) ~[elasticsearch-5.5.0.jar:5.5.0] at org.elasticsearch.search.aggregations.bucket.filter.FilterAggregator.buildAggregation(FilterAggregator.java:72) ~[elasticsearch-5.5.0.jar:5.5.0] at org.elasticsearch.search.aggregations.bucket.BucketsAggregator.bucketAggregations(BucketsAggregator.java:116) ~[elasticsearch-5.5.0.jar:5.5.0] at org.elasticsearch.search.aggregations.bucket.global.GlobalAggregator.buildAggregation(GlobalAggregator.java:59) ~[elasticsearch-5.5.0.jar:5.5.0] at org.elasticsearch.search.aggregations.AggregationPhase.execute(AggregationPhase.java:129) ~[elasticsearch-5.5.0.jar:5.5.0] at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:114) ~[elasticsearch-5.5.0.jar:5.5.0] at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:248) ~[elasticsearch-5.5.0.jar:5.5.0] at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:263) ~[elasticsearch-5.5.0.jar:5.5.0] at org.elasticsearch.action.search.SearchTransportService$6.messageReceived(SearchTransportService.java:330) ~[elasticsearch-5.5.0.jar:5.5.0] at org.elasticsearch.action.search.SearchTransportService$6.messageReceived(SearchTransportService.java:327) ~[elasticsearch-5.5.0.jar:5.5.0] at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.5.0.jar:5.5.0] at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:644) [elasticsearch-5.5.0.jar:5.5.0] at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:638) [elasticsearch-5.5.0.jar:5.5.0] at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.5.0.jar:5.5.0] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_151] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_151] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151] ``` ---
**Author:** ![s1monw](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/s1monw/32/3637_2.png) [@s1monw](https://discuss.elastic.co/u/s1monw)\ **Post date:** [January 26, 2018, 3:34pm UTC](https://discuss.elastic.co/t/failed-shards-with-top-hits/116407/7 "2018-01-26T15:34:45Z")
thanks for the stacktrace, that might be indeed a bug right [here](https://github.com/elastic/elasticsearch/blob/5.5/core/src/main/java/org/elasticsearch/search/aggregations/bucket/BestBucketsDeferringCollector.java#L160) I will look into it and report back ---
**Author:** ![colings86](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/colings86/32/44960_2.png) [@colings86](https://discuss.elastic.co/u/colings86)\ **Post date:** [January 26, 2018, 3:53pm UTC](https://discuss.elastic.co/t/failed-shards-with-top-hits/116407/8 "2018-01-26T15:53:36Z")
@mihalikv Thanks for reporting this. I have raised [https://github.com/elastic/elasticsearch/issues/28394](https://github.com/elastic/elasticsearch/issues/28394) in the Elasticsearch repo as this seems like its a bug and will look into it further. In the meantime could you provide the mappings for this index (maybe in a [Gist](http://gist.github.com) if its very long) as it may help us reproduce the failure? ---
**Author:** ![mihalikv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mihalikv/32/26793_2.png) [@mihalikv](https://discuss.elastic.co/u/mihalikv)\ **Post date:** [January 26, 2018, 8:33pm UTC](https://discuss.elastic.co/t/failed-shards-with-top-hits/116407/9 "2018-01-26T20:33:23Z")
Thanks for help. I am away of computer, i will provide mapping tommorow. ---
**Author:** ![jimczi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jimczi/32/47985_2.png) [@jimczi](https://discuss.elastic.co/u/jimczi)\ **Post date:** [January 26, 2018, 10:35pm UTC](https://discuss.elastic.co/t/failed-shards-with-top-hits/116407/10 "2018-01-26T22:35:23Z")
Thanks for reporting @mihalikv. The workaround for now is to change the sort order of your `top_hits` aggregation. Using: ```auto "top_attributes":{ "top_hits":{ "size":1, "sort": ["_doc"] } ``` should work fine and return the same result. The reason is that we don't handle retrieving the `_score` on a `global` aggregation that uses `nested` contexts and the `top_hits` aggregation uses the `_score` as the default sort criteria. ---
**Author:** ![mihalikv](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mihalikv/32/26793_2.png) [@mihalikv](https://discuss.elastic.co/u/mihalikv)\ **Post date:** [January 27, 2018, 11:47am UTC](https://discuss.elastic.co/t/failed-shards-with-top-hits/116407/11 "2018-01-27T11:47:44Z")
Thanks for solution. I don't if provide mapping here or in issue but here is mapping: > ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 24, 2018, 11:48am UTC](https://discuss.elastic.co/t/failed-shards-with-top-hits/116407/12 "2018-02-24T11:48:31Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.