# Top3 in Line graph and Top values in Available fields are different **URL:** \ **Category:** Kibana\ **Created:** [July 22, 2022, 10:09am UTC](https://discuss.elastic.co/t/top3-in-line-graph-and-top-values-in-available-fields-are-different/310382 "2022-07-22T10:09:25Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\ **Post date:** [July 22, 2022, 10:09am UTC](https://discuss.elastic.co/t/top3-in-line-graph-and-top-values-in-available-fields-are-different/310382/1 "2022-07-22T10:09:25Z")
I'm working on visualizing the statistics of logs collected by Kibana. I would like to know how certain items in the logs have changed over time. For example, I would like to aggregate the IP addresses of remote hosts from the access logs and see how the accesses from the most frequent addresses have changed. I created a panel in Kibana's Dashboard and created a Line graph of the Top 3, but I am skeptical that they are really the Top 3. I'm not sure if they are really Top 3, because they are different from the Top 3 values that are displayed when I select an item listed in the Available fields. Why is this? Are these aggregation rules different? ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/0/30f18c4a5e241d21a8fd103c91b522d5fa064513.png) ---
**Author:** ![flash1293](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/flash1293/32/41227_2.png) [@flash1293](https://discuss.elastic.co/u/flash1293)\ **Post date:** [July 22, 2022, 10:26am UTC](https://discuss.elastic.co/t/top3-in-line-graph-and-top-values-in-available-fields-are-different/310382/2 "2022-07-22T10:26:13Z")
Top values are approximate - especially for high cardinality fields. In 8.3 we introduced an “accuracy mode” in Lens: [Kibana highlights | Elastic Installation and Upgrade Guide [8.3] | Elastic](https://www.elastic.co/guide/en/elastic-stack/current/kibana-higlights.html#_accuracy_mode_in_lens) It sets the shard\_size parameter on Elasticsearch side - it's heavier on the cluster resources but yields more accurate results. More information about this: [Terms aggregation | Elasticsearch Guide [8.3] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#_ordering_by_a_sub_aggregation) ---
**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\ **Post date:** [July 22, 2022, 10:30am UTC](https://discuss.elastic.co/t/top3-in-line-graph-and-top-values-in-available-fields-are-different/310382/3 "2022-07-22T10:30:17Z")
Thanks for answering. It doesn't have to be so exact, but I am puzzled by these different values. Does this mean that the methods of calculation are different from each other? Even if Lens' Top 3 is also an estimate, is it the Top 3 calculated over that time period? ---
**Author:** ![flash1293](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/flash1293/32/41227_2.png) [@flash1293](https://discuss.elastic.co/u/flash1293)\ **Post date:** [July 22, 2022, 12:45pm UTC](https://discuss.elastic.co/t/top3-in-line-graph-and-top-values-in-available-fields-are-different/310382/4 "2022-07-22T12:45:28Z")
It is calculated over the time period, but separately per node in the cluster (actually per shard of the queried index pattern). More thorough explanation: Elasticsearch splits up data across multiple indices and shards sitting on different nodes - every shard is doing it's own processing, then sending the result to the coordinating node (the one Kibana talks to) which merges the individual nodes results and sends the response to Kibana. However ordering a list of terms can't be distributed across multiple nodes. One option would be for each node to send the full list of local terms to the coordinating node which merges and orders all of these lists, then sends the top 3 to the client. However, if there are millions of terms in total, this would be super expensive as a lot of data would have to be transferred to the coordinating node (and it would also use up a lot of memory on the coordinating node). So Elasticsearch isn't doing this, instead it just sends the top 15 terms or so per shard to the coordinating node. This keeps the memory usage and network traffic low, but it means the list can be wrong. Consider the following example: node one has the following data: | term | count | | --- | --- | | A | 95 | | B | 94 | | C | 93 | | X | 1 | | Y | 2 | | Z | 3 | node two has the following data: | term | count | | --- | --- | | A | 9 | | B | 8 | | C | 7 | | X | 98 | | Y | 97 | | Z | 96 | Both nodes send their top 3 (A-B for node one and X-Z for node two) to the coordinating node which merges and sorts the partial lists and sends the top 3 of that list to the client: combined top 3 lists from both nodes | term | count | | --- | --- | | X | 98 | | Y | 97 | | Z | 96 | | A | 95 | | B | 94 | | C | 93 | User sees: | term | count | | --- | --- | | X | 98 | | Y | 97 | | Z | 96 | However, if the data nodes had sent their top 6 lists respectively, the outcome would have been very different: | term | count | | --- | --- | | A | 95 + 9 = 104 | | B | 94 + 8 = 102 | | C | 93 + 7 = 100 | | X | 1 + 98 = 99 | | Y | 2 + 97 = 99 | | Z | 3 + 96 = 99 | So the client would eventually see | term | count | | --- | --- | | A | 104 | | B | 102 | | C | 100 | This is what the "accuracy mode" is about - it increases the number of terms transferred from the data nodes to the coordinating node which is more expensive to calculate, but there is a smaller chance of providing the wrong top values ---
**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\ **Post date:** [July 22, 2022, 1:20pm UTC](https://discuss.elastic.co/t/top3-in-line-graph-and-top-values-in-available-fields-are-different/310382/5 "2022-07-22T13:20:28Z")
What a perfect answer! I'm so impressed! Ah. So, in the case of multi-node, the above is the case? Well. It's just an approximation. I also found out that we can choose "accuracy mode" for exact calculation. So we have a choice. Now, I have another question. I am using a single node cluster. In this case, the above case will not happen, right? ---
**Author:** ![flash1293](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/flash1293/32/41227_2.png) [@flash1293](https://discuss.elastic.co/u/flash1293)\ **Post date:** [July 22, 2022, 2:50pm UTC](https://discuss.elastic.co/t/top3-in-line-graph-and-top-values-in-available-fields-are-different/310382/6 "2022-07-22T14:50:46Z")
It will still happen if you have more than one index in your pattern or more than one shard in your index. I glanced over this in my example above to simplify, but the logic I described happens per shard even if they are on the same node ---
**Author:** ![its-ogawa](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/its-ogawa/32/120829_2.png) [@its-ogawa](https://discuss.elastic.co/u/its-ogawa)\ **Post date:** [July 25, 2022, 1:10am UTC](https://discuss.elastic.co/t/top3-in-line-graph-and-top-values-in-available-fields-are-different/310382/7 "2022-07-25T01:10:37Z")
Hmmm... So it occurs in every shard. But I collect logs on the same shard on a daily, and in many cases the rankings seem to be different even on a single shard. ![image](https://us1.discourse-cdn.com/elastic/original/3X/1/3/13140fd2190ab4ea5c60c2f645b4602e50854175.png) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 22, 2022, 1:11am UTC](https://discuss.elastic.co/t/top3-in-line-graph-and-top-values-in-available-fields-are-different/310382/8 "2022-08-22T01:11:14Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.