# TSVB dashboard - Performance issue with huge data **URL:** \ **Category:** Kibana\ **Tags:** ilm-index-lifecycle-management\ **Created:** [April 23, 2024, 12:33am UTC](https://discuss.elastic.co/t/tsvb-dashboard-performance-issue-with-huge-data/357949 "2024-04-23T00:33:21Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![david89](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/david89/32/133515_2.png) [@david89](https://discuss.elastic.co/u/david89)\ **Post date:** [April 23, 2024, 12:33am UTC](https://discuss.elastic.co/t/tsvb-dashboard-performance-issue-with-huge-data/357949/1 "2024-04-23T00:33:21Z")
I have a performance issue with TSVB dashboard with 70 millions records and more ([TSVB | Kibana Guide [8.13] | Elastic](https://www.elastic.co/guide/en/kibana/current/tsvb.html)) I have applied rollover index in ILM, so caching can be used for normal query (request\_cache = true), but for TSVB, I have not and found the way to set it Do you know: - the way to set cache for TSVB - or the way to optimize the performance here is one chart in TSVB ![Screenshot 2024-04-23 at 07.32.39](https://us1.discourse-cdn.com/elastic/original/3X/1/7/177a6eb4213b217a92eaa6033f34ceab4d3c212b.png) ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [April 23, 2024, 3:08am UTC](https://discuss.elastic.co/t/tsvb-dashboard-performance-issue-with-huge-data/357949/2 "2024-04-23T03:08:02Z")
Hi @david89 Welcome to the community. Exactly what calculation / aggregation are you trying to calculate? You can use filters to reduce dataset etc to only use documents that include the field you want to aggregate on. Are you trying to do a cardinality on large cardinality data set? [Here](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-cardinality-aggregation.html) Did you try lens? ---
**Author:** ![david89](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/david89/32/133515_2.png) [@david89](https://discuss.elastic.co/u/david89)\ **Post date:** [April 23, 2024, 9:18am UTC](https://discuss.elastic.co/t/tsvb-dashboard-performance-issue-with-huge-data/357949/3 "2024-04-23T09:18:16Z")
Dear @stephenb , thank you for your attention we are doing "sum" on some field We must scan all records, so no filter can be applied Do you have any suggest about caching or any approach? "lens" =\> I will try it ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [April 23, 2024, 1:53pm UTC](https://discuss.elastic.co/t/tsvb-dashboard-performance-issue-with-huge-data/357949/4 "2024-04-23T13:53:24Z")
@david89 What is the performance you are seeing? What is the total size of the index? What are the specs and JVM heap for the cluster? You could also pre aggregate the Data... Also look at > **[Tune for search speed | Elasticsearch Guide \[8.13\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-search-speed.html)** > **[Tune for search speed | Elasticsearch Guide \[8.13\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/tune-for-search-speed.html)** ---
**Author:** ![david89](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/david89/32/133515_2.png) [@david89](https://discuss.elastic.co/u/david89)\ **Post date:** [April 24, 2024, 4:04am UTC](https://discuss.elastic.co/t/tsvb-dashboard-performance-issue-with-huge-data/357949/5 "2024-04-24T04:04:29Z")
@stephenb , good morning, here are the detail information > What is the performance you are seeing? =\> My index has more than 400 fields, in one dashboard I need to load 10 TSVB charts which aggregate information from some fields The time for loading 1 TSVB chart is short (~1s) but for 10 charts at time: error (more than 10s) > What is the total size of the index? > total size: 74 millions records ~ 70GB (cluster has 3 primary shard + 1 replica for each primary) > What are the specs and JVM heap for the cluster? > CPU: 14, RAM: 16G, SSD: 120G > =\> this specs is good for total size = 40G (time for loading 10 charts is smaller than 10s) > but for more than 50G, there are errors > I would like to apply caching before thinking upgrade the specs (more CPU, RAM) because the cost is high > You could also pre aggregate the Data... > Also look at > =\> thank you, I will take a look ---
**Author:** ![david89](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/david89/32/133515_2.png) [@david89](https://discuss.elastic.co/u/david89)\ **Post date:** [May 2, 2024, 2:36am UTC](https://discuss.elastic.co/t/tsvb-dashboard-performance-issue-with-huge-data/357949/6 "2024-05-02T02:36:42Z")
@stephenb do you have any suggest for us we load one chart fast, but for many charts (\> 10) is very slow and fail ![Screenshot 2024-05-02 at 09.35.07](https://us1.discourse-cdn.com/elastic/original/3X/e/7/e7d1ab77214fa245f67a59b33865998309b5d9b7.png) ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [May 2, 2024, 3:59am UTC](https://discuss.elastic.co/t/tsvb-dashboard-performance-issue-with-huge-data/357949/7 "2024-05-02T03:59:04Z")
What version are you on? How many nodes? You are probably just overwhelming your cluster... May be no simple answer ---
**Author:** ![david89](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/david89/32/133515_2.png) [@david89](https://discuss.elastic.co/u/david89)\ **Post date:** [May 2, 2024, 4:04am UTC](https://discuss.elastic.co/t/tsvb-dashboard-performance-issue-with-huge-data/357949/8 "2024-05-02T04:04:01Z")
@stephenb here is the specification: 1 cluster: 3 nodes (all are data node, 1 master), 3 primary shard + 1 one replica (per shard) ---
**Author:** ![david89](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/david89/32/133515_2.png) [@david89](https://discuss.elastic.co/u/david89)\ **Post date:** [May 2, 2024, 7:20am UTC](https://discuss.elastic.co/t/tsvb-dashboard-performance-issue-with-huge-data/357949/9 "2024-05-02T07:20:48Z")
And ES version: 8.12.0 ---
**Author:** ![Marco\_Liberati](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_liberati/32/82953_2.png) [@Marco\_Liberati](https://discuss.elastic.co/u/Marco_Liberati)\ **Post date:** [May 2, 2024, 8:14am UTC](https://discuss.elastic.co/t/tsvb-dashboard-performance-issue-with-huge-data/357949/10 "2024-05-02T08:14:17Z")
What error is TSVB reporting? ---
**Author:** ![david89](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/david89/32/133515_2.png) [@david89](https://discuss.elastic.co/u/david89)\ **Post date:** [May 2, 2024, 8:25am UTC](https://discuss.elastic.co/t/tsvb-dashboard-performance-issue-with-huge-data/357949/11 "2024-05-02T08:25:18Z")
Here is error when loading huge data (time \> 3 weeks ago, if we choose \< 2 week, there is not any error): # Error: Server Error ## The server encountered a temporary error and could not complete your request. Please try again in 30 seconds. ![Screenshot 2024-05-02 at 15.23.50](https://us1.discourse-cdn.com/elastic/original/3X/5/0/500374ce073271e047b86a008b54700aa47f2958.png) ---
**Author:** ![Marco\_Liberati](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_liberati/32/82953_2.png) [@Marco\_Liberati](https://discuss.elastic.co/u/Marco_Liberati)\ **Post date:** [May 2, 2024, 8:40am UTC](https://discuss.elastic.co/t/tsvb-dashboard-performance-issue-with-huge-data/357949/12 "2024-05-02T08:40:40Z")
Interesting. How the stack deployed? Is it on premise or a cloud deployment? ---
**Author:** ![david89](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/david89/32/133515_2.png) [@david89](https://discuss.elastic.co/u/david89)\ **Post date:** [May 2, 2024, 8:55am UTC](https://discuss.elastic.co/t/tsvb-dashboard-performance-issue-with-huge-data/357949/13 "2024-05-02T08:55:41Z")
It is ECK (Basic - free) on GKE (Google Kubernetes Engine) ---
**Author:** ![Marco\_Liberati](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/marco_liberati/32/82953_2.png) [@Marco\_Liberati](https://discuss.elastic.co/u/Marco_Liberati)\ **Post date:** [May 2, 2024, 9:16am UTC](https://discuss.elastic.co/t/tsvb-dashboard-performance-issue-with-huge-data/357949/14 "2024-05-02T09:16:06Z")
Probably you need to check at the kubernetes side of things: is it using Ingress or a Load balancer?