Kibana query performance

[paigekim]

Hello,

I am currently drawing graphs and charts from Kibana, but it takes longer than the time that I started to draw. Compared to the beginning, there are obviously more data, but I am not sure it is still count as slow, and I need to find a way to make the performance better or not.

For example, I have 'test' index pattern and it has 28 fields and 1 scripted field. From that index pattern, I have total of 17 charts, pies, and map. Among those, one of pie charts hits of 12,374,715 hits and takes qurey time as 11835ms.

Is there any other information I should provide for you to determine whether the performance of my kibana is slow or not? Please let me know.

[markov00 (Marco Vettorello)]

Hi, what kibana/es version are you using?
Are you building the pie visualization through Lens or the Visualize (aggregated) pie chart?
Can you share the configuration of the pie (a screenshot or a set of them is also fine)?

[paigekim]

Thank you for your prompt reply!

Current Kibana version is v7.12.1.

I am using both lens and Visualize pie chart, but the example I mentioned on the question was Visualize pie chart. Please see the attached for your reference.

[markov00 (Marco Vettorello)]

The pie chart configuration looks simple enough and doesn't looks like too heavy.
17 visualization on a dashboard looks a bit too much for a dashboard, do you see a slow performance also with a dashboard with less visualizations (4/8 max) ? Does the pie chart decrease the speed or is a general poor performance?

[paigekim]

If it is less visualization, it speeds up and also if there is less data to draw, it shortens time as well.

I made an example of pie chart because i can track the query time through options easily. On the network tab, I do see lots of bssearch api request but I cannot really tell which one slows down.

If the problem is the amount of data, is there any way I can improve the performance by not decreasing numbers of visualizations? All of those are needed to be shown in one dashboard if possible.

Lastly, I see 12,374,715 total hits from the discover tab for the past 10 years range. If Kibana draws Visualize with this, and there are 17 visualizations, do you think approximate 12 seconds to render is proper time I should wait for? Or should I look for a way to improve the time? If I do, would you give me some suggestions? And the data is currently added more every day and I am worried if it will slow more..

[markov00 (Marco Vettorello)]

Does the problem persist when you are in the visualize editor? does the visualization takes a lot to render?

[paigekim]

It happens both.

I do have 6 other dashboards and only experience on several dashboards that have lots of data in total on certain index patterns. So I was thinking about the problem can be the amounts of data.

[paigekim]

I actually found out that one of pie charts that I drew by Lens has a loading sign and render at the end even though other charts are already drawn. I attached the screenshot of loading(just hide the title of the chart) and its configuration.

I know you asked me earlier whether I drew by Lens or Visualize pie chart. Does Visualize pie chart has better performance than Lens?

[markov00 (Marco Vettorello)]

The pie chart configuration looks actually simple enough to be computed without any specific performance impact.
Can you please share the content of GET _cluster/health to understand how your cluster is configured? Can you also give me some details on the current cluster hardware specification?

[paigekim]

The following is the content of GET _cluster/health

{
  "status": "green",
  "number_of_nodes": 3,
  "unassigned_shards": 0,
  "number_of_pending_tasks": 0,
  "number_of_in_flight_fetch": 0,
  "timed_out": false,
  "active_primary_shards": 201,
  "task_max_waiting_in_queue_millis": 0,
  "cluster_name": "...",
  "relocating_shards": 0,
  "active_shards_percent_as_number": 100,
  "active_shards": 402,
  "initializing_shards": 0,
  "number_of_data_nodes": 2,
  "delayed_unassigned_shards": 0
}

And the total number of shards is 402.

[markov00 (Marco Vettorello)]

On the piechart configuration, you are using a timestamp_year field to split the chart into small multiples. Is that the scripted field? if so this could be the bottleneck, as the scripted field needs to be executed on every document and the result is not indexed.
You can gain more speed if that field is indexed as a numeric field instead of a scripted one. Before reindexing your data, you can try if this speeds up the chart using a different indexed field instead of the scripted one.

[paigekim]

Actually timestamp_year is not the scripted field. It is a keyword that I created manually.

Do you think is there any issue on the cluster health or cluster hardware specification?

[markov00 (Marco Vettorello)]

The number of shards looks a bit high for 3 nodes, how many indices do you have?
We usually link to those resources to understand exactly how many shards are needed

Size your shards | Elasticsearch Guide [7.13] | Elastic

How many shards should I have in my Elasticsearch cluster?

btw, you are not the first user that describes such kind of bad performance in the dashboard environment and the Kibana team is currently triaging another issue, similar to yours
I will keep you posted

[paigekim]

Current indices are 200, but it is adding up daily.

I will wait for the update!