I have a cluster consisting of 1 hot (40 cores, 2 SSDs) and 2 cold nodes. I use a data stream to index a lot of data into 50gb one-shard indices with rollover, ILM, etc. Data is also sorted by @timestamp (asc) -- so I believe I've done everything to optimize Elasticsearch.
Yet when I try to create a visualization in Kibana I end up with requests taking a lot of time. Requesting data for 15 minutes takes a few seconds, for a day -- several minutes, for a week -- couldn't even count.
Could this be a problem with Kibana, perhaps? I don't know how else can I optimize Elastic.
May I ask in which context does the problem shows up: does it happen within a dashboard or within the Lens editor?
Also what version of the stack do you have?
Since version 7.17 the ability to hide suggestions has been introduced, which should improve performances within the Lens editor.
Because suggestions are always rendered quite often on changes made in the configuration, this can take up some time especially for Over Time suggestion. In recent releases some performance tuning have been introduced in order to discard the Over time suggestion in some heavy scenarios, or disable altogether the suggestions system:
In 7.14 the only option I think you may have is for metric columns with formula to use them and enable the fullscreen in formula: this is the only case when suggestions are disabled in older versions.
Do I understand correctly that this is an issue with Kibana's performance, and not Elasticsearch? Does Kibana send more requests because of suggestions or just takes more time to calculate something?
You can test this via opening the Inspector panel for a Lens panel in a dashboard, copy the request sent to ES and check in the DevTools the response time.
This was true in previous version of Kibana prior to 7.17
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
