We noticed a serious performance degradation of date_histogram aggregations on our cluster after upgrading to 7.8.0.
We have two indexes with 1.2billion documents and use only date_histogram, but this query need 10S.
We use Profiler api to generate the following result.
We have 14 machines , per node has 128G memory and 5T SSD disk。
Did someone experienced the same behaviour?Is this something we can tune?
Hiya @Chan_Terence! We've been making some changes recently to aggregations, date_histo included. But the changes have all been performance optimizations for the most part... we've not seen any large regressions so far. But perhaps you've stumbled into an edge-case or something.
What was the performance like before upgrading? Is the upgrade the only thing that has changed recently?
Thanks for the profiler output! Would it be possible to also capture a HotThreads API output while the query is running? That'll give us a glimpse into what's going on internally
Make sure it runs without profiling enabled... profiling adds a fair amount of overhead and can skew hot thread outputs.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.