Slow aggregation queries while ingesting data

Elastic Stack Elasticsearch
1

I have a large aggregation query that becomes incredibly slow when I am updating my data. I am not saving the data to a tmp index (and then renaming it when it's done) but saving it directly to the index I'm querying.

  • What are some ways to improve querying performance while indexing is occurring?
  • What are the usual bottlenecks that I'm seeing here (possibly memory?)?
2

Try to identify what is limiting performance. Check CPU usage as well as disk I/O (including iowait).

What is the size of your cluster? What type of hardware? How much data and shards do you have?