We have a elasticsearch setup in which we index documents and with the user
data pouring in, we update documents. Every document has an expiry of 30
days and with every update the _ttl is also updated. At any given point, we
have around 150 million documents.
This setup has been running successfully in production from last couple of
months but from last few days we are experiencing enormously slow document
fetch and update. Earlier it used to take 1-2 seconds to update 100
documents, while now on an average it takes around 50-70 secs. The rate at
which documents are indexed and update requests are generated is more or
less the same, so it cant be that this behaviour is because of more traffic
Any suggestions on what might possibly be going wrong?
ES Setup: 10 shards, 1 replica, 10 nodes, 24 GB RAM, 16GB Heap, spinning