We are building an ES index to archive billions of docs. The index are
distributed across 4 servers, each with 4 cpus, each with 32G memory. And
we splitted the index into 32 shards.
When we just began to index two weeks ago, the performance was like 6K
docs each second. Now, as the index gets big, with 3Billion docs indexed
already, total disk space is about 1.2T. The indexing speed decreases to
Initially we set refresh_interval to 120s, but in the past two days, the
ES server randomly dropped one shard and the cluster health status became
red. We had to decrease the refresh_interval lower to make the cluster
stable. When the cluster dropped one shard, the log has no any error info.
- Is there anyway to increase the performance ?
- What's the cause of the random drop of shard?
We are using ES 0.90.3.
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to firstname.lastname@example.org.
For more options, visit https://groups.google.com/groups/opt_out.