My reindex operation has slowed considerably over time - what can explain this trend of decrease in throughput? Please see attached.
While reindexing is happening, the search latency has also gone up too. See attached
CPU utilization is fairly constant
- Total number of documents to index is about 300 million.
- Index is configured with 15 shards
- Total number of data nodes is 3
- Refresh interval is set to 10s
- Replica count is set to 2
Here is the Reindex request:
ReindexRequest request = new ReindexRequest(); request.setSourceIndices("sourceIndex"); request.setDestIndex("destIndex"); request.setDestVersionType(VersionType.EXTERNAL); request.setDestOpType("index"); request.setConflicts("proceed"); request.setScript(new Script(ScriptType.INLINE, "painless", "ctx._source.sortId = ctx._id", Collections.emptyMap())); request.setRefresh(true);