Hi,
I recently just deployed ElasticSearch 5 on production (4 months ago). Our average document per day is around 5M. I index my document per day based on some datetime format. Initially it went just fine. But lately I noticed the Insert threshold getting slower. Please help, I'm already out of ideas. Below are some info related to my environment.
OS - Windows
Current Storage Used - 300GB per node
_cluster/health
"cluster_name" : "sds-escluster",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 2,
"number_of_data_nodes" : 2,
"active_primary_shards" : 750,
"active_shards" : 1500,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 100.0
_nodes/stats
http://pastebin.com/M7fCwFrr