Beats write_index causes single hotspots

[coudenysj (Jachim Coudenys)]

Hi,

I'm using beats to ship most of my data to elasticsearch. I'm using the suggested ILM in a hot-warm-cold setup, which works fine.

The biggest problem I see with the way beats utilizes the write_indices with aliases, is that (even if I have 6 hot nodes), all of the current ingest traffic is going to that single hot node.

I have filebeat, metricbeat and heartbeat, so I can spread the load over these three write indices, but if I want to include more data in the filebeats, I'm not able to spread the load.

Any thoughts about how I can tackle this problem? Because I sometimes see bulk rejections (see https://www.elastic.co/blog/why-am-i-seeing-bulk-rejections-in-my-elasticsearch-cluster).

[coudenysj (Jachim Coudenys)]

Ok, so after giving it some more thoughts, I just have to change the default number_of_shards from 1 to 3 (for example).

The "shrink" index option in the warm phase, will take care of the rest .