We (in our company) recently upgraded our ELK environment (using new servers) from 1.7.2 to 6.2.4.
Another change we made is that before we used the logstash-forwarder on the hosts but now we use filebeat to forward the hosts.
Next to this we also changed the cluster setup:
- Previous setup
- 1 server installed with Kibana, Elasticsearch & Logstash
- New setup
- 2 elasticsearch servers (node)
- 1 server with kibana & logstash
The amount of logs changed practically the same (a few more) but there is a lot more parsing (dividing our logs into more fields and mapping some fields to integer) but next to that only the version and the setup has changed.
But we do notice that the old ELK stack (one server) has an average of 175 write IOPS while the new ELK stack has the following amount of write IOPS:
- Master elasticsearch node: 542 write IOPS
- Data elasticsearch node: 523 write IOPS
Is there any known reason that this happens or is this a configuration mistake?
Any help would be appreciated