If you can do option 4, I'd go for it as there are much less things to maintain ![]()
Let me answer to:
It seems that logstash is becoming replaced by beats ? Any word about this ?
and
(where beats have the role of logstash)
About beats/logstash, that's not the same thing. You can compare logstash inputs vs beats and compare logstash filters vs elasticsearch node ingest feature though.
In short: if you can do all the parsing/processing of your data in elasticsearch with ingest node, just use beats and Elasticsearch.
If you have a more advanced pipeline or if you want to send the data collected by beats to elasticsearch AND to another output such as a storage or kafka or whatever, then use Logstash.
We want to use Kafka as a buffer.
So if this is mandatory, then go for option 2.