I am exploring the ELK framework for creating a centralized logging system across all our application(java,ruby,php) running on ec2 ubuntu servers . After going through some readings I have few doubts :
How Is log stash indexer scalable horizontally ? Can we simply configure it behind a load balancer and simple keep adding/removing boxes in variance with load ? Is it stateless ?
How to decide whether to use kafka queue or redis queue ?
3)Will I need to correct logging formats in all my applications for it to be useful or that can be done by logstash itself ? Can single logstash server handle logs with different formats from different applications ?
- How is filebeat in comparison to logstash forwarder ?