What happens when using Logstash and Filebeat in production and any of them fails?

(Svetoslav Anastasov) #1

Hello, I am currently testing the ELK. My system contains of a few machines producing logs with Filebeat on them. One machine with centralized Logstash and from there pushing logs into AWS ES. When I push this into production and sometimes for some reason my Logstash or any of my Filebeat machines crash, or just the service crashes. What happens after that? How do I get notified that something has crashed? When resumed from where does Filebeat pick up from, can it know where it left of, as there will be a lot more logs in the folder by then? What happens when Logstash crashes?

(Jymit Singh Khondhu) #2

Are you using watcher for notifications?
Have you thought about implementing an ingest pipeline with Kafka. Something interesting to share: https://www.elastic.co/blog/integrating-elastic-stack-with-arcsight-siem-part-3

(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.