I'm looking for a way to monitor Filebeat (running on windows)
log and generate alerts in case of errors. Is there a way Filebeat can send
events/log to windows event log?
Do you want that filebeat is monitoring your logs or to monitor the filebeat log itself? In the first case, you should have a look at Watcher: https://www.elastic.co/products/watcher
Filebeat itself doesn't do any log processing and can't send events to the windows event log.
1 Like
Thank you for the detailed response Ruflin. I'm interested to see if Filebeats or a plugin in the Beats family can monitor Filebeats own log.
One option here would be having a second filebeat instance for the filebeat logs.
An issue you could be interested in: https://github.com/elastic/beats/issues/463 We plan to add some monitoring to beats.
1 Like
If an issue is encountered at the second Filebeat also then it would be like a loop of issues. I will try to look for some options like NXLog that can pull the issues from Filebeat logs and publish them to windows event logs.
Don't your have then the same problem? Who monitors NYLog? Sounds like the general problem of who monitors the monitoring system.
1 Like
Yes that is true, we still have the general problem of who monitors the monitor either with NX Log or Filebeat.
While I was playing with Filebeat I got to know that Filebeat can ship its own log along with other application logs to Logstash/Elasticsearch. This can be a great option for alerting; Cons - yet to be evaluated.
Yes, that is possible but I don't think I would recommend it. The reason is that it could become and endless loop, means if a log line written causes a new log line to be sent etc. If you only have ERR logging enabled, this should normally not be the case, but it could happen (for example connection down). That is why I would recommend a second filebeat instances.
Yes Ruflin, I'm planning to ship only the ERR Logs.
As far as I understand, there will be endless loop even with the second instance of Filebeat, for the example of connection down scenario unless the second instance of Filebeat is shipping logs to a different destination.
For Example, the first Filebeat instance will ship logs to Elasticsearch.
If the first Filebeat cannot reach to Elasticsearch, then the second Filebeat will also fail to reach Elasticsearch and there will be a loop.
If I have one Filebeat instance shipping application logs and its own logs then I think I will have one less thing to worry about. Feel free to correct me if I'm wrong.
If using a single Filebeat instance (to ship app logs and its own logs) can lead to significant performance issues or some other conflicts (while reading live logs) leading to some loss of logs, then I think its not a good candidate for my use-case.
The part for me that is not fully clear yet is why you want to monitor Filebeat? What is the part you are worried about? Elasticsearch not reachable? Filebeat loosing log lines? Filebeat follows the at least once principle, so you should not loose any log lines also in case elasticsearch is down.
I'm worried about the cases in which Filebeat may not be able to communicate with Logstash/Elasticsearch due to any reason (network issues/ corrupt certs/ any other internal server errors).
Filebeat may get the logs delivered to Elasticsearch eventually, But I'm trying to see if I can generate alerts and be aware that Filebeat is not able to reach Elasticsearch at the moment.
Perhaps in this case it would be more helpful to have this as part of your server monitoring system?
This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.