# Filebeat log monitoring/alerting

**URL:** https://discuss.elastic.co/t/filebeat-log-monitoring-alerting/53350
**Category:** Beats
**Tags:** filebeat
**Created:** [June 20, 2016, 5:39pm UTC](https://discuss.elastic.co/t/filebeat-log-monitoring-alerting/53350 "2016-06-20T17:39:14Z")
**Posts on this page:** 14
**Page:** 1

<div class="post-metadata">

### Author: ![maheshraju](https://avatars.discourse-cdn.com/v4/letter/m/977dab/32.png) [@maheshraju](https://discuss.elastic.co/u/maheshraju)
#### Post date: [June 20, 2016, 5:39pm UTC](https://discuss.elastic.co/t/filebeat-log-monitoring-alerting/53350/1 "2016-06-20T17:39:14Z")

</div>

I'm looking for a way to monitor Filebeat (running on windows)  
log and generate alerts in case of errors. Is there a way Filebeat can send  
events/log to windows event log?

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [June 21, 2016, 11:17am UTC](https://discuss.elastic.co/t/filebeat-log-monitoring-alerting/53350/2 "2016-06-21T11:17:17Z")

</div>

Do you want that filebeat is monitoring your logs or to monitor the filebeat log itself? In the first case, you should have a look at Watcher: [https://www.elastic.co/products/watcher](https://www.elastic.co/products/watcher)

Filebeat itself doesn't do any log processing and can't send events to the windows event log.

---

<div class="post-metadata">

### Author: ![maheshraju](https://avatars.discourse-cdn.com/v4/letter/m/977dab/32.png) [@maheshraju](https://discuss.elastic.co/u/maheshraju)
#### Post date: [June 21, 2016, 5:21pm UTC](https://discuss.elastic.co/t/filebeat-log-monitoring-alerting/53350/3 "2016-06-21T17:21:18Z")

</div>

Thank you for the detailed response Ruflin. I'm interested to see if Filebeats or a plugin in the Beats family can monitor Filebeats own log.

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [June 22, 2016, 6:21am UTC](https://discuss.elastic.co/t/filebeat-log-monitoring-alerting/53350/4 "2016-06-22T06:21:57Z")

</div>

One option here would be having a second filebeat instance for the filebeat logs.

An issue you could be interested in: [https://github.com/elastic/beats/issues/463](https://github.com/elastic/beats/issues/463) We plan to add some monitoring to beats.

---

<div class="post-metadata">

### Author: ![maheshraju](https://avatars.discourse-cdn.com/v4/letter/m/977dab/32.png) [@maheshraju](https://discuss.elastic.co/u/maheshraju)
#### Post date: [June 23, 2016, 1:01am UTC](https://discuss.elastic.co/t/filebeat-log-monitoring-alerting/53350/5 "2016-06-23T01:01:30Z")

</div>

If an issue is encountered at the second Filebeat also then it would be like a loop of issues. I will try to look for some options like NXLog that can pull the issues from Filebeat logs and publish them to windows event logs.

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [June 24, 2016, 7:13am UTC](https://discuss.elastic.co/t/filebeat-log-monitoring-alerting/53350/6 "2016-06-24T07:13:01Z")

</div>

Don't your have then the same problem? Who monitors NYLog? Sounds like the general problem of who monitors the monitoring system.

---

<div class="post-metadata">

### Author: ![maheshraju](https://avatars.discourse-cdn.com/v4/letter/m/977dab/32.png) [@maheshraju](https://discuss.elastic.co/u/maheshraju)
#### Post date: [June 27, 2016, 9:57pm UTC](https://discuss.elastic.co/t/filebeat-log-monitoring-alerting/53350/7 "2016-06-27T21:57:44Z")

</div>

Yes that is true, we still have the general problem of who monitors the monitor either with NX Log or Filebeat.

While I was playing with Filebeat I got to know that Filebeat can ship its own log along with other application logs to Logstash/Elasticsearch. This can be a great option for alerting; Cons - yet to be evaluated.

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [June 28, 2016, 11:05am UTC](https://discuss.elastic.co/t/filebeat-log-monitoring-alerting/53350/8 "2016-06-28T11:05:10Z")

</div>

Yes, that is possible but I don't think I would recommend it. The reason is that it could become and endless loop, means if a log line written causes a new log line to be sent etc. If you only have ERR logging enabled, this should normally not be the case, but it could happen (for example connection down). That is why I would recommend a second filebeat instances.

---

<div class="post-metadata">

### Author: ![maheshraju](https://avatars.discourse-cdn.com/v4/letter/m/977dab/32.png) [@maheshraju](https://discuss.elastic.co/u/maheshraju)
#### Post date: [June 29, 2016, 6:05pm UTC](https://discuss.elastic.co/t/filebeat-log-monitoring-alerting/53350/9 "2016-06-29T18:05:03Z")

</div>

Yes Ruflin, I'm planning to ship only the ERR Logs.

As far as I understand, there will be endless loop even with the second instance of Filebeat, for the example of connection down scenario unless the second instance of Filebeat is shipping logs to a different destination.

For Example, the first Filebeat instance will ship logs to Elasticsearch.  
If the first Filebeat cannot reach to Elasticsearch, then the second Filebeat will also fail to reach Elasticsearch and there will be a loop.

If I have one Filebeat instance shipping application logs and its own logs then I think I will have one less thing to worry about. Feel free to correct me if I'm wrong.

If using a single Filebeat instance (to ship app logs and its own logs) can lead to significant performance issues or some other conflicts (while reading live logs) leading to some loss of logs, then I think its not a good candidate for my use-case.

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [July 4, 2016, 7:11am UTC](https://discuss.elastic.co/t/filebeat-log-monitoring-alerting/53350/10 "2016-07-04T07:11:00Z")

</div>

The part for me that is not fully clear yet is why you want to monitor Filebeat? What is the part you are worried about? Elasticsearch not reachable? Filebeat loosing log lines? Filebeat follows the at least once principle, so you should not loose any log lines also in case elasticsearch is down.

---

<div class="post-metadata">

### Author: ![maheshraju](https://avatars.discourse-cdn.com/v4/letter/m/977dab/32.png) [@maheshraju](https://discuss.elastic.co/u/maheshraju)
#### Post date: [July 6, 2016, 11:21pm UTC](https://discuss.elastic.co/t/filebeat-log-monitoring-alerting/53350/11 "2016-07-06T23:21:58Z")

</div>

I'm worried about the cases in which Filebeat may not be able to communicate with Logstash/Elasticsearch due to any reason (network issues/ corrupt certs/ any other internal server errors).

Filebeat may get the logs delivered to Elasticsearch eventually, But I'm trying to see if I can generate alerts and be aware that Filebeat is not able to reach Elasticsearch at the moment.

---

<div class="post-metadata">

### Author: ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)
#### Post date: [July 11, 2016, 8:47am UTC](https://discuss.elastic.co/t/filebeat-log-monitoring-alerting/53350/12 "2016-07-11T08:47:07Z")

</div>

Perhaps in this case it would be more helpful to have this as part of your server monitoring system?

---

<div class="post-metadata">

### Author: ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)
#### Post date: [July 11, 2016, 9:11pm UTC](https://discuss.elastic.co/t/filebeat-log-monitoring-alerting/53350/14 "2016-07-11T21:11:11Z")

</div>



---

<div class="post-metadata">

### Author: ![andrewkroh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/andrewkroh/32/3784_2.png) [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)
#### Post date: [July 13, 2016, 9:11pm UTC](https://discuss.elastic.co/t/filebeat-log-monitoring-alerting/53350/15 "2016-07-13T21:11:11Z")

</div>

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.
