# How do we know if the filebeat processed log successfully

**URL:** <https://discuss.elastic.co/t/how-do-we-know-if-the-filebeat-processed-log-successfully/181454>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [May 16, 2019, 5:07pm UTC](https://discuss.elastic.co/t/how-do-we-know-if-the-filebeat-processed-log-successfully/181454 "2019-05-16T17:07:07Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![chandukreddi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chandukreddi/32/29241_2.png) [@chandukreddi](https://discuss.elastic.co/u/chandukreddi)\
**Post date:** [May 16, 2019, 5:07pm UTC](https://discuss.elastic.co/t/how-do-we-know-if-the-filebeat-processed-log-successfully/181454/1 "2019-05-16T17:07:08Z")

</div>

Hello Experts,

how do we know if filebeat successfully sent log entry to logstash?

**Here is my filebeat logs prints:**

":387}},"total":{"ticks":1270,"time":{"ms":1281},"value":1270},"user":{"ticks":890,"time":{"ms":894}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":10},"info":{"ephemeral\_id":"c345ac40-0c0b-4799-b11c-a8dadea8af99","uptime":{"ms":30043}},"memstats":{"gc\_next":30194336,"memory\_alloc":25457200,"memory\_total":47430720,"rss":41299968}},"filebeat":{"events":{"active":4199,"added":4206,"done":7},"harvester":{"open\_files":2,"running":2,"started":2}},"libbeat":{"config":{"module":{"running":0},"reloads":3},"output":{"events":{"active":4096,"batches":2,"total":4096},"read":{"bytes":30},"type":"logstash","write":{"bytes":393078}},"pipeline":{"clients":1,"events":{"active":4117,"filtered":88,"published":4116,"retry":2048,"total":4205}}},"registrar":{"states":{"current":2,"update":7},"writes":{"success":8,"total":8}},"system":{"cpu":{"cores":10},"load":{"1":0.17,"15":0.23,"5":0.29,"norm":{"1":0.017,"15":0.023,"5":0.029}}}}}}  
2019-05-16T09:54:18.393-0700 INFO [monitoring] log/log.go:144 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":390,"time":{"ms":3}},"total":{"ticks":1290,"time":{"ms":14},"value":1290},"user":{"ticks":900,"time":{"ms":11}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":10},"info":{"ephemeral\_id":"c345ac40-0c0b-4799-b11c-a8dadea8af99","uptime":{"ms":60039}},"memstats":{"gc\_next":30194336,"memory\_alloc":25890448,"memory\_total":47863968,"rss":241664}},"filebeat":{"harvester":{"open\_files":2,"running":2}},"libbeat":{"config":{"module":{"running":0},"reloads":2},"output":{"read":{"bytes":36}},"pipeline":{"clients":1,"events":{"active":4117}}},"registrar":{"states":{"current":2}},"system":{"load":{"1":0.1,"15":0.22,"5":0.26,"norm":{"1":0.01,"15":0.022,"5":0.026}}}}}}

2019-05-16T09:54:48.393-0700 INFO [monitoring] log/log.go:144 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":390,"time":{"ms":6}},"total":{"ticks":1300,"time":{"ms":13},"value":1300},"user":{"ticks":910,"time":{"ms":7}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":10},"info":{"ephemeral\_id":"c345ac40-0c0b-4799-b11c-a8dadea8af99","uptime":{"ms":90038}},"memstats":{"gc\_next":30194336,"memory\_alloc":26210912,"memory\_total":48184432,"rss":229376}},"filebeat":{"harvester":{"open\_files":2,"running":2}},"libbeat":{"config":{"module":{"running":0},"reloads":3},"output":{"read":{"bytes":36}},"pipeline":{"clients":1,"events":{"active":4117}}},"registrar":{"states":{"current":2}},"system":{"load":{"1":0.12,"15":0.22,"5":0.25,"norm":{"1":0.012,"15":0.022,"5":0.025}}}}}}

2019-05-16T09:59:41.488-0700 INFO [monitoring] log/log.go:152 Total non-zero metrics {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":440,"time":{"ms":444}},"total":{"ticks":1520,"time":{"ms":1529},"value":1520},"user":{"ticks":1080,"time":{"ms":1085}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":8},"info":{"ephemeral\_id":"c345ac40-0c0b-4799-b11c-a8dadea8af99","uptime":{"ms":383136}},"memstats":{"gc\_next":35592592,"memory\_alloc":17850816,"memory\_total":51343096,"rss":42246144}},"filebeat":{"events":{"active":4197,"added":4206,"done":9},"harvester":{"closed":2,"open\_files":0,"running":0,"started":2}},"libbeat":{"config":{"module":{"running":0},"reloads":38},"output":{"events":{"active":4096,"batches":2,"total":4096},"read":{"bytes":456},"type":"logstash","write":{"bytes":393078}},"pipeline":{"clients":0,"events":{"active":4116,"failed":2,"filtered":88,"published":4116,"retry":2048,"total":4206}}},"registrar":{"states":{"current":2,"update":7},"writes":{"success":9,"total":9}},"system":{"cpu":{"cores":10},"load":{"1":0,"15":0.17,"5":0.11,"norm":{"1":0,"15":0.017,"5":0.011}}}}}}

Thanks  
Chandra

---

<div class="post-metadata">

**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\
**Post date:** [May 23, 2019, 1:03am UTC](https://discuss.elastic.co/t/how-do-we-know-if-the-filebeat-processed-log-successfully/181454/2 "2019-05-23T01:03:44Z")

</div>

> [@chandukreddi](#):
>
> how do we know if filebeat successfully sent log entry to logstash?

Looking at the provided log I would look at the "published" stats, that confirm that events is successfully send to LS. But a more convenient way would be to use [monitoring](https://www.elastic.co/guide/en/beats/filebeat/current/monitoring.html).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 20, 2019, 1:03am UTC](https://discuss.elastic.co/t/how-do-we-know-if-the-filebeat-processed-log-successfully/181454/3 "2019-06-20T01:03:47Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
