# Content Field - Number of \*\*NEW\*\* Log Lines Harvested

**URL:** <https://discuss.elastic.co/t/content-field-number-of-new-log-lines-harvested/138656>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [July 5, 2018, 8:08am UTC](https://discuss.elastic.co/t/content-field-number-of-new-log-lines-harvested/138656 "2018-07-05T08:08:02Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![olnrao](https://avatars.discourse-cdn.com/v4/letter/o/3bc359/32.png) [@olnrao](https://discuss.elastic.co/u/olnrao)\
**Post date:** [July 5, 2018, 8:08am UTC](https://discuss.elastic.co/t/content-field-number-of-new-log-lines-harvested/138656/1 "2018-07-05T08:08:02Z")

</div>

Can we have filebeat share the no. of new log lines

- It considered for processing
- It matched/filtered

These numbers can then be used in multiple ways to assess the health of an application.

Here are very simple use cases:  
Case-1) Application is running fine, but log lines are not emitted at all due to unknown issue  
Case-2) Application got hung, so init.d status check won't catch as init.d sees the process is still running  
Case-3) Application is running at reduced speed than normal - may be due to a service/server the app depends on has become slow

Solution-Step1) If no. of new log lines that harvester found (or considered) for every harvester run is shared out, we could then use that count to be sent to Graphite

Solution-Step2) Alert on this metric being zero for first two cases. Alert on this metric with a threshold for case-3

I have read Filebeat documentation available @ [https://www.elastic.co/guide/en/beats/filebeat/current/index.html](https://www.elastic.co/guide/en/beats/filebeat/current/index.html), also searched existing questions on this forums and thru general Google search. I see that log line offset is shared (per [https://www.elastic.co/guide/en/beats/filebeat/current/exported-fields-log.html](https://www.elastic.co/guide/en/beats/filebeat/current/exported-fields-log.html)), but not no. of **new** log lines which is what we need.

Thanks,  
Laxmi

---

<div class="post-metadata">

**Author:** ![adrisr](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adrisr/32/25423_2.png) [@adrisr](https://discuss.elastic.co/u/adrisr)\
**Post date:** [July 6, 2018, 8:49am UTC](https://discuss.elastic.co/t/content-field-number-of-new-log-lines-harvested/138656/2 "2018-07-06T08:49:38Z")

</div>

Have you given the existing [Beats Monitoring Metrics](https://www.elastic.co/guide/en/kibana/6.3/beats-page.html) a chance?

I think it's a good fit to cover all the use cases you're interested in, out of the box.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 3, 2018, 8:49am UTC](https://discuss.elastic.co/t/content-field-number-of-new-log-lines-harvested/138656/3 "2018-08-03T08:49:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.