# Filebeat INFO output - libbeat.pipeline.events.filtered
**URL:** https://discuss.elastic.co/t/filebeat-info-output-libbeat-pipeline-events-filtered/115798
**Category:** Beats
**Tags:** filebeat
**Created:** [January 16, 2018, 10:20pm UTC](https://discuss.elastic.co/t/filebeat-info-output-libbeat-pipeline-events-filtered/115798 "2018-01-16T22:20:18Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@rkearsley](https://discuss.elastic.co/u/rkearsley)
#### Post date: [January 16, 2018, 10:20pm UTC](https://discuss.elastic.co/t/filebeat-info-output-libbeat-pipeline-events-filtered/115798/1 "2018-01-16T22:20:18Z")
Hello
Watching filebeat run in the console with '-e' option, I can see it spitting out "INFO Non-zero metrics in the last 30s:"
I looked closer at some of the values:
beat.info.uptime.ms=29996
beat.memstats.gc\_next=26967824
beat.memstats.memory\_alloc=20770720
beat.memstats.memory\_total=5068039560
filebeat.events.added=144
filebeat.events.done=144
filebeat.harvester.closed=3
filebeat.harvester.open\_files=0
filebeat.harvester.running=0
filebeat.harvester.started=3
libbeat.config.module.running=0
libbeat.output.read.bytes=1404
libbeat.output.write.bytes=112515
libbeat.pipeline.clients=1
libbeat.pipeline.events.active=0
**libbeat.pipeline.events.filtered=6**
libbeat.pipeline.events.published=138
libbeat.pipeline.events.total=144
libbeat.pipeline.queue.acked=138
registrar.states.current=343
registrar.states.update=144
registrar.writes=6
I could not find much info on what these values mean, but I am particularly interested in the one called 'libbeat.pipeline.events.filtered' because it seems to suggest 6 of my log lines didn't make it through to be published. I could be wrong but looking for clarification... (and a way to find the problem with those 6 lines if indeed they are not being published)
Filebeat version 6.1, json log input and ES output
Thanks
Richard
---
### Author:  [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)
#### Post date: [January 16, 2018, 10:34pm UTC](https://discuss.elastic.co/t/filebeat-info-output-libbeat-pipeline-events-filtered/115798/2 "2018-01-16T22:34:12Z")
Please share the complete Filebeat configuration you are using.
---
### Author:  [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)
#### Post date: [January 16, 2018, 10:39pm UTC](https://discuss.elastic.co/t/filebeat-info-output-libbeat-pipeline-events-filtered/115798/3 "2018-01-16T22:39:46Z")
I believe this metric gets incremented with a processor intentional drops an event. Are you using processors?
- Related source: [https://github.com/elastic/beats/blob/84e021d2ca391a2a924e214bd039517933ec3f5a/libbeat/publisher/pipeline/client.go#L87-L89](https://github.com/elastic/beats/blob/84e021d2ca391a2a924e214bd039517933ec3f5a/libbeat/publisher/pipeline/client.go#L87-L89)
---
### Author:  [@rkearsley](https://discuss.elastic.co/u/rkearsley)
#### Post date: [January 16, 2018, 10:42pm UTC](https://discuss.elastic.co/t/filebeat-info-output-libbeat-pipeline-events-filtered/115798/4 "2018-01-16T22:42:09Z")
Hi, thanks for looking. Here's the config
```
filebeat:
prospectors:
-
paths:
- /data/logs/json/*.json
input_type: log
close_inactive: 5m
close_eof: true
clean_inactive: 25h
ignore_older: 24h
harvester_limit: 2
scan.sort: "filename"
json.keys_under_root: false
setup.template.name: "edge"
setup.template.pattern: "edge-*"
setup.template.fields: "template.json"
output.elasticsearch:
hosts: ["http://x.x.x.x:9200"]
index: "edge-nginx-%{+yyyy.MM.dd}"
worker: 8
bulk_max_size: 4096
```
---
### Author:  [@andrewkroh](https://discuss.elastic.co/u/andrewkroh)
#### Post date: [January 17, 2018, 3:30pm UTC](https://discuss.elastic.co/t/filebeat-info-output-libbeat-pipeline-events-filtered/115798/5 "2018-01-17T15:30:32Z")
No processors are used in the config so I'm surprised that `libbeat.pipeline.events.filtered` is \> 0.
@steffens What would cause `libbeat.pipeline.events.filtered > 0` ?
---
### Author:  [@GaryHuang](https://discuss.elastic.co/u/GaryHuang)
#### Post date: [January 18, 2018, 12:25pm UTC](https://discuss.elastic.co/t/filebeat-info-output-libbeat-pipeline-events-filtered/115798/6 "2018-01-18T12:25:30Z")
It may be caused by the empty lines. And there are many other events, like files state updated that u can find in Prospector.updateState.
---
### Author:  [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [January 19, 2018, 4:55pm UTC](https://discuss.elastic.co/t/filebeat-info-output-libbeat-pipeline-events-filtered/115798/7 "2018-01-19T16:55:15Z")
Right, filebeat creates empty events on file state updates. These should not be published to the output, but must be processed (ACKed) in order with the other events, so to have the registry file updated in order. These empty events are filtered out by the publisher pipeline.
It's some 'internals' we hope to get rid of in the future.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 16, 2018, 4:55pm UTC](https://discuss.elastic.co/t/filebeat-info-output-libbeat-pipeline-events-filtered/115798/8 "2018-02-16T16:55:50Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.