Filebeat Stats and Metrics

Elastic Stack Beats
, ,
2

Hello @Emily_Hontoria

I am not a Beat developer so the provided information might not be accurate.

I think this post might explain the dropped meaning: Safely publish an event using libbeat - #3 by steffens

An example of an event dropped in libbeat output is a document which is malformed (e.g. bad encoding) when being sent to a final destination (e.g. Elasticsearch).
An example of an event dropped in libbeat pipeline is a document which is excluded (e.g. exclude_lines in log input or drop_event processor.

We document those fields:

  • In GoDoc
  • The exported fields when monitoring Filebeat using Metricbeat Filebeat module (doc)

I think:

  • the output stats are related to the declared output in the configuration (e.g. Beats are able to send to different destinations)
  • the pipeline stats are related to the declared processors, filters, conditionals and the internal queue

We document those fields in GoDoc.

I think the difference is the following:

  • a filebeat event is an event with specific fields, respecting the filebeat definition of event (e.g. a filebeat event coming from syslog tcp input with its dedicated fields or from a log input), built on top of a libbeat event
  • a libbeat event is the common event format shared by all beats. Every event must have a timestamp and provide encodable Fields in Fields

In any case, those metrics are meant to be digested by the Elastic Stack Monitoring tools (see here).

1 Like