Custom beat - re-use Filebeat log readers?

Elastic Stack Beats
1

I'm creating a new beat application, primarily to remove the logstash requirements for getting this data into Elastic Search. The file being read is new line delimited JSON, and gets rotated by logrotate.

Anyone reusing filebeat code for the "tailing" and "registry" tracking in their custom beat applications? Or should I just roll my own.

Thanks.

2

primarily to remove the logstash requirements for getting this data into Elastic Search.

You are aware of Filebeat's ability to send data directly to ES, right?

3

As JSON? Last I tried it sent to the JSON as text lines. I had to run it through Logstash to be added as JSON. Has this been changed?

4

Filebeat 5.0 alpha1 adds json per line support. The JSON is parsed and send as event to elasticsearch. Try console output with pretty: true to check out your events. Even better, Elasticsearch 5.0 will have Ingest-Node adding some filtering/parsing capabilities to Elasticsearch input.