I am working on a new beat called brobeat here - https://github.com/blacktop/brobeat.
However, it looks like what I want is to build a filebeat module? Does it make sense for me to create a copy/fork of filebeat and add my bro-module?
All of the community beats seem to consume some API. What I need is to read log files off the disk and ingest them into elasticsearch and 'grok pattern match them' and maybe rename a few fields etc etc.
I am trying to avoid using Logstash if I can for now.
So I guess that libbeat doesn't do a logstash like function, it just ships logs, but can tell an Elastic Ingest node what pipeline to use?
So I would have to define a bunch of ingest-pipelines?
What do you think my path should be?
I would also love to talk to someone about how the filebeat modules integrate into filebeat. I don't see mentions of them in the filebeat golang code? Maybe it is happening deeper in libbeat somewhere?
Any help MUCH appreciated.