What would be the easiest way to hook into filebeat?


(Blacktop) #1

I would like to hook into Filebeat once it has a handle on a new file. I think what I need is to create my own reader that is tsv.go (tab separated value) and implement the Reader interface?

The reason I want to do this is that the log files I am parsing have a header that can change a lot and explains how to parse the file field name/types etc.

From what I can see in the code that would allow me to define what the Next() method would output for message

Does that sound right or am I completely lost?

Thanks.


(Ed) #2

Why not try the CSV filter, it deals with headers in files, just specify an alternate delimiter
https://www.elastic.co/guide/en/logstash/current/plugins-filters-csv.html


(ruflin) #3

Can you share an example of such a file? If it is a new file type, I would opt for a new prospector type (input_type). This makes things a little bit more complex. To get started and play around with it, adding a reader is definitively a good idea :wink:


(Blacktop) #4

Do you think the new go1.8 plugin system will allow for hookin into filebeat? Or have you considered abstracting the log file read/parse/registry (state tracking) back into libbeat?


(ruflin) #5

There is currently no plugin architecture yet in filebeat. But it is planned to push the abstraction forward in filebeat to have prospectors plugin like.


(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.