What would be the easiest way to hook into filebeat?

I would like to hook into Filebeat once it has a handle on a new file. I think what I need is to create my own reader that is tsv.go (tab separated value) and implement the Reader interface?

The reason I want to do this is that the log files I am parsing have a header that can change a lot and explains how to parse the file field name/types etc.

From what I can see in the code that would allow me to define what the Next() method would output for message

Does that sound right or am I completely lost?


Why not try the CSV filter, it deals with headers in files, just specify an alternate delimiter

Can you share an example of such a file? If it is a new file type, I would opt for a new prospector type (input_type). This makes things a little bit more complex. To get started and play around with it, adding a reader is definitively a good idea :wink:

Do you think the new go1.8 plugin system will allow for hookin into filebeat? Or have you considered abstracting the log file read/parse/registry (state tracking) back into libbeat?

There is currently no plugin architecture yet in filebeat. But it is planned to push the abstraction forward in filebeat to have prospectors plugin like.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.