I would like to hook into Filebeat once it has a handle on a new file. I think what I need is to create my own reader that is tsv.go (tab separated value) and implement the Reader interface?
The reason I want to do this is that the log files I am parsing have a header that can change a lot and explains how to parse the file field name/types etc.
From what I can see in the code that would allow me to define what the Next() method would output for message
Can you share an example of such a file? If it is a new file type, I would opt for a new prospector type (input_type). This makes things a little bit more complex. To get started and play around with it, adding a reader is definitively a good idea
Do you think the new go1.8 plugin system will allow for hookin into filebeat? Or have you considered abstracting the log file read/parse/registry (state tracking) back into libbeat?
There is currently no plugin architecture yet in filebeat. But it is planned to push the abstraction forward in filebeat to have prospectors plugin like.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
