I just wanted to drop a note that I'm in the making of writing a community beat that I call journalbeat.
I'm interested in getting some feedback on the idea of this project, and also - of course - to gather some ideas on what people would expect it to do.
Besides from the obvious use case (shipping logs), I'm also developing it to get a central and common data source for more advanced topics like FIM, SIEM, Audit Logs / Monitoring / Alerting. Input and ideas on this particular use case are highly appreciated.
The software is working so far (I tested and developed on Fedora 23), and has the following features:
- starting following the system journal from 3 different locations: the beginning of the journal, the end of the journal, where you stopped parsing the last time
- reads all journal fields into one event
- adds journal catalog entries if possible
- can normalize / "clean" field names (__REALTIME_TIMESTAMP becomes realtime_timestamp)
- can try to convert number fields to numbers
- can move all journal fields to an object field
My current plans are to add the following features near term:
- whitelists and blacklists for fields
- filtering of messages (like
- JSON parsing of message fields if possible
Any ideas / suggestions / feedback are highly appreciated!