How to decode non-text logs

Elastic Stack Beats
1

Hi,

We have some logs in the "FixLog" format, for which users can generally use a tool called fixlog on the machine to read them as one would use the "less" command.

http://www.fixtradingcommunity.org/
http://fixlogviewer.com/

These logs are annoyingly not regular text logs, but not far off. Is there a way to use a third party tool for filebeat to parse logs? Or any suggestions as to how best to deal with a log that filebeat can't read natively?

Thanks in advance

2

Can you share an example of such a log?

3

Turns out I am an idiot, and actually Filebeat parses it nicely - I just couldn't read it on the command line.

That said, I wonder if there's a way to direct another application at Filebeat to read the log. I can't imagine many instances where this is necessary (especially on Linux) but I bet there are strangely formatted logs on Windows that would require a particular application to read. Are there any ways in which these could be ingested like this?

Or, from another angle, what if one wanted to ingest compressed logs without having to decompress them - ad-hoc ingestion from archived logs for example. I've needed to do this myself more than once.

4

I assume that is what you are looking for? https://github.com/elastic/beats/issues/637 For the common case of a compress log file the idea is to have it as a separate prospector in filebeat.

5

This topic was automatically closed after 21 days. New replies are no longer allowed.