My current logging infrastructure uses just filebeats and elasticsearch.
Would like to enhance the quality of the logs by parsing out individual fields columns.
Looking for recommendation on best practice to do this.
- Add an extra logstash layer.
- Format the logs with https://pypi.python.org/pypi/logstash_formatter ; but still send the logs directly to ElasticSearch. Because the output is json; it does not seem as if the Logstash is necessary.
In particular one problem I am struggling with is multiline outputs.
Thanks for the help.