Filebeat, raw logs to JSON format to send to elasticsearch directly


I had googled a lot and spent many hours but I am not able to find a satisfying answer. How can I parse the raw log strings to JSON.

Here is sample logs:
[11/Nov/2019 18:39:15] INFO [] Company name: Facebook, Inc.
[11/Nov/2019 18:39:15] INFO [] Company id: fsdfsfs3213
[11/Nov/2019 18:39:15] ERROR [] Company not found etc.

I want to send as { 'timestamp' : '[11/Nov/2019 18:39:15]', 'message': 'Company name: Facebook, Inc.' },

How can I do that? I want filebeat oriented solution. Hoping for a quick solution.

Welcome! If your logs are in format with module support you can use the appropriate module to parse them. If it's a custom format, you can generate fields by setting up processors -- the dissect processor is good for simple transformations, and for more general behavior you can try the script processor.

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.