Filebeat, raw logs to JSON format to send to elasticsearch directly

[ali_khalil]

Hi,

I had googled a lot and spent many hours but I am not able to find a satisfying answer. How can I parse the raw log strings to JSON.

Here is sample logs:
[11/Nov/2019 18:39:15] INFO [services2.abc:123] Company name: Facebook, Inc.
[11/Nov/2019 18:39:15] INFO [services2.abc:121] Company id: fsdfsfs3213
[11/Nov/2019 18:39:15] ERROR [services2.abc:121] Company not found etc.

I want to send as { 'timestamp' : '[11/Nov/2019 18:39:15]', 'message': 'Company name: Facebook, Inc.' },

How can I do that? I want filebeat oriented solution. Hoping for a quick solution.

[faec (Fae Charlton)]

Welcome! If your logs are in format with module support you can use the appropriate module to parse them. If it's a custom format, you can generate fields by setting up processors -- the dissect processor is good for simple transformations, and for more general behavior you can try the script processor.