One line JSON parsing into separate events/logs


I am trying to ingest some logs via filebeats, the logs get to logstash but only one event is created, the logs are all on one line as per the example below, I have trimmed it down.

{"requests": [{"originId": 283019416, "originType": "Networks", "internalIp": "", "externalIp": "", "categories": ["Malware"], "tags": , "destination": "", "originLabel": "primary int", "actionTaken": "BLOCKED", "datetime": "2019-07-16T15:21:49.000Z"}, {"originId": 283019416, "originType": "Networks", "internalIp": "", "externalIp": "", "categories": ["Malware"], "tags": , "destination": "", "originLabel": "primary in", "actionTaken": "BLOCKED", "datetime": "2019-07-16T14:18:14.000Z"}]}

An example of my filebeat config is as below

  • type: log
    • /opt/logcollection/*.json
      json.keys_under_root: false
      json.add_error_key: true
      tags: ["dns"]

my logstash conf is:
input {
beats {
port => "5044"
tags => [ "beat" ]
codec => "json"
I have also tried json_lines and the json filter. I have validated the JSON i just cannot seem to get filebeats or logstash to separate the events. Are filebeats/logstash the right place to do this or should I be using the python request to parse the data and create the events on separate lines?

I have gone through a number of threads and the elastic guides, tried a lot of permutations of the settings and I do not seem to be getting anyway and hoping if someone can point me in the right direction.


can you please format config and sample events using </>

just to be on the same page. you expect event to be created for each request out of set of requests, right?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.