How to parse json in grok

Elastic Stack Logstash
1

I was trying to parse my Neutron log through grok pattern, the last two fileds are two json string, here is the line.

2018-04-19T05:08:42.833+02:00 localhost neutron-api INFO [type:operate] [pid:2157] [MainThread] [accesslog.py:77 call] 172.28.0.24 - - - encode-ZDk3ZDQ0ZmZiODA5MTNiMGMxMzg5OWFj [19/Apr/2018:03:08:42.737146][19/Apr/2018:03:08:42.832755] POST https://network.az1.dc1.huawei.com/v2.0/ports HTTP/1.0 409 191 {"port":{"name":"VM001_1_NIC_1","admin_state_up":true,"network_id":"b358bc9f-6891-4f32-bbe7-4e8e474f95dc","fixed_ips":[{"subnet_id":"352faa22-e68c-4d4e-9399-a39310888918","ip_address":"19.167.210.56"}],"binding:vnic_type":"normal"}} {"NeutronError": {"message": "Unable to complete operation for network b358bc9f-6891-4f32-bbe7-4e8e474f95dc. The IP address 19.167.210.56 is in use.", "type": "IpAddressInUse", "detail": ""}}

I am using the pattern like this

%{TIMESTAMP_ISO8601:timestamp}%{SPACE}%{SYSLOGHOST:host}%{SPACE}%{WORD}-%{WORD}%{SPACE}%{LOGLEVEL:level}%{SPACE}(?[.{1,100}])%{SPACE}%{IP:ip}%{SPACE}(?.{1,100}])(?[.{1,100}])%{SPACE}%{WORD:operation}%{SPACE}%{URI:uri}%{SPACE}%{WORD}/%{BASE10NUM}%{SPACE}%{BASE10NUM}%{SPACE}%{BASE10NUM}%{SPACE}(?{"port".{1,1000}}})

How to have two fields that contains the last two json string so that I can pass to json filter?

Really appreciate your help.

2

If the first JSON object never contains a space then the following would work. If it can contain a space I cannot think of a way of doing it except for a complicated ruby filter that takes a string containing the the two JSON objects and counts brackets to split them.

  dissect { mapping => { "message" => '%{timestamp} %{host} %{appname} %{level} [%{}] [pid:%{}] [%{threadname}] %{ip} - - -  %{} [%{}][%{}] %{operation} %{uri} %{}/%{} %{} %{} %{json1} %{json2}' } }
  json { source => "json1" target => "first" }
  json { source => "json2" target => "second" }
3

Thank you! It works perfectly in my enviroment!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.