Parse Json

Elastic Stack Logstash
1

Hello
Help me parse the logs, I'm a beginner, I don't understand how to do it.
My log:
{"bucket":"22-11-2023-test1111111111","time":"2024-04-15T10:01:53.539568Z","time_local":"2024-04-15T10:01:53.539568+0000","remote_addr":"10.1.106.86","object_owner":"<mark>lc</mark>-<mark>ogay_52511</mark>","user":"<mark>lc</mark>-<mark>ogay_52511</mark>","operation":"get_obj","uri":"HEAD /22-11-2023-test1111111111/Linux.txt HTTP/1.1","http_status":"200","error_code":"","bytes_sent":0,"bytes_received":0,"object_size":12771,"total_time":10,"user_agent":"S3 Browser/11.5.7 (https://s3browser.com)","referrer":"","trans_id":"tx000009bb45168c7caae9b-00661cfb11-14c2ce9-second","authentication_type":"Local","access_key_id":"VT534608KBQHA5A595U2P","temp_url":false}

I want to get the field "method" => "HEAD"

My conf (This is one of the options):

filter
{
    json
    {
        source => "message"
        split => ["message",", "]
        add_field => { "method" => "%{[message][2]}" }
    }
}
2 
input {
  generator {
       message => '{"bucket":"22-11-2023-test1111111111","time":"2024-04-15T10:01:53.539568Z","time_local":"2024-04-15T10:01:53.539568+0000","remote_addr":"10.1.106.86","object_owner":"<mark>lc</mark>-<mark>ogay_52511</mark>","user":"<mark>lc</mark>-<mark>ogay_52511</mark>","operation":"get_obj","uri":"HEAD /22-11-2023-test1111111111/Linux.txt HTTP/1.1","http_status":"200","error_code":"","bytes_sent":0,"bytes_received":0,"object_size":12771,"total_time":10,"user_agent":"S3 Browser/11.5.7 (https://s3browser.com)","referrer":"","trans_id":"tx000009bb45168c7caae9b-00661cfb11-14c2ce9-second","authentication_type":"Local","access_key_id":"VT534608KBQHA5A595U2P","temp_url":false}'
	   count => 1
  }
}
filter {

  json{ source=>"message"}

  prune { whitelist_names => [ "uri" ]}

  dissect { mapping => { "uri" => "%{method} %{}" } }
	 
  mutate { remove_field => ["uri"] } 
}
output {
 stdout { codec => rubydebug{} }
}

Result:

{
    "method" => "HEAD"
}
2 Likes
3

The same result, with "@timestamp" and "@version" which can be removed in the mutate remove_field list :

filter {
  grok { match => { "message" => [",\"uri\":\"%{DATA:method} %{DATA}\",\"http_status\":"]} }
	 
  mutate { remove_field => [ "event", "host", "message"] } 
}

The first version json/prune is better in case you decide to have all fields or change white_list.
The grok is simpler in case of you need only one field.

4

Thanks! It's work )))