Cant parse multiline log

Elastic Stack Logstash
1

i have a one kind of multiline log like this and i dont know how to parse it, all the lines after the first always is the same but can be any number of lines

2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd UPDATE w/ attr: nexthop 172.16.11.1, origin i, localpref 100, metric 13020, community 32098:2000 32098:2015, originator 172.16.11.1, clusterlist 172.16.10.0, path 174 1299 9829
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 103.194.28.0/22
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 45.249.136.0/22
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 117.236.192.0/20
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 117.208.240.0/20
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 103.213.32.0/22
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 192.140.240.0/22
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 58.84.20.0/22
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 160.202.204.0/22
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 103.218.232.0/22
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 103.247.52.0/22
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 160.202.200.0/22
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 160.202.180.0/22
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 103.213.36.0/22
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 43.239.60.0/22
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 43.239.168.0/22
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 114.29.248.0/22
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 117.236.176.0/20
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 137.59.132.0/23
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 103.216.148.0/22
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 59.93.64.0/20
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 117.236.208.0/20
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 103.61.214.0/23
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 117.248.128.0/22
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 117.219.0.0/20
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 45.124.154.0/23
2019/02/14 11:54:33 BGP: 172.16.10.254 rcvd 117.240.225.0/24

2

Do you want to combine those lines, or parse each line as a separate event?

3

All the lines are one event so i want to combine all.

4

The log means all the lines after the first to the end line have the same properties of the first line.

5

Well, you could combine all the lines into one using a multiline code with a regexp that never matches. Something like

codec => multiline { pattern => "^Spalanzani" negate => true what => "previous" auto_flush_interval => 2 }

You can pick apart the first line using grok or dissect. To deal with the list of subnets I would use a ruby filter to scan for a regex. That's going to be easier if we switch all the newlines to be some other character.

    mutate { gsub => [ "message", "
", "|" ] }
    ruby {
        code => '
            a = event.get("message").scan(/rcvd ([0-9\.\/]+)\|/)
            event.set("subnets", a.flatten)
        '
    }

will get you

   "subnets" => [
    [ 0] "103.194.28.0/22",
    [ 1] "45.249.136.0/22",
    [ 2] "117.236.192.0/20",
[...]
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.