Logstash Filter

Elastic Stack Logstash
1

<166>1 2021-05-11T17:50:40+0000 flowIdLog, applianceName=nameofsight, app=app1

How do i parse this, the KV section works but i dont know how to deal with this section
" <166>1 2021-05-11T17:50:40+0000 flowIdLog "

filter{
dissect { mapping => { "message" => "%{[@metadata][ts]} %{} %{[@metadata][restOfLine]}" } }
kv { source => "%{[@metadata][restOfLine]}" field_split => "," value_split => "=" }
}
2

I would change the dissect to be

"%{} %{[@metadata][ts]} %{} %{[@metadata][restOfLine]}"

The first %{} will consume the <166>1

You may want to use trim_key in the kv filter to remove spaces, or else use field_split_pattern match comma and space.

3

I have it changed to this . but it is not paring at all

filter{
dissect { mapping => { "message" => "%{} %{[@metadata][ts]} %{}, %{[@metadata][restOfLine]}" } }
kv { source => "%{[@metadata][restOfLine]}" field_split => "," value_split => "=" }
}
4

Can you post an example of a message that it fails to parse? Please use </> in the toolbar to quote it.

5

<166>1 2021-05-12T13:26:21+0000 flowIdLog, applianceName=asdasdas, tenantName=fgdfdcbcvb, flowId=234234, flowCookie=1620826004, sourceIPv4Address=10.0.0.0, destinationIPv4Address=0.0.0.0, sourcePort=55838, destinationPort=443, tenantId=1, vsnId=0, applianceId=1, ingressInterfaceName=vni-0/4.4, egressInterfaceName=vni-0/0.0, fromCountry=, toCountry=United States, protocolIdentifier=6, fromZone=Intf-LAN-Zone, fromUser=Unknown, toZone=Intf-DIA-Zone, icmpTypeIPv4=0

6

That should be

source => "[@metadata][restOfLine]"

I suggest you also add

trim_key => " "

Otherwise you will have spaces at the start of the keys, like this:

    " protocolIdentifier" => "6",
            " tenantName" => "fgdfdcbcvb",
                 " vsnId" => "0",
1 Like
7

thanks that did the trick!

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.