Extract IP from message into a separate field

Elastic Stack Logstash
1

Hey all,
I've got syslog setup on my router and I've been wanting to extract the IPV4 field from the message into a new field

This is what one type of message looks like

36> 2017-12-11 11:46:3 0 System Detected UDP port scan attack, scan packet from 1xx.1xx.94.1.

Basic setup

filter {
if [type] == "syslog" {
grok {
match => {
message" => "%{GREEDYDATA}" }
remove_field => [ "@version" ]
}

I would like to somehow add a separate field called client for example using the %{IPV4}, is this possible or do I need to create a custom pattern which includes %{IPV4}
I take it it's not so i'd like to just confirm that.

On the topic of custom patterns, what would be the best way to set them up if I've got a range of syslog messages all different, would I just log all the different types and create a pattern for each?

Thank you

2

You need to do more with your grok than match the entire message. Break it down into chunks so you end up with an IP field :slight_smile:

3

Thanks warkolm :slight_smile:

I've been trying to keep it as simple as possible so I noticed
%{TIMESTAMP_ISO8601} should match my timestamp but when I test it in grok debugger the seconds are null so I tried something like this:

%{INT:Priority}%{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T ]%{HOUR}:?%{MINUTE}:%{INT:SECONDS}
%{INT:not_sure} %{WORD:System}%{GREEDYDATA}%{IPV4}

Once I get to the "System Detected UDP port scan attack, scan packet from.." do I just keep using WORD pattern here until I reach IPV4? or (?[a-zA-Z\s,.]

I should be able to get it working but there are other types of messages that get sent from this device and some contain the ip address in the middle, or not at all. I take it I will need to capture each difference and match it from scratch until I have all the different patterns?

4

After a few more hours of banging my head against the wall, this worked

filter {
if [type] == "syslog" {
grok {
match => { "message" => "%{IPV4:client}" }
add_tag => [ "contains_ip" ]
}
}#if
}#filter

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.