# Snort Logs with FileBeat

**URL:** <https://discuss.elastic.co/t/snort-logs-with-filebeat/177879>\
**Category:** Logstash\
**Created:** [April 22, 2019, 4:04pm UTC](https://discuss.elastic.co/t/snort-logs-with-filebeat/177879 "2019-04-22T16:04:07Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![johndowe](https://avatars.discourse-cdn.com/v4/letter/j/3ab097/32.png) [@johndowe](https://discuss.elastic.co/u/johndowe)\
**Post date:** [April 22, 2019, 4:04pm UTC](https://discuss.elastic.co/t/snort-logs-with-filebeat/177879/1 "2019-04-22T16:04:07Z")

</div>

Hi,

I have setup filebeat on a pi running Snort sending logs to a cloud ELK stack. I am trying to figure out how to arrange logs and doing the following process:

on the beats side i have this in the filebeat.yml:  
paths:  
- /var/log/snort/alert  
tags: ["snort"]

Whilst in the filter on the logstash side I have the below:

if "snort" in [tags] {

```
# parse the message into individual fields
grok {
    match => { "message" => "(?<ts>.*\d{2}:\d{2}:\d{2})\s(?<host>.*?)\s.*?\s\[(?<generator_id>.*?)::(?<signature_id>.*?):.*?\]\s(?<signature>.*?)\s\[Classification:\s(?<classification>.*?)\]\s\[Priority:\s(?<priority>.*?)\].*?{(?<protocol>.*?)\}\s(?<source_ip>.*?):(?<source_port>.*?)\s-\>\s(?<destination_ip>.*?):(?<destination_port>.*)" }
}

# remove the original message if parsing was successful
if !("_grokparsefailure" in [tags]) {
    mutate {
        remove_field => ["message"]
    }
}

# parse the timestamp and save in a new datetime field
if [ts] {
    date {
        match => ["ts", "MMM dd HH:mm:ss"]
        target => "sys_timestamp"
    }

    # remove the original timestamp if date parsing was successful
    if !("_dateparsefailure" in [tags]) {
        mutate {
            remove_field => ["ts"]
        }
    }
}

```

}

A sample log file in snort looks as such:  
04/22-17:47:29.436774 [**] [1:29456:2] PROTOCOL-ICMP Unusual PING detected [**] [Classification: Information Leak] [Priority: 2] {ICMP} 192.168.4.1 -\> 192.168.4.9

Unfortunately the above is not working and noticed a tag in Kibana with snort logs as follows: \_grokparsefailure.

Any thoughts on making this work?

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [April 23, 2019, 9:21pm UTC](https://discuss.elastic.co/t/snort-logs-with-filebeat/177879/2 "2019-04-23T21:21:03Z")

</div>

I moved it to the Logstash forums.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 23, 2019, 10:06pm UTC](https://discuss.elastic.co/t/snort-logs-with-filebeat/177879/3 "2019-04-23T22:06:35Z")

</div>

Firstly, your pattern first breaks at (?.\*\d{2}:\d{2}:\d{2})\s since that does not match the microsecond part of the timestamp.

Secondly, application of the common options like remove\_field, which is referred to as decoration, only occurs if the filter successfully executes. Thus

```
grok {
    match => { "message" => "(?<ts>.*\d{2}:\d{2}:\d{2})\..." }
}
# remove the original message if parsing was successful
if !("_grokparsefailure" in [tags]) {
    mutate { remove_field => ["message"] }
}

```

is exactly equivalent to

```
grok {
    match => { "message" => "(?<ts>.*\d{2}:\d{2}:\d{2})\..." }
    mutate { remove_field => ["message"] }
}

```

Thirdly, I would not use grok at all to parse that line, I would use dissect.

```
dissect { mapping => { "message" => "%{ts} [%{field1}] [%{field2}] %{signature} [%{field5}] [Classification: %{classification}] [Priority: %{priority}] %{restOfLine}" } }

```

would get you

```
    "restOfLine" => "{ICMP} 192.168.4.1 ->\n192.168.4.9",
        "field1" => "",
        "field5" => "",
    "@timestamp" => 2019-04-23T22:02:44.738Z,
     "signature" => "PROTOCOL-ICMP Unusual PING detected",
      "priority" => "2",
"classification" => "Information Leak",
       "message" => "04/22-17:47:29.436774 [] [1:29456:2] PROTOCOL-ICMP Unusual PING detected [] [Classification: Information Leak] [Priority: 2] {ICMP} 192.168.4.1 ->\n192.168.4.9",
        "field2" => "1:29456:2",
            "ts" => "04/22-17:47:29.436774"

```

}

I can see you are trying to chop up what I have labelled as field2, but your regular expression appears not to be valid so I cannot tell what you want there. In the dissect you can replace [%{field2}] with [%{field2}:%{field3}:%{field4}] if you want the three subfields split. And obviously you can call them anything that like.

If you do not want the contents of field1 then you can replace [%{field1}] with [%{}], or even if it is always empty.

---

<div class="post-metadata">

**Author:** ![johndowe](https://avatars.discourse-cdn.com/v4/letter/j/3ab097/32.png) [@johndowe](https://discuss.elastic.co/u/johndowe)\
**Post date:** [April 30, 2019, 8:59pm UTC](https://discuss.elastic.co/t/snort-logs-with-filebeat/177879/4 "2019-04-30T20:59:53Z")

</div>

Thanks a lot badger for this... I have tried it out and it works, if I want to chop up further restofline field , so one ip as the source IP and the other as source IP is that possible?

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 30, 2019, 9:20pm UTC](https://discuss.elastic.co/t/snort-logs-with-filebeat/177879/5 "2019-04-30T21:20:29Z")

</div>

Yes, you can use a second grok. For that ICMP message I think this would work

```
grok { match => { "restOfLine" => "^{(?<protocol>.*?)\}\s(?<source_ip>.*?)\s->\s(? 
<destination_ip>.*)" } }

```

Note that if you have other types of lines (other than ICMP) you can match restOfLine against an array of patterns. That's why I dealt with the first part of the line (which is constant across lines) in the first grok.

Note also that I anchored the ICMP pattern with ^, so that it fails quickly if it is not an ICMP line.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 28, 2019, 9:20pm UTC](https://discuss.elastic.co/t/snort-logs-with-filebeat/177879/6 "2019-05-28T21:20:45Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.