# How to grok those multiline log syntax?

**URL:** https://discuss.elastic.co/t/how-to-grok-those-multiline-log-syntax/314118
**Category:** Logstash
**Created:** [September 10, 2022, 4:37pm UTC](https://discuss.elastic.co/t/how-to-grok-those-multiline-log-syntax/314118 "2022-09-10T16:37:35Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![cowderwelsh](https://avatars.discourse-cdn.com/v4/letter/c/90db22/32.png) [@cowderwelsh](https://discuss.elastic.co/u/cowderwelsh)
#### Post date: [September 10, 2022, 4:37pm UTC](https://discuss.elastic.co/t/how-to-grok-those-multiline-log-syntax/314118/1 "2022-09-10T16:37:35Z")

</div>

Hi,  
I'm new to Elastic and after creating my first pipeline, I'm now trying to process a simple, local multiline Firewall log, it looks like this:

Time: 01/28/2022 01:27:22  
Event: Traffic  
IP-Address: 20.199.120.85  
Description:  
Path:  
Message: Blocked Incoming TCP - Source 20.199.120.85 : https (443) Destination  
192.168.0.158 : (49694)  
Rule: Block all traffic

I have configured my filebeat.yml for multiline events but was not able to test it so far, because I'm failing completely when it comes to grokking the fields here with the Grok debugger.  
Afaik, after getting treated as a multiline, all lines are packed into one single line.  
The fields "Description: " and "Path: " are sometimes empty, but sometimes they contain something.  
I started with something like this:  
%{DATESTAMP:Time}%{GREEDYDATA}%{WORD:Event}%{GREEDYDATA}%{IPV4:IP}%{GREEDYDATA}%{WORD:Description}%{GREEDYDATA}%{WORD:Path}... and then gave up.

After hours of back and forth, I'm completely frustrated and I have now clue how to handle this. I guess I might need regex, but how? It sounds easy: Create a description field and ingest the following data. If fields like "Description: " and "Path: " are empty, leave them blank and continue.

I hope my explanation was somehow clear, I guess for you all this might be one of the easiest tasks ever, but everything I tried so far, failed.

I would be thankful if someone could help me out with this.

Thank you!

---

<div class="post-metadata">

### Author: ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)
#### Post date: [September 10, 2022, 7:12pm UTC](https://discuss.elastic.co/t/how-to-grok-those-multiline-log-syntax/314118/2 "2022-09-10T19:12:58Z")

</div>

This should work

```auto
filter {
    # if you get by filebeat, grok should start with (?m)
    # grok { match => { "message" => "(?m)Time....
	grok { 
		match => { "message" => "Time:\s*%{DATESTAMP:timestamp}\s*Event:\s*%{WORD:event}\s*IP-Address:\s*%{IP:ip}\s*Description:\s*%{DATA:description}\s*Path:\s*%{DATA:path}\s*Message:\s*%{DATA:msg}\s*Rule:\s*%{GREEDYDATA:rule}" }
	}
	
	date {
		match => ["timestamp", "MM/dd/yyyy HH:mm:ss"]
		remove_field => ["log", "host", "message","timestamp"] #, "tags"
	}
	
	if [ip]{
	  geoip {
		source => "[ip]"
		ecs_compatibility => "disabled"
	  }
	}

}
output {
 stdout {codec => rubydebug}
}

```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [September 10, 2022, 7:33pm UTC](https://discuss.elastic.co/t/how-to-grok-those-multiline-log-syntax/314118/3 "2022-09-10T19:33:02Z")

</div>

Another approach would be

```
    grok {
        break_on_match => false
        match => {
            message => [
                "^Time: (?<time>[^\n]+)$",
                "^Event: (?<event>[^\n]+)$",
                "^IP-Address: (?<ipAddress>[^\n]+)$",
                "^Description: (?<description>[^\n]+)$",
                "^Message: (?<msg>[^\n]+)$",
                "^Rule: (?<rule>[^\n]+)$"
            ]
        }
    }

```

which produces

```
       "msg" => "Blocked Incoming TCP - Source 20.199.120.85 : https (443) Destination",
 "ipAddress" => "20.199.120.85",
      "rule" => "Block all traffic",
      "time" => "01/28/2022 01:27:22"

```

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 8, 2022, 7:33pm UTC](https://discuss.elastic.co/t/how-to-grok-those-multiline-log-syntax/314118/4 "2022-10-08T19:33:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.