# How i to parse log

**URL:** https://discuss.elastic.co/t/how-i-to-parse-log/126310
**Category:** Logstash
**Created:** [March 31, 2018, 9:33am UTC](https://discuss.elastic.co/t/how-i-to-parse-log/126310 "2018-03-31T09:33:00Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![edwinhsieh](https://avatars.discourse-cdn.com/v4/letter/e/76d3ee/32.png) [@edwinhsieh](https://discuss.elastic.co/u/edwinhsieh)
#### Post date: [March 31, 2018, 9:33am UTC](https://discuss.elastic.co/t/how-i-to-parse-log/126310/1 "2018-03-31T09:33:00Z")

</div>

Hi everyone,

I'm a novice to handle log with grok pattern on logstash.

I have a log like that:  
`[32mINFO \u001b[0m[03-31|09:09:27|print/print.go:20] fire a hole in the room, give you a burst way \tprint=testing tickat=2018-03-31T09:09:27+0000 hash=0x12345 previous=no after=true drop=event:sandbox events=\"bank did added\"`

My grok patten filter can be follows:  
`%{NOTSPACE:notSpace}\[%{DATA:textColor}%{LOGLEVEL:logLevel} %{NOTSPACE:notSpace}\[%{DATA:textColor}\[%{MONTHNUM:month}-%{MONTHDAY:day}\|%{TIME:HHmmss}\|%{GREEDYDATA:logPath}\] %{GREEDYDATA:logContext} %{GREEDYDATA:keyPairs}`

But the filter result doesn't follow my request 😭😭😭  
{  
"HHmmss": "07:51:53",  
"textColor": "32m",  
"logLevel": "INFO",  
"month": "03",  
"logPath": "print/print.go:20",  
"keyPairs": "added"",  
"logContext": "fire a hole in the room, give you a burst way print=testing tickat=2018-03-31T07:51:53+0000 hash=0x12345 previous=no after=true drop=event:sandbox events="bank did",  
"day": "31",  
"notSpace": "\u001b"  
}

The keyPairs string should be

> print=testing tickat=2018-03-31T07:51:53+0000 hash=0x12345 previous=no after=true drop=event:sandbox events="bank did added"

and logContext is

> fire a hole in the room, give you a burst way

Could anyone please give me some advice?

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [March 31, 2018, 11:17am UTC](https://discuss.elastic.co/t/how-i-to-parse-log/126310/2 "2018-03-31T11:17:27Z")

</div>

What indicates the beginning of the key/value pairs? The string "print="? Or the first "key=value" string? Something else?

For a human it's pretty obvious that the key/value pairs begin with "print=testing" but how can this be formalized?

---

<div class="post-metadata">

### Author: ![edwinhsieh](https://avatars.discourse-cdn.com/v4/letter/e/76d3ee/32.png) [@edwinhsieh](https://discuss.elastic.co/u/edwinhsieh)
#### Post date: [March 31, 2018, 12:23pm UTC](https://discuss.elastic.co/t/how-i-to-parse-log/126310/3 "2018-03-31T12:23:15Z")

</div>

"=" indicates key-value pairs while space separates the pairs.

My original problem is how to split a logContext into **message** and **key/value pairs string**.

message is mean that

> fire a hole in the room, give you a burst way

key/value pairs string is

> print=testing tickat=2018-03-31T07:51:53+0000 hash=0x12345 previous=no after=true drop=event:sandbox events="bank did added"

The purpose of key/value pairs string is that i can tell KV filter what's my source.

But I found a magical and powerful KV filter on logstash, it can solve my problem.

I pass the logContext  
`filter { kv { source => "logContext" } }`  
and it feedback the KV result.  
`{ print=testing tickat=2018-03-31T07:51:53+0000 hash=0x12345 previous=no after=true drop=event:sandbox events="bank did added" }`

But if i can get the message(`fire a hole in the room, give you a burst way`) by grok pattern it would be great!

---

<div class="post-metadata">

### Author: ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [March 31, 2018, 8:46pm UTC](https://discuss.elastic.co/t/how-i-to-parse-log/126310/4 "2018-03-31T20:46:59Z")

</div>

Okay, so something like

```
... %{DATA:logContext} (?<keyPairs>\w+=\S+ .*)

```

should work, i.e. store everything in `logContext` until you've found a `\w+=\S+` sequence

---

<div class="post-metadata">

### Author: ![edwinhsieh](https://avatars.discourse-cdn.com/v4/letter/e/76d3ee/32.png) [@edwinhsieh](https://discuss.elastic.co/u/edwinhsieh)
#### Post date: [April 1, 2018, 4:02am UTC](https://discuss.elastic.co/t/how-i-to-parse-log/126310/5 "2018-04-01T04:02:10Z")

</div>

thank you!!! 😄😄😄

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [April 29, 2018, 4:02am UTC](https://discuss.elastic.co/t/how-i-to-parse-log/126310/6 "2018-04-29T04:02:18Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
