# Separating fields using grok with custom patterns

**URL:** <https://discuss.elastic.co/t/separating-fields-using-grok-with-custom-patterns/257765>\
**Category:** Logstash\
**Created:** [December 6, 2020, 9:07am UTC](https://discuss.elastic.co/t/separating-fields-using-grok-with-custom-patterns/257765 "2020-12-06T09:07:56Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![rojin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rojin/32/97697_2.png) [@rojin](https://discuss.elastic.co/u/rojin)\
**Post date:** [December 6, 2020, 9:07am UTC](https://discuss.elastic.co/t/separating-fields-using-grok-with-custom-patterns/257765/1 "2020-12-06T09:07:56Z")

</div>

Hello! I have a problem while parsing my log file: I can not get the separated fields in stdout using ruby debug. The pattern is okay while using DevTools or [the recommended debugger](https://grokdebug.herokuapp.com/). also I have different log lines which have different patterns and I am using grok with an array of match. Here is a line from my log file:  
`[2020-10-12T21:04:15.090] [ERROR] Log - test-include : body: {"FirstName":"sdfsfd","MiddleName":"","LastName":"sfsf"} error: [object Object], status: 400`

I would like to separate "body" fields in logstash output lile so:

```
{ 
  "first_name":""
  "last_name":""
  ...
}

```

or nested like this:

```
body {
     "first_name":""
     ...
}

```

here is my grok pattern:

`\[%{TIMESTAMP_ISO8601:timestamp}\] \[%{LOGLEVEL:log_level}\] %{MSG:log_type} - %{MSG:action}-%{MSG:error_on} : %{MSG:info_type}: %{PERSON} error: %{MSG:error_message}, status: %{NUMBER:status_code:int}`

and this is my custom grok pattern:

```
MSG %{DATA}
TIMESTAMP_ISO8601 %{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T]%{HOUR}:?%{MINUTE}(?::?%{SECOND})?
    
PERSON {"FirstName":\"%{PERSONDATA:first_name}\","MiddleName":\"%{PERSONDATA:middle_name}\","LastName":\"%{PERSONDATA:last_name}\"}

PERSONDATA %{DATA}

```

here is the output by debugger:  
` { "timestamp": [[ "2020-10-12T21:04:15.090"] ], "log_level": [[ "ERROR"] ], "log_type": [[ "Log"] ], "action": [[ "test"] ], "error_on": [[ "include"] ], "info_type": [[ "body"] ], "first_name": [[ "sdfsfd"] ], "middle_name": [[ ""] ], "last_name": [[ "sfsf"] ], "error_message": [[ "[object Object]" ] ], "status_code": [] }`

this is the logstash config:

```
input {
    file {
	path => "/path/to/log"
        start_position => "beginning"
        sincedb_path => "/dev/null"
    }
}

filter {
    grok {
	patterns_dir => ["/path/to/patterns"]
        match => { "message" => ["pattern1","pattern2",...,"mentioned_pattern"]}
        remove_field => ["message","@timestamp"]
    }
}

output {
    stdout { codec => rubydebug }
}

```

here is the picture:

 ![Screenshot_2020-12-06 Grok Debugger](https://us1.discourse-cdn.com/elastic/original/3X/5/d/5d90bbad4aca505a274441f25222ea57670bede3.png)  
I would be thankful if anyone could help me with this.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 3, 2021, 9:07am UTC](https://discuss.elastic.co/t/separating-fields-using-grok-with-custom-patterns/257765/2 "2021-01-03T09:07:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.