# Optional fields in grok pattern are not working

**URL:** <https://discuss.elastic.co/t/optional-fields-in-grok-pattern-are-not-working/51854>\
**Category:** Logstash\
**Created:** [June 4, 2016, 1:13pm UTC](https://discuss.elastic.co/t/optional-fields-in-grok-pattern-are-not-working/51854 "2016-06-04T13:13:44Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![punu31090](https://avatars.discourse-cdn.com/v4/letter/p/b2d939/32.png) [@punu31090](https://discuss.elastic.co/u/punu31090)\
**Post date:** [June 4, 2016, 1:13pm UTC](https://discuss.elastic.co/t/optional-fields-in-grok-pattern-are-not-working/51854/1 "2016-06-04T13:13:44Z")

</div>

Following are my log lines:

## Timestamp: 27-05-2016 13:25:17 Title:Customer.GetCustomer() Message: Searched Customers for number : 123 **InnerException** : An error has occurred **StackTrace** : An error has occurred Machine: PQR

## Timestamp: 27-05-2016 13:25:17 Title:Customer.GetCustomer() Message: Searching Customers for number : 123 Machine: PQR

## Timestamp: 27-05-2016 13:25:17 Title:Customer.GetCustomer() Message: Searching Customers for number : 123 **StackTrace** : An error has occurred Some exception has taken place Machine: PQR

Fields like 'InnerException' and 'StackTrace' are optional in my log.  
Following is the grok pattern which I have constructed, **but its not working**.  
'InnerException' and 'Trace' are becoming part of 'Message' field, they are not getting created as separate fields, if present in log.

(?m)%{DATESTAMP:Timestamp}\s+%{TITLE:ServiceTitle}\s+%{MESSAGE:ServiceMessage}(\s+%{IEXCEPTION:InnerException})?(\s+%{TRACE:Trace})?\s+%{MACHINE:ServiceMachine}

The extra patterns used are:  
TITLE ^\s_Title:.+  
MESSAGE ^\s_Message:.+  
IEXCEPTION ^\s_InnerException:.+  
TRACE ^\s_StackTrace:.+  
MACHINE ^\s\*Machine:.\S+

The multi-line filter applied is:  
^\Timestamp

Negate the multi-line regex:  
True

Can you please tell me, what is wrong in my grok pattern? And how 'InnerException' and 'Trace' will get created as separate fields, if present in log lines.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [June 8, 2016, 6:07pm UTC](https://discuss.elastic.co/t/optional-fields-in-grok-pattern-are-not-working/51854/2 "2016-06-08T18:07:28Z")

</div>

The problem is that `.+` is greedy. If it can gobble up your optional InnerException and Trace captures it will. Perhaps you can try to match everything up to a newline character instead? I'm not sure `\n` works here, but if it does I'd expect

```
(?m)Timestamp: %{DATESTAMP:Timestamp}\nTitle: %{TITLE:ServiceTitle}\nMessage: %{MESSAGE:ServiceMessage}\n(InnerException: %{IEXCEPTION:InnerException}\n)?(StackTrace: %{TRACE:Trace}\n)?Machine: %{MACHINE:ServiceMachine}

```

to work.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:54am UTC](https://discuss.elastic.co/t/optional-fields-in-grok-pattern-are-not-working/51854/3 "2017-07-06T04:54:07Z")

</div>