# Grok regex with escaped "\[", "\]", "(", and ")" chars doesn't work

**URL:** <https://discuss.elastic.co/t/grok-regex-with-escaped-and-chars-doesnt-work/93842>\
**Category:** Logstash\
**Created:** [July 19, 2017, 9:32pm UTC](https://discuss.elastic.co/t/grok-regex-with-escaped-and-chars-doesnt-work/93842 "2017-07-19T21:32:25Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![timk519](https://avatars.discourse-cdn.com/v4/letter/t/bcef8e/32.png) [@timk519](https://discuss.elastic.co/u/timk519)\
**Post date:** [July 19, 2017, 9:32pm UTC](https://discuss.elastic.co/t/grok-regex-with-escaped-and-chars-doesnt-work/93842/1 "2017-07-19T21:32:25Z")

</div>

Elastic newbie here - working with a new 5.5 install. I have a log line that looks like so:  
`[2015/10/01@19:48:22.785-0400] P-4780 T-2208 I DBUTIL : (451) prostrct create session begin for timk519 on CON:.`

I have the following regex:  
`\[%{DATE:date}@%{TIME:time}-(?<gmtoffset>\d{4})\]\s*(?<procid>P-[0-9]+)\s*(?<threadid>T-[0-9]+)\s*(?<msgtype>[ifIF])\s*(?<processtype>[a-zA-Z]+)\s*(?<usernumber>[0-9]+|[:])\s*\((?<msgnum>[0-9]+|[\-]+)\)\s*%{GREEDYDATA:message}`

When I try it in the kibana grok debugger it doesn't work and I get the following error:  
`GrokDebugger: [parse_exception] [pattern_definitions] property isn't a map, but of type [java.lang.String], with { header={ processor_type="grok" & property_name="pattern_definitions" } }`

This regex works _if_ I remove the [and] from the regex and the log line. I've tried doing single, double, and triple escape on the [and] - to no avail.

I was able to escape the () around the msgnum tag which leaves me puzzled why escaping the [] characters doesn't work.

What am I missing?

---

<div class="post-metadata">

**Author:** ![timk519](https://avatars.discourse-cdn.com/v4/letter/t/bcef8e/32.png) [@timk519](https://discuss.elastic.co/u/timk519)\
**Post date:** [July 19, 2017, 10:44pm UTC](https://discuss.elastic.co/t/grok-regex-with-escaped-and-chars-doesnt-work/93842/2 "2017-07-19T22:44:47Z")

</div>

This hack works -  
`%{DATE:date}@%{TIME:time}-(?<timezone>\d{4}).\s*(?<procid>P-[0-9]+)\s*(?<threadid>T-[0-9]+)\s*(?<msgtype>[ifIF])\s*(?<processtype>[a-zA-Z]+)\s*(?<usernumber>[0-9]+|[:])\s*\((?<msgnum>[0-9]+|[\-]+)\)%{GREEDYDATA:message}`

I added a "." after the {4}) to get past the ] - and it looks like so:

`%{DATE:date}@%{TIME:time}-(?<timezone>\d{4}).`

I'd prefer a solution which is explicit about the [and].

---

<div class="post-metadata">

**Author:** ![timk519](https://avatars.discourse-cdn.com/v4/letter/t/bcef8e/32.png) [@timk519](https://discuss.elastic.co/u/timk519)\
**Post date:** [July 20, 2017, 2:40pm UTC](https://discuss.elastic.co/t/grok-regex-with-escaped-and-chars-doesnt-work/93842/3 "2017-07-20T14:40:54Z")

</div>

This also works - add a "." as the leading char, and add a ] to capture the trailing ].

.%{DATE:date}@%{TIME:time}-(?\d{4})\]

I get the impression the regex processor doesn't like a leading [ or \[

Well isn't that interesting - the editor a interpreting a \ as an escape character and only showing the next character...

---

<div class="post-metadata">

**Author:** ![timk519](https://avatars.discourse-cdn.com/v4/letter/t/bcef8e/32.png) [@timk519](https://discuss.elastic.co/u/timk519)\
**Post date:** [July 20, 2017, 10:10pm UTC](https://discuss.elastic.co/t/grok-regex-with-escaped-and-chars-doesnt-work/93842/4 "2017-07-20T22:10:35Z")

</div>

This pattern works in grok debut test in kibana, yet fails in production:

```
.%{DATE:date}@%{TIME:time}-(?<timezone>\d{4}).\s*(?<procid>P-[0-9]+)\s*(?<threadid>T-[0-9]+)\s*(?<msgtype>[ifIF])\s*(?<processtype>[a-zA-Z]+)\s*(?<usernumber>[0-9]+|[:])\s*\((?<msgnum>[0-9]+|[\-]+)\)\s*%{GREEDYDATA:message}

```

The problem is the escaped "(" and ")" used to extract the msgnum number.

Anyone?

---

<div class="post-metadata">

**Author:** ![timk519](https://avatars.discourse-cdn.com/v4/letter/t/bcef8e/32.png) [@timk519](https://discuss.elastic.co/u/timk519)\
**Post date:** [July 31, 2017, 1:17pm UTC](https://discuss.elastic.co/t/grok-regex-with-escaped-and-chars-doesnt-work/93842/5 "2017-07-31T13:17:13Z")

</div>

turns out the {DATE} was the incorrect format in that it only had a 2-digit year while the log had a 4 digit year. Once that was changed to a custom format that matched the 4 digit year in the log date everything behaved normally.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 28, 2017, 1:17pm UTC](https://discuss.elastic.co/t/grok-regex-with-escaped-and-chars-doesnt-work/93842/6 "2017-08-28T13:17:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.