# Grok pattern is not working

**URL:** <https://discuss.elastic.co/t/grok-pattern-is-not-working/65976>\
**Category:** Logstash\
**Created:** [November 14, 2016, 12:41pm UTC](https://discuss.elastic.co/t/grok-pattern-is-not-working/65976 "2016-11-14T12:41:34Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ramesh\_Janagam](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ramesh_janagam/32/11718_2.png) [@Ramesh\_Janagam](https://discuss.elastic.co/u/Ramesh_Janagam)\
**Post date:** [November 14, 2016, 12:41pm UTC](https://discuss.elastic.co/t/grok-pattern-is-not-working/65976/1 "2016-11-14T12:41:34Z")

</div>

Hi All,

I need help on grok patterns. I have a log file where few fields are missing(several fields actually), based on that I have come up with below grok patterns but its not working. In the grok contructor - it showing as matched but most fields are showing blank.

for example AlgoString in first line & Client are showing blank. Please anyone help on this.

logs lines:

```
07:11:02.002015|INF|RAW(0x2590ed0)|=> Received OrderNew={ Transaction=574F5D76-4BD-3-65 AlgoString=TEST:0:0:50:50 ClOrdId=-1 Client=TEST1
10:30:20.790316|INF|RAW(0x2b77e400ab70)|=> Received OrderNew={ Transaction=574F8C2C-C0E63-7456-65 Price=2121 ClOrdId=1 Client=TEST2
09:11:36.682557|INF|RAW(0x307fbc0)|=> Received OrderNew={ Transaction=574F79B8-A6943-91F-65 Price=664 ClOrdId=32 

```

pattern:

```
%{TIME:logtime}\|%{DATA:severity}\|.*\|.*Received %{DATA:TransType}={ Tran.*(?: AlgoString=%{DATA:AlgoString}|)?(?: Price=%{NUMBER:Price}|)? Cl.* (?: Client=%{DATA:Client}|)?(?:Account=%{DATA:Account}|)?.*
```

---

<div class="post-metadata">

**Author:** ![eperry](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eperry/32/551_2.png) [@eperry](https://discuss.elastic.co/u/eperry)\
**Post date:** [November 14, 2016, 4:45pm UTC](https://discuss.elastic.co/t/grok-pattern-is-not-working/65976/2 "2016-11-14T16:45:21Z")

</div>

> [@Ramesh\_Janagam](#):
>
> 07:11:02.002015|INF|RAW(0x2590ed0)|=\> Received OrderNew={ Transaction=574F5D76-4BD-3-65 AlgoString=TEST:0:0:50:50 ClOrdId=-1 Client=TEST1

First  
I would take your query to [https://grokdebug.herokuapp.com/](https://grokdebug.herokuapp.com/) and test it out

next, there is a "space" before your Client= which is throwing off the query

Next, you are using a lot of greedy statements .\* I would replace these with .+ or better yet actual data. \S is nice cause it matches everything not a space. .\* has some wierd effects and sometimes matches more then you expect.

Finally, Maybe you don't want to mach the json data with Grok and use the json filter. Something like this would work nicely (oh wait this is not json. Well try the KV filter

\*See if something like this helps, Sorry I have some typo's but you should get the jist of what I was doing with it.

```auto
filter{
   #get message and create new field with json like data
 grok => ["message", "^%{TIME:logtime}\|%{DATA:severity}\|(|.+)\|.+Received %{DATA:TransType}=%{GREEDYDATA:app_data}"
}
kv{
   source =>"app_data"
}
}

```

Of course if you do have valid json you can replace kv with the json filter and parsing would be automagic

---

<div class="post-metadata">

**Author:** ![Ramesh\_Janagam](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ramesh_janagam/32/11718_2.png) [@Ramesh\_Janagam](https://discuss.elastic.co/u/Ramesh_Janagam)\
**Post date:** [November 15, 2016, 9:09am UTC](https://discuss.elastic.co/t/grok-pattern-is-not-working/65976/3 "2016-11-15T09:09:52Z")

</div>

Thanks for your reply but I am trying below filter but I donot see any outpu. Any issue withbelow filter condition?

```
      grok {
              match => ["message", "^%{TIME:logtime}\|%{DATA:severity}\|(|.+)\|.+Received %{DATA:TransType}=%{GREEDYDATA:app_data}"]
              add_field => ["received_at" , "%{@timestamp}"]
              add_field => ["received_from" , "%{host}"]
              add_field => ["application" , "SOR"]
      }
	  
      kv {
	       source => "app_data"
	  }
      mutate {
              gsub => ["logtime","\d\d\d$",""]
      }
      date {
            locale => "en"

            match => ["logtime" , "HH:mm:ss.SSS"]
            target => "@timestamp"
            timezone => "Asia/Kolkata"
            add_field => { "debug" => "timestampMatched"}
      }
```

---

<div class="post-metadata">

**Author:** ![eperry](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eperry/32/551_2.png) [@eperry](https://discuss.elastic.co/u/eperry)\
**Post date:** [November 16, 2016, 1:01pm UTC](https://discuss.elastic.co/t/grok-pattern-is-not-working/65976/4 "2016-11-16T13:01:06Z")

</div>

Well the regex is working (using the first input of your first post

I put in at Grokdebugger and got this as a result.

```auto
{
  "logtime": [
    [
      "07:11:02.002015"
    ]
  ],
  "HOUR": [
    [
      "07"
    ]
  ],
  "MINUTE": [
    [
      "11"
    ]
  ],
  "SECOND": [
    [
      "02.002015"
    ]
  ],
  "severity": [
    [
      "INF"
    ]
  ],
  "TransType": [
    [
      "OrderNew"
    ]
  ],
  "app_data": [
    [
      "{ Transaction=574F5D76-4BD-3-65 AlgoString=TEST:0:0:50:50 ClOrdId=-1 Client=TEST1"
    ]
  ]
}

```

Can you provide what output you are receiving

```auto
output{ stdout{ codec=>rubydebug}}

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 14, 2016, 1:01pm UTC](https://discuss.elastic.co/t/grok-pattern-is-not-working/65976/5 "2016-12-14T13:01:19Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.