# Grok formatting help

**URL:** <https://discuss.elastic.co/t/grok-formatting-help/179406>\
**Category:** Logstash\
**Created:** [May 2, 2019, 4:26pm UTC](https://discuss.elastic.co/t/grok-formatting-help/179406 "2019-05-02T16:26:18Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![JB\_Pollard](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jb_pollard/32/45345_2.png) [@JB\_Pollard](https://discuss.elastic.co/u/JB_Pollard)\
**Post date:** [May 2, 2019, 4:26pm UTC](https://discuss.elastic.co/t/grok-formatting-help/179406/1 "2019-05-02T16:26:18Z")

</div>

Just getting started with elastic this is my second attempt now that I have some free time I'm getting somewhere with it. Having some trouble finding the correct way to format my grok to correctly grab the fields.

Main Goals  
I'd like to correctly format the date to iso8601  
Grab the teamviewer id which varies from 9 to 10 digits "my ^\d{9.10}$ regex doesn't seem to work"

Here's some sample log data I'm looking at.

```
2019/05/02 07:38:05.431 904 5468 G1 CMainWindow::ShouldShowDynamicPassword(): dynPw=1 allowIncoming=1 id=0123456789 ka=0 lanAllowed=0 lanOnly=0 networkState=1 showDynPwd=
2019/05/02 07:38:06.134 2796 3772 S0!!!Connect to Master master5.teamviewer.com / 185.188.32.5 failed!
2019/05/02 07:38:06.134 2796 3772 S0!! CMasterConnectorAsio::HandleMasterResponseLogin(): MasterConnect failed. ErrorCode=10
2019/05/02 07:38:09.134 2796 3772 S0 Activating Router carrier
2019/05/02 10:51:00.083 2404 5904 D1!! CGrabMethodGDI::Grab: Fullscreen grab with BitBlt failed with code 0
2019/05/02 11:13:19.469 2156 2272 S0 Negotiating session encryption: client hello received from 123456789, RSA key length = 2048
2019/05/02 11:14:20.188 2156 2272 S0 CGatewaySession[44]::EndSession(): Session to 123456789 ended. Estimated capacity=83294kBit/s, Latency=1ms
2019/05/02 11:40:18.516 8740 3408 H64 explorer.exe: ResumeAllThreads: resumed 17 threads, max count 17
```

---

<div class="post-metadata">

**Author:** ![TiNhO](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tinho/32/41986_2.png) [@TiNhO](https://discuss.elastic.co/u/TiNhO)\
**Post date:** [May 2, 2019, 6:39pm UTC](https://discuss.elastic.co/t/grok-formatting-help/179406/2 "2019-05-02T18:39:49Z")

</div>

To the first line you can do something like this using grok filter:

```
%{YEAR:year}/%{MONTHNUM:Month}/%{MONTHDAY:Day} %{TIME:Time}%{GREEDYDATA}(?<id>id=%{NUMBER})

```

I do not know if you've seen [https://grokdebug.herokuapp.com/](https://grokdebug.herokuapp.com/)  
Check it out if you want better tuning your own filter

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [May 2, 2019, 6:46pm UTC](https://discuss.elastic.co/t/grok-formatting-help/179406/3 "2019-05-02T18:46:57Z")

</div>

Alternatively

```
    grok { match => { "message" => "^(?<[@metadata][timestamp]>.{23})\s+(?<id>\d{3,4} \d{4}) %{GREEDYDATA:restOfLine}" } }
    date { match => ["[@metadata][timestamp]", "YYYY/MM/dd HH:mm:ss.SSS" ] }

```

There are many ways to get what you want...

---

<div class="post-metadata">

**Author:** ![JB\_Pollard](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jb_pollard/32/45345_2.png) [@JB\_Pollard](https://discuss.elastic.co/u/JB_Pollard)\
**Post date:** [May 3, 2019, 12:05pm UTC](https://discuss.elastic.co/t/grok-formatting-help/179406/6 "2019-05-03T12:05:01Z")

</div>

> [@TiNhO](#):
>
> To the first line you can do something like this using grok filter:
> 
> ```
> %{YEAR:year}/%{MONTHNUM:Month}/%{MONTHDAY:Day} %{TIME:Time}%{GREEDYDATA}(?<id>id=%{NUMBER})
> 
> ```
> 
> I do not know if you've seen [https://grokdebug.herokuapp.com/](https://grokdebug.herokuapp.com/)  
> Check it out if you want better tuning your own filter

Yeah I had something close to this problem is the log varies so much what works on one line does not on the other. I was using the online debugger but ended up finding the one in kibana under dev tools.

---

<div class="post-metadata">

**Author:** ![JB\_Pollard](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jb_pollard/32/45345_2.png) [@JB\_Pollard](https://discuss.elastic.co/u/JB_Pollard)\
**Post date:** [May 3, 2019, 2:29pm UTC](https://discuss.elastic.co/t/grok-formatting-help/179406/7 "2019-05-03T14:29:47Z")

</div>

Got it close but I just need to figure out how to ignore part of the filter if it doesn't exist

`%{YEAR:year}/%{MONTHNUM:month}/%{MONTHDAY:day}%{SPACE}%{TIME:time}%{SPACE}%{INFO:id1}%{SPACE}%{INFO:id2}%{SPACE}%{GREEDYDATA:messagestart}%{SPACE}%{GREEDYDATA}%{SPACE}(?<teamviewerid>%{TEAMVIEWER_ID})%{GREEDYDATA:messageend}`

Custom pattern  
TEAMVIEWER\_ID \d{9,10}(?!0)  
INFO \d{3,4}(?!0)

---

<div class="post-metadata">

**Author:** ![TiNhO](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/tinho/32/41986_2.png) [@TiNhO](https://discuss.elastic.co/u/TiNhO)\
**Post date:** [May 3, 2019, 2:57pm UTC](https://discuss.elastic.co/t/grok-formatting-help/179406/8 "2019-05-03T14:57:06Z")

</div>

I guess you can use greedydata and leave as unnamed data. Because the default configuration only parse named data.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 31, 2019, 2:57pm UTC](https://discuss.elastic.co/t/grok-formatting-help/179406/9 "2019-05-31T14:57:14Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.