Just getting started with elastic this is my second attempt now that I have some free time I'm getting somewhere with it. Having some trouble finding the correct way to format my grok to correctly grab the fields.
Main Goals
I'd like to correctly format the date to iso8601
Grab the teamviewer id which varies from 9 to 10 digits "my ^\d{9.10}$ regex doesn't seem to work"
Here's some sample log data I'm looking at.
2019/05/02 07:38:05.431 904 5468 G1 CMainWindow::ShouldShowDynamicPassword(): dynPw=1 allowIncoming=1 id=0123456789 ka=0 lanAllowed=0 lanOnly=0 networkState=1 showDynPwd=
2019/05/02 07:38:06.134 2796 3772 S0!!!Connect to Master master5.teamviewer.com / 185.188.32.5 failed!
2019/05/02 07:38:06.134 2796 3772 S0!! CMasterConnectorAsio::HandleMasterResponseLogin(): MasterConnect failed. ErrorCode=10
2019/05/02 07:38:09.134 2796 3772 S0 Activating Router carrier
2019/05/02 10:51:00.083 2404 5904 D1!! CGrabMethodGDI::Grab: Fullscreen grab with BitBlt failed with code 0
2019/05/02 11:13:19.469 2156 2272 S0 Negotiating session encryption: client hello received from 123456789, RSA key length = 2048
2019/05/02 11:14:20.188 2156 2272 S0 CGatewaySession[44]::EndSession(): Session to 123456789 ended. Estimated capacity=83294kBit/s, Latency=1ms
2019/05/02 11:40:18.516 8740 3408 H64 explorer.exe: ResumeAllThreads: resumed 17 threads, max count 17
TiNhO
(Jaírton Fortuna)
May 2, 2019, 6:39pm
2
To the first line you can do something like this using grok filter:
%{YEAR:year}/%{MONTHNUM:Month}/%{MONTHDAY:Day} %{TIME:Time}%{GREEDYDATA}(?<id>id=%{NUMBER})
I do not know if you've seen https://grokdebug.herokuapp.com/
Check it out if you want better tuning your own filter
Alternatively
grok { match => { "message" => "^(?<[@metadata][timestamp]>.{23})\s+(?<id>\d{3,4} \d{4}) %{GREEDYDATA:restOfLine}" } }
date { match => [ "[@metadata][timestamp]", "YYYY/MM/dd HH:mm:ss.SSS" ] }
There are many ways to get what you want...
TiNhO:
To the first line you can do something like this using grok filter:
%{YEAR:year}/%{MONTHNUM:Month}/%{MONTHDAY:Day} %{TIME:Time}%{GREEDYDATA}(?<id>id=%{NUMBER})
I do not know if you've seen https://grokdebug.herokuapp.com/
Check it out if you want better tuning your own filter
Yeah I had something close to this problem is the log varies so much what works on one line does not on the other. I was using the online debugger but ended up finding the one in kibana under dev tools.
Got it close but I just need to figure out how to ignore part of the filter if it doesn't exist
%{YEAR:year}/%{MONTHNUM:month}/%{MONTHDAY:day}%{SPACE}%{TIME:time}%{SPACE}%{INFO:id1}%{SPACE}%{INFO:id2}%{SPACE}%{GREEDYDATA:messagestart}%{SPACE}%{GREEDYDATA}%{SPACE}(?<teamviewerid>%{TEAMVIEWER_ID})%{GREEDYDATA:messageend}
Custom pattern
TEAMVIEWER_ID \d{9,10}(?!0)
INFO \d{3,4}(?!0)
TiNhO
(Jaírton Fortuna)
May 3, 2019, 2:57pm
8
I guess you can use greedydata and leave as unnamed data. Because the default configuration only parse named data.
system
(system)
Closed
May 31, 2019, 2:57pm
9
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.