I'm just confused in using grok

Elastic Stack Logstash
1

Hello,

I'm beginner with grok and I'm trying to extract log patterns using match property. This what I do :

grok {
match => { "message" => "REQUEST\t%{WORD:senario_group}\t{NOTSPACE:user_id}\t%
{WORD:senario_name}\t%{INT:request_time}\t%{INT:response_time}\t%
{WORD:status}\t%{DATA:extras_infos}"
}
}

I would like to extract every pattern into specific field ( exemple : metting senario_group in a field , user_id in another field and etc ...)

I tried event.get('id_pattern') but il doesn't work.

I do a lot of research and I found others ways to do this but I'm very confused.

Can any one help me ?

2

What does your data look like?

3

Il is the Gatling simulation.log and this is part of it :

REQUEST GET A 25 GET A 1527154142238 1527154142455 OK
REQUEST GET A 29 GET A 1527154142242 1527154142457 OK
REQUEST GET A 21 GET A 1527154142228 1527154142458 OK
REQUEST POST A 4 POST A 1527154142186 1527154142417 OK
REQUEST GET C 14 GET C 1527154142206 1527154142416 OK
REQUEST POST A 10 POST A 1527154142192 1527154142466 OK
REQUEST POST A 6 POST A 1527154142188 1527154142467 OK
REQUEST GET C 18 GET C 1527154142211 1527154142469 OK
REQUEST POST A 2 POST A 1527154142176 1527154142470 OK
REQUEST PUT B 42 PUT B 1527154142273 1527154142471 OK
REQUEST GET A 22 GET A 1527154142235 1527154142473 OK
REQUEST PUT B 46 PUT B 1527154142290 1527154142474 OK
REQUEST GET A 30 GET A 1527154142243 1527154142476 OK
REQUEST GET A 26 GET A 1527154142239 1527154142477 OK
REQUEST PUT B 50 PUT B 1527154142294 1527154142481 OK
REQUEST GET X 38 GET X 1527154142269 1527154142482 OK
REQUEST GET Y 56 GET Y 1527154142300 1527154142482 OK
REQUEST GET X 34 GET X 1527154142247 1527154142483 OK
REQUEST GET C 20 GET C 1527154142218 1527154142483 OK
REQUEST GET A 28 GET A 1527154142241 1527154142484 OK
REQUEST GET Y 54 GET Y 1527154142298 1527154142484 OK
REQUEST GET Y 58 GET Y 1527154142308 1527154142485 OK
REQUEST GET C 12 GET C 1527154142194 1527154142485 OK
REQUEST POST A 8 POST A 1527154142190 1527154142486 OK
REQUEST PUT B 48 PUT B 1527154142292 1527154142488 OK
REQUEST PUT B 44 PUT B 1527154142275 1527154142489 OK
REQUEST GET A 24 GET A 1527154142237 1527154142490 OK
REQUEST GET C 16 GET C 1527154142209 1527154142491 OK
REQUEST GET X 40 GET X 1527154142271 1527154142493 OK
REQUEST GET Y 60 GET Y 1527154142311 1527154142494 OK
REQUEST GET Y 52 GET Y 1527154142296 1527154142495 OK
REQUEST POST A 9 POST A 1527154142191 1527154142497 OK
REQUEST GET C 17 GET C 1527154142210 1527154142499 OK
REQUEST POST A 5 POST A 1527154142187 1527154142500 OK
REQUEST GET C 13 GET C 1527154142196 1527154142501 OK
REQUEST PUT B 49 PUT B 1527154142293 1527154142502 OK
REQUEST PUT B 45 PUT B 1527154142289 1527154142503 OK
REQUEST GET X 37 GET X 1527154142268 1527154142504 OK
REQUEST GET Y 57 GET Y 1527154142307 1527154142506 OK
REQUEST GET C 15 GET C 1527154142208 1527154142461 OK
REQUEST GET C 11 GET C 1527154142193 1527154142508 OK
REQUEST POST A 7 POST A 1527154142189 1527154142509 OK
REQUEST POST A 3 POST A 1527154142180 1527154142511 OK
REQUEST GET A 27 GET A 1527154142240 1527154142512 OK
REQUEST GET Y 53 GET Y 1527154142297 1527154142507 OK
REQUEST GET A 23 GET A 1527154142236 1527154142517 OK
REQUEST GET X 33 GET X 1527154142246 1527154142519 OK
REQUEST PUT B 43 PUT B 1527154142274 1527154142520 OK
REQUEST GET X 31 GET X 1527154142244 1527154142522 OK
REQUEST GET X 35 GET X 1527154142266 1527154142523 OK
REQUEST GET X 39 GET X 1527154142270 1527154142524 OK
REQUEST GET X 36 GET X 1527154142267 1527154142496 OK
REQUEST GET X 32 GET X 1527154142245 1527154142532 OK
REQUEST PUT B 47 PUT B 1527154142291 1527154142538 OK
REQUEST GET Y 55 GET Y 1527154142299 1527154142540 OK
REQUEST GET Y 51 GET Y 1527154142295 1527154142541 OK
REQUEST GET Y 59 GET Y 1527154142310 1527154142542 OK

4

Have a look at https://www.elastic.co/blog/a-practical-introduction-to-logstash. Given that the format seems quite structured, it might be easier to use the dissect filter to parse it.

5

Thank you very much for your help, I'll try this :grinning:

6

for Grok : it works as Dissect, returns the patterns as fields or not ?

7

Did you read the blog post I linked to?

8

Dissect isn't based on REGEX it uses word position. Since your data follows a standard format DISSECT will work and probably be faster.

9

Yes I did and, it is like DISSECT for creating the fields. Thank you very much for your help :grinning:

10

Yes it's clear now, Thank you :grinning:

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.