Logstash GROK filter for tomcat logs

Elastic Stack Logstash
1

Hello elk team ,

i am new to elk , i need pattern that can find " transactionid following value from tomcat logs and create separate field

sample logs are like below ...

2018-03-14 10:58:36,853 INFO so:165 - Female Value : 0.084370888769626617 for transactionId ABCsdf62969
2018-03-14 10:58:36,853 INFO so:165 - White Value : 0.90355902910232544 for transactionId ABtgF62969
2018-03-14 10:58:36,853 INFO so:165 - Black Value : 0.001742142834700644 for transactionId ZBCBfg2969
2018-03-14 10:58:36,853 INFO so:165 - Asian Value : 0.0055485325865447521 for transactionId TBCBF62969
2018-03-14 10:58:36,853 INFO so:165 - Hispanic Value : 0.079676181077957153 for transactionId L45BF62969
2018-03-14 10:58:36,853 INFO so:165 - Other Value : 0.0094741648063063622 for transactionId A56BF62969
2018-03-14 10:58:36,853 INFO so:165 - Flattened size : 1764 bytes for transactionId AB67F62969
2018-03-14 10:58:36,853 INFO so:165 - FaceTemplateCreation for Transaction Id : ABCBF62969, Processed in : 303

2

What have you tried? What did it do? How did it vary from your expectations?

The grok constructor may be a good place to start :slight_smile:

3

Thank for quick reply i tried below regex but didn't work
filter {
grok {
match => { "message" => {%{TIMESTAMP_ISO8601:time} %{LOGLEVEL:log_level} %{GREEDYDATA:some_data} transactionId %{WORD:transaction_id}} }

}

but it didn't work , grok filter parsing failed .message grtting in elk logs

4

When I pasted your pattern and the example lines in the Grok Constructor, you can see that many of the messages match :tada:

However, it looks like your format for logging the transaction id is inconsistent; you'll need to adjust your pattern to capture either:

transactionId TBCBF62969
Transaction Id : ABCBF62969

I'd advise using the above-linked constructor to keep honing your pattern until it matches all of the example input.