Grok match everything up to certain character sequence?

Elastic Stack Logstash
1

Hi guys,

does anybody know how to grok all text from log entry until certain character sequence?

For example, how to seperate only

SiteController::actionThankYou() Displaying random premium game.

from the following log entry:

[Wed Feb 22 18:09:30.705389 2017] [:error] [pid 28652] [client 192.168.10.111:53660] INFO: SiteController::actionThankYou() Displaying random premium game. [game id 3439] [Project Id: 32] [Project AdrenoGame Id: 32], referer: http://www.adrenogame.com/thank-you?aj=OTQzMDUxOTkwMaX934OiiCKghdfipgshjigshjgsdhjkfshiwFAW4G%2B5ftq68QBZVHpcbmMc6tmZ%2B%2FAxVqz51501mBQVrv4bY7ZWSkNHOpi%2BLs686IavsBkGOfTYUIvbfNLvY%2FRCP&c=

I have grok construction for the preceding data (I can post it if needed).

Thank you in advance

2

You can use %{GREEDYDATA} (i.e. .*) to match any character sequence.

3

@magnusbaeck Thank you for quick reply

%{GREEDYDATA} will select all text upon the end of the log entry. I need to select text only upon character sequence "[game id 3439]" (game id will be another field).

4

I've made following grok construction:

\[%{APACHE_ERROR_TIME:timestamp}\] \[:%{DATA:messagetype}\] \[pid %{NUMBER:pid}\] \[client %{IPV4:proxyaddr}:%{NUMBER:localport}\] %{LOGLEVEL:loglevel}: (?<action>.*\[g)

Applied to mentioned log entry, I got following:

{
  "timestamp": [
    [
      "Wed Feb 22 18:09:30.705389 2017"
    ]
  ],
  "DAY": [
    [
      "Wed"
    ]
  ],
  "MONTH": [
    [
      "Feb"
    ]
  ],
  "MONTHDAY": [
    [
      "22"
    ]
  ],
  "TIME": [
    [
      "18:09:30.705389"
    ]
  ],
  "HOUR": [
    [
      "18"
    ]
  ],
  "MINUTE": [
    [
      "09"
    ]
  ],
  "SECOND": [
    [
      "30.705389"
    ]
  ],
  "YEAR": [
    [
      "2017"
    ]
  ],
  "messagetype": [
    [
      "error"
    ]
  ],
  "pid": [
    [
      "28652"
    ]
  ],
  "BASE10NUM": [
    [
      "28652",
      "53660"
    ]
  ],
  "proxyaddr": [
    [
      "192.168.10.111"
    ]
  ],
  "localport": [
    [
      "53660"
    ]
  ],
  "loglevel": [
    [
      "INFO"
    ]
  ],
  "action": [
    [
      "SiteController::actionThankYou() Displaying random premium game. [g"
    ]
  ]
}

I just need to cut last 3 characters in action field - " [g"

I tried multiple combinations, without success...

5

\[%{APACHE_ERROR_TIME:timestamp}\] \[:%{DATA:messagetype}\] \[pid %{NUMBER:pid}\] \[client %{IPV4:proxyaddr}:%{NUMBER:localport}\] %{LOGLEVEL:loglevel}: %{DATA:Action}. \[%{DATA:gid}\] %{GREEDYDATA}

Try this!!

1 Like
6

Yes, you got it, it is working :slight_smile:

Just seperated dot and added another %{DATA} field. Great.

Thank you very much

7

%{GREEDYDATA} will select all text upon the end of the log entry.

No, that's not true. It matches any (possibly empty) character sequence.

8

Thank you @magnusbaeck and @riddhijit_roy

I now figured out how %{GREEDYDATA} works

:sunglasses:

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.