Help with Grok pattern with space in field?

Elastic Stack Logstash
1

I have the following logs looks like:
query: s03.amazon.com IN A + (21.12.21.21)
query: s2.tabtest.com IN AAAA +++ (213.12.21.21)
query: t4.asaptest.com OUT TD OP- (21.124.21.21)

I need output to look like this:
hostname [s03.amazon.com, s2.tabtest.com, t4.asaptest.com]
level [A +, AAAA +++, TD OP-]
ip [21.12.21.21, 213.12.21.21, 21.124.21.21]

I have problems with "level" string, because it contains a space between data. How to write a working grok parser for that? Big thanks for any help.
I've tried something like:
query: %{NOTSPACE:hostname} IN %{NOTSPACE:level}+ %{NOTSPACE:level} %{GREEDYDATA:ip}
but it not works as I mentioned above.

2

Is that three separate log lines, or have you already combined them into one event? grok will only match a pattern once. If the pattern is repeated it will not return multiple matches. You can use a ruby filter to do that. Here is an example.

3

There are three separate logs, I added them just to show how it's look like, I know that grok will only match once.
My question is - how to add something like"AAA ++" (with space(s) between data) in one field, and additional task, how to grab ip from '()'
Thanks in advance!

4

If your messages have always this format, you do not need grok, you can parse it with dissect.

The following dissect filter parses this message format:

    dissect {
        mapping => {
            "message" => "query: %{hostname} %{} %{level} (%{ip})"
        }
    }

So, considering the examples you shared, you will have as output:

{
       "message" => "query: t4.asaptest.com OUT TD OP- (21.124.21.21)",
      "@version" => "1",
      "hostname" => "t4.asaptest.com",
         "level" => "TD OP-",
    "@timestamp" => 2021-12-13T21:10:15.846Z,
          "host" => "logstash",
            "ip" => "21.124.21.21"
}
{
       "message" => "query: s03.amazon.com IN A + (21.12.21.21)",
      "@version" => "1",
      "hostname" => "s03.amazon.com",
         "level" => "A +",
    "@timestamp" => 2021-12-13T21:10:15.834Z,
          "host" => "logstash",
            "ip" => "21.12.21.21"
}
{
       "message" => "query: s2.tabtest.com IN AAAA +++ (213.12.21.21)",
      "@version" => "1",
      "hostname" => "s2.tabtest.com",
         "level" => "AAAA +++",
    "@timestamp" => 2021-12-13T21:10:15.846Z,
          "host" => "logstash",
            "ip" => "213.12.21.21"
}
5

I agree that dissect is a better tool for this use case, but to answer the original question

grok { match => { "message" => "query: %{HOSTNAME:hostName} %{DATA:someField} \(%{IPV4:ip}\)" } }

would produce events like

 "someField" => "OUT TD OP-",
        "ip" => "21.124.21.21",
  "hostName" => "t4.asaptest.com",
6

Thanks @leandrojmp @Badger very much! Everything works fine, tried like @Badger posts with micro-reconfiguring:
query: %{HOSTNAME:hostName} IN %{DATA:level} (%{IPV4:ip})

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.