Splitting based on different lines and generating multiple events

Elastic Stack Logstash
1

Say I have a file of this format:

a11
a21 b22 c23
a31
a41
a51 b52 c53

I.e. lines can be either:

  • aX
  • aX bY cZ

I want to generate:

  • A signle event for aX lines (emitting {a: aX, c: 0})
  • Two events for aX bY cZ lines (emitting {a: aX, c: 0} and {a: bY, c: Z})

How do I approach this?

2

I would start with

    if [message] !~ /[^ ]+ [^ ]+ [^ ]+/ {
        mutate { add_field => { "a" => "%{message}" "c" => "0" } }
        mutate { convert => { "c" => "integer" } }
    } else {
        grok { match => { "message" => "^%{WORD:[@metadata][first]} %{WORD:[@metadata][second]} %{WORD:[@metadata][third]}$" } }
        mutate { add_field => { "foo" => [ "%{[@metadata][first]} 0", "%{[@metadata][second]} %{[@metadata][third]}" ] } }
        split { field => "foo" }
    }

That will create stuff that looks like this

{
         "a" => "a41",
         "c" => 0,
"@timestamp" => 2019-07-08T23:27:28.313Z,
   "message" => "a41"
}
{
       "foo" => "a21 0",
"@timestamp" => 2019-07-08T23:27:28.313Z,
   "message" => "a21 b22 c23"
}
{
       "foo" => "b22 c23",
"@timestamp" => 2019-07-08T23:27:28.313Z,
   "message" => "a21 b22 c23"
}

You just need to add the grok and mutate+add_field to convert [foo] to the fields that you want. It's really ugly code, but I do not have time to write something prettier right now.

Where you write bN I hope you meant bY :slight_smile:

3

Thanks @Badger!

Where you write bN I hope you meant bY :slight_smile:

Ah, right, typo, I fixed in the q.

Let me look into what you suggested.

Is there a way to grok first for the relevant parts (i.e. have 3 separate groks for aX, bY and cY, where the last two should be optional), put everything in an event (i.e. have an event that has aX, bY and cY as fields) and then clone based on the resulting event?

4

I am sure there are many ways to do it.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.