Logstash SPLIT plugin help

Elastic Stack Logstash
#1

Hi, I would like to ask thing about Split plugin.
I have event like:
{"name":"JASON",
"phones":[{
"mark":"Nokia",
"model":"3310"},{"mark":"Apple","model":"12 Pro"}]}

I use clone, and want object like:
{"name":"JASON",
"phonesCount":2}

and from other cloned event receive with use of SPLIT two events, lets say like:
{"name":"JASON",
"phoneIndex":0,
"phoneMark":"Nokia",
"phoneModel":"3310"}

and

{"name":"JASON",
"phoneIndex":1,
"phoneMark":"Apple",
"phoneModel":"12 Pro"}

is there any way how to achieve this? with combination of CLONE, SPLIT, MUTATE,
or
RUBY? I am RUBY analphabet but I can read the code of course...

thanks guys upfront

#2

Can you post what the finished data set you would like looks like?

#3

Hi, as I wrote, source is:

{
   "name":"JASON",
   "phones":[
      {
         "mark":"Nokia",
         "model":"3310"
      },
      {
         "mark":"Apple",
         "model":"12 Pro"
      }
   ]
}

and

one expected output is:

{
   "name":"JASON",
   "phonesCount":2
}

and

two another like this in form of two events:
event 1:

{
   "name":"JASON",
   "phoneIndex":0,
   "phoneMark":"Nokia",
   "phoneModel":"3310"
}

and

event two:

{
   "name":"JASON",
   "phoneIndex":1,
   "phoneMark":"Apple",
   "phoneModel":"12 Pro"
}

I know I have to clone original object, but than with first one make just agregation and on the second Split, but how to enrich the output events with element order index from original phone array?

I found solution with rubby, maybe, but Rubby is not my weapon of choice

thanks anyway

#4

Got it. Didn't know if you wanted to combine all events into one before the output.

But just so you know I haven't found a way to do this without Ruby yet. Going to keep looking some.

#5

Realy thanks, I am c# backend dev so Rubby way is way to go, but ZeroCode solution would be nicer.
Anyway, thanks

#6

I cannot think of any way to get the count without using ruby. If you know the upper limit of how many phones someone can have you can do the rest without ruby.

clone { clones => [ "count" ] }
if [type] == "count" {
    ruby { code => 'event.set("phonesCount", event.get("phones").length)' }
    mutate {
        convert => { "phonesCount" => "integer" }
        remove_field => [ "type", "phones" ]
    }
} else {
    mutate {
        add_field => {
            "[phones][0][phoneIndex]" => 0
            "[phones][1][phoneIndex]" => 1
            "[phones][2][phoneIndex]" => 2
            "[phones][3][phoneIndex]" => 3
        }
    }
    mutate {
        convert => {
            "[phones][0][phoneIndex]" => "integer"
            "[phones][1][phoneIndex]" => "integer"
            "[phones][2][phoneIndex]" => "integer"
            "[phones][3][phoneIndex]" => "integer"
        }
    }
    split { field => "phones" }
    if ! [phones][mark] { drop {} }
    mutate {
        rename => {
            "[phones][phoneIndex]" => "phoneIndex"
            "[phones][mark]" => "phoneMark"
            "[phones][model]" => "phoneModel"
        }
        remove_field => [ "phones" ]
    }
}

However, I think it is much easier to add the index using ruby instead of those two mutate filters, and then the else branch becomes

} else {
    ruby { code => 'event.get("phones").each_index { |x| event.set("[phones][#{x}][phoneIndex]", x) }' }
    split { field => "phones" }
    mutate {
        rename => {
        ...
1 Like