Split, get the order

Elastic Stack Logstash
1

Hi

I'm successfully splitting a json with array into several events using "split". What I'm looking for is a way to enumerate the events from the order in the array.

Simplified example of the problem:

Original message:
{"Messages":[
"Message":"Hello",
"Message":"and",
"Message":"Goodbye"
]}

What I want:
{
"Message": "Hello",
"MessageNumber": "1"
}
{
"Message": "and",
"MessageNumber": "2"
}
{
"Message": "Goodbye",
"MessageNumber": "3"
}

Thanks in advance

2

I was a little irritated by your example because the key "Message" is repeated multiple times in the original array, so my test file is a little different. But I guess you'll get the idea and be able to adjust it to fit your actual data. I'm adding the numbers before splitting the event and then I just assign them to the right field afterwards.

input {
  stdin{}
}
filter {
  mutate { replace => {"message" => "Hello"}}
  mutate { add_field => {"message" => "and"}}
  mutate { add_field => {"message" => "Goodbye"}}
  # Now my test data is ["Hello","and","Goodbye"]
  ruby {
    code => "
      i = 0
      while i < event.get('message').length do
        event.set('[message]['+i.to_s+']', [i+1,event.get('message')[i]])
        i += 1
      end
    "
  }
  # Now it's [[1,"Hello"],[2,"and"],[3,"Goodbye"]]
  split {
    field => "message"
  }
  # Now there are multiple events with "message" => [2, "and"] etc.
  mutate { copy => {"[message][0]" => "MessageNumber"}}
  mutate { add_field => {"Message" => "%{[message][1]}"} }
  mutate { remove_field => ["message"] }
  # Now everything is where it should be
}
output { stdout { codec => rubydebug } }
3

Thanks
I didn’t really understand the i.to_s but will try it tomorrow.
Sorry about the Json-missmatch, can be when I tried to simplify the complex original data to an example.
/Karl

4

I was just trying to create the keys "[message][0]", "[message][1]", etc. there by concatenation. Ruby likes to complain about type conversions, so I had to do an explicit conversion from Integer to String to include my counter variable.

5

New to Ruby so the syntax were a bit confusing, now I understand the formula and it seems to work.

Thank you for helping
/Karl

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.