How to find last element of array in logstash

Elastic Stack Logstash
1

Hi, I am trying to split string and add a new event with last index as value.
Here below is code snippet which I am trying.

Could someone please help me out on the same.
Input : eqdomain/eqlogs/abc/hyper.log

Script:

input {
 stdin {}	
}
filter{
    grok{
        match => {"message" => "%{GREEDYDATA:originalMsg}"}
    }
	mutate {
		split => { "message" => "/" }
	}
	ruby {
		code => 'event.set("count",event.get("message").length)'
	}
	mutate {
		add_field => [ "newMessage" , "%{[message]["count"]}" ]
	}		
}	
 output {
		stdout {}		
}

Here in add_field I am trying to add a field with last element, this part I am not able to figure out.

Thanks,
Rakesh

2

Two things that stood out to me:

  • You're trying to use count as the last array index, but it isn't. Array indices start at zero, so the last index is the array length minus one.
  • You can't use the field in your add_field like this. This kind of syntax to use one field as the index to select something from another in those placeholders just doesn't exist in Logstash. You can only solve that with Ruby.

The following code should give you what you want:
event.set("newMessage", event.get("message")[-1])

Edit: Just two more comments: If you don't need the rest of the array, you could forego the split filter and do it all with Ruby as event.get("message").split('/')[-1]. And what purpose does that grok filter have? It seems unnecessary?

1 Like
3

Yes Agreed, just trying out with GROK to map and add additional events.

And Thanks Jenni for quick reply and solution provided is working as expected.

As well could you please suggest why "\r" is getting appended to newMessage field value.

{
    "newMessage" => "abc.log\r",
          "host" => "723L",
      "@version" => "1",
    "@timestamp" => 2020-07-30T20:07:44.576Z,
       "message" => [
        [0] "D:",
        [1] "Oracle",
        [2] "Middleware",
        [3] "Oracle_Home",
        [4] "user_projects",
        [5] "domains",
        [6] "eqdomain",
        [7] "eqlogs",
        [8] "ssasti",
        [9] "abc.log\r"
    ]
}
4

That seems to be the rest of a windows line break (CRLF: \r\n) . Your original message was split from the rest of a string at \n (Unix line break LF) and the Carriage Return \r stayed. I don't know why or where that happened. But you can probably get rid of it with mutate with the strip or gsub option.

5

Okay @Jenni , I will give a try on the same.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.