Using Ruby to count the number of fields in a message, then apply CSV depending on the count?

Elastic Stack Logstash
1

I have logs coming from dozens of servers, and two different appliance versions, both of which are in comma-separated format.

In one version, there are 73 fields; in the other, there are 93 fields. I need to be able to count the number of fields in each message, then apply the CSV filter+column names depending on the count. My thinking would be to split the message by commas, then counting the fields.

How would this be accomplished using Ruby? Something like this:

filter {
ruby {
#Pseudo code
code => "if count of event.['message'].split(",") == 73: apply these column names;
else if count of event.['message'].split(",") == 93: apply these column names"
}
}

2

You can count the fields using ruby

ruby { code => 'event.set("[@metadata][fields]", 1 + event.get("message").count(","))' }
if [@metadata][fields] == 73 {
     csv { ... }
} else {
     csv { ... }
}
2 Likes
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.