Split value in logstash

[youssef]

Hi all,

i have field called CONGESTION_LEVEL that may contain the values 0 , 1 , 2 , 0;N , 1;N , 2;N or may be empty, i want to add integer field that may contain 0 ,1 ,2 values:
if the CONGESTION_LEVEL field is 0 1 or 2 then no change on the value
if the CONGESTION_LEVEL field is 0:N 1;N or 2:N then i need to split it and stay with only the number
else the new field will contain the vaue 0
i am using Logstash 6.0

input {
file {
path => "/var/log/Optimize/reports/transactionsLog/transaction.log.402"
start_position => "beginning"
}
}
filter {
csv {
separator => " "
columns => ["START_TIME", "END_TIME","CONTENT_TYPE", "UPSTREAM_SIZE","DOWNSTREAM_SIZE","NUMBER_OF_STALLS", "CONGESTION_LEVEL"]
convert => {
"UPSTREAM_SIZE" => "integer"
"DOWNSTREAM_SIZE" => "integer"
"NUMBER_OF_STALLS" => "integer"
"AVERAGE_STALLS_LENGTH" => "integer"
"CONGESTION_LEVEL" => "integer"
"START_TIME" => "date"
"END_TIME" => "date"
}
}
}
filter{
if ([CONGESTION_LEVEL]) and (([CONGESTION_LEVEL])>-1) and ([CONGESTION_LEVEL]<3){
ruby {
code => "event.set('CONGESTION_LEVEL_NEW', event.get('CONGESTION_LEVEL'))"
}
}else if ";" in [CONGESTION_LEVEL]{
ruby {
code => "event.set('CONGESTION_LEVEL_NEW', event.get('CONGESTION_LEVEL')[0..1])"
}
}else{
ruby {
code => "event.set('CONGESTION_LEVEL_NEW', 0)"
}
}
}

output {
elasticsearch
{
hosts => ["localhost:9200"]
}
}

Regards,
Yoskaly

[magnusbaeck (Magnus Bäck)]

You can use the gsub option of a mutate filter to remove ";N" from the end of the string. You also need a conditional wrapping a mutate filter to set the field value to zero if the field is empty.

[paz]

Your logic seems fine, but you're making it a bit more convoluted than needed. This should work (since you're using inline Ruby anyway).

filter {
    ruby {
        code => "
            event.get('CONGESTION_LEVEL')[0].to_i < 3 ? event.set('CONGESTION_LEVEL', event.get('CONGESTION_LEVEL')[0].to_i) : 0
        "
    }
}