As per field condition should change fields

Elastic Stack Logstash
1

Hi,

I have csv file, as per field value next fields should be change.

i.e Field name : ABC

ABC field have different values: x, y, z

if ABC=x value next fields are A1,A2,A3,A4,A5... which should be combine in new field called EFG
if ABC =y value next fields are B1,B2,B3... which should be combine in new field called JKL
if ABC =z value next fields are C1,C2,C3C4, C5,C6,C7,C8... which should be combine in new field called LMN

Please suggest how should I configure my config file in Logstash.

Thanks!

2

Use a conditional to peek at the beginning of the message, then choose different csv filters.

if [message] =~ /^x,/ {
  csv {
    columns => ["ABC", "A1", "A2", ...]
    ...
  }
} else if [message] =~ /^y,/ {
  csv {
    columns => ["ABC", "B1", "B2", ...]
    ...
  }
}
...
3

Hi,

My sample log file is like :

100,bbb,a,,,,{REGISTER,,0,0,68e805.ucc,kjj,0,-3,IEEE-802,,,0,[],}
300,p19w,c,,,,{INVITE,,[sip:U-f1,2.0+13,2.01,20,2.0,IEEE-802,,0,[[NA,1,59:15.4,982,88,0,0,215,93,0],[NA,1,59:15.7,47:00.4,0,113,0,0,1051,93,0]],xyz}

Based on 7 field value {REGISTER / {INVITE next fields should be change

As you suggested I have created config file:

input {
file {
path => [ "/xyz/sample.csv" ]
start_position => "beginning"
type => "abc"
sincedb_path => "/dev/null"
ignore_older => 0
}

}
filter {

if [message] =~ /^{REGISTER,/ {
csv {
columns => ["x1","x2","x3","x4","x5","x6"]

}
}
else if [message] =~ /^{INVITE,/ {
csv {
columns => ["x1","x2","x3","x4","x5",x6","x7","x8","x9","x10"]

}
}

}
output {
elasticsearch {

            hosts => ["x.x.x.x:9200"]
            index => "test-%{+YYYY.MM}"

    }

    stdout { codec => rubydebug }

}

Not getting any error but unable to parse logs. Its taking time and not getting any output:

[2017-06-27T14:04:33,294][INFO ][logstash.pipeline ] Pipeline main started
[2017-06-27T14:04:33,326][INFO ][logstash.agent ] Successfully started Logstash API endpoint {:port=>9600}

Please suggest...

Thanks!

4

if [message] =~ /^{REGISTER,/ {

The line doesn't begin with {REGISTER so drop the "^".

But you should receive output anyway. Comment out the elasticsearch output for now and focus on getting stuff from the stdout output.