Add (;) sépartor in csv file before analysing it


(Ismael Boumedien) #1

Hi guys,

I want to add (:wink: sépartor in csv file before analysing it.

Example befor e:
2017/01/08-18:32:08mydomain.fr;151;ACC_EFFETMETEO_20170108181410.tar.gz;/arc/dacc/staging/max/SCORE;1
Result wanted:
2017/01/08-18:32:08;mydomain.fr;151;ACC_EFFETMETEO_20170108181410.tar.gz;/user/arc/staging/max/SCORE;1

Can you please give a tip to do it in logstash.

Best regards

Ismael


(Christian Dahlqvist) #2

Why not instead run a grok filter to capture the timestamp and put the rest of the log line into a separate variable and then apply the csv filter to this field?


(Ismael Boumedien) #3

Hi,

Is it authorized to grok before csv filter?

Best regards

Ismael


(Christian Dahlqvist) #4

You can run processing filters in any order you want. It is not uncommon to parse part of the message using one filter and then apply another to some of the fields.


(Ismael Boumedien) #5

Hi,

Thanks for your help. It's working. :wink:

Best regards

Ismael

--- For information
input {
file {
path => "/projects/elastic/logs/stream.log"
start_position => "beginning"
}
}
filter {
grok{
match => ["message", "(?\d{4}/\d{2}/\d{2}-\d{2}:\d{2}:\d{2})%{GREEDYDATA:serviceMessage}"]
}
csv {
source => "serviceMessage"
columns => ["host", "number", "filename", "path", "Status"]
separator => ";"
}
date {
locale => "en"
match => ["time", "YYYY/MM/dd-HH:mm:ss"]
timezone => "Europe/Paris"
add_tag => "checkpoint_fw"
}
mutate {
add_field => { "IndexType" => "hadoop_prod" }
}
if "_grokparsefailure" in [tags] {
drop {}
}
if "_csvparsefailure" in [tags] {
drop {}
}

}
output {
stdout { codec => rubydebug }
}


(system) #6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.