Converting string to date - getting parse error

Elastic Stack Logstash
1

Hi,

I would like to convert date in string to date in integer. In below example I would like format of "TS" to be similar to that of @timestamp

{
"type" => "perf",
.............
"@timestamp" => 2017-04-29T01:25:21.279Z,
.............
"TS" => "Fri Apr 28 16:16:29 2017"
}

My /etc/logstash/conf.d/logstash.conf has

input {
file {
path => "/var/www/html/processed/perf.csv"
type => "perf"
start_position => beginning
sincedb_path => "/dev/null"
ignore_older => 0
}
}

filter {
if [type] == "perf" {
csv {
separator => ","
columns => ['TS','CAPACITY']
convert => {
'CAPACITY' => 'integer'
}
}

    date {
        match => [ "TS", "dd/MMM/YYYY:HH:mm:ss Z" ]
    }
}

}

output {
if [type] == "perf" {
elasticsearch {
hosts => "localhost:9200"
action => "index"
index => "lstash-%{+YYYY.MM.dd}"
}
}
stdout { codec => rubydebug }
}

I am getting following parse error and also TS not converted. Tried multiple methods mentioned in various discussions nothing worked.

{
"type" => "perf",
..........
"tags" => [
[0] "_dateparsefailure"
],
"@timestamp" => 2017-04-29T01:30:53.806Z,
..........
"TS" => "Fri Apr 28 16:13:49 2017"
}

2

Hi,

Any suggestions to make this working?

-Raghu

3

Made it working. Since "TIMESTAMP" is "Fri Apr 28 16:16:29 2017"
Filter should have been:

    date {
        match => [ "TS", "EEE MMM dd HH:mm:ss YYYY" ]
        timezone => "UTC"
        remove_field => "timestamp"
    }

Nice documentation. Thanks

4

Does not match this at all;

You need to recheck that pattern so that it actually matches.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.