Date parse error

Elastic Stack Logstash
1

Hi guys,

i'm trying to parse a date like this one : 2017-04-29 00:00:00 +0200
I'm using this date filter : filter { date { match => [ "txechecczhier_date", "yyyy-MM-dd HH:mm:ss Z" ] } }
I'm getting a result like this {"txechecczhier_date":"2017-04-29 00:00:00 +0200","@version":"1","@timestamp":"2017-05-02T13:08:01.755Z","Name":"echec_cz_hier","tags":["_dateparsefailure"]}
What's wrong? could you say me why i'm getting a dateparsefailure please?

Thank you

2

Use YYYY for year instead of yyyy.
And maybe Z option does not work well. Put +0200 instead if you've only got one timezone.

But can you try this? (not sure about the space between time and timezone, but worth testing)

date {
  match => ["txechecczhier_date", "ISO8601"]
}
3

Hi Nico,

I tried it but i'm always getting this error message (in all cases) :
Failed parsing date from field {:field=>"txechecczhier_date, : value=>#<Date: 2017-04-29 ((2457873j,0s,0n),0s,2299161j)), :exception=>"cannot convert instance of class org.jruby.RubyoObject to class java.lang.string", :config_parsers=>"ISO8601", : config_locale=>"default=fr_FR", :level=>:warn}

4

Ok, then, there's something that I have done when I had kinda the same problem.
I used to have a timestamp like this:
20170420 10:02:04,876 +0000

So I created the following pattern:
F_TIMESTAMP %{YEAR:year}%{MONTHNUM:month}%{MONTHDAY:monthday}\s*%{TIME:time}\s*%{ISO8601_TIMEZONE:timezone}

So now, I don't have a timestamp in my document (which would be txechecczhier_date for you)
Then, I do:

add_field => {
  "timestamp" => "%{year}-%{month}-%{monthday}T%{time}%{timezone}"
}

And now I can use

date {
  match => ["timestamp", "ISO8601"]
  target => "@timestamp"
}

Clunky as hell, but it works

1 Like
5

So if i understand, i have to create a grok pattern too? bacause in my case i need the txechecczhier_date and also the @timestamp which have differents values

6

No, you don't have to. It's just something that I have done to kinda avoid this date problem.
(You can keep both your timestamp, in my case I override it because I don't need it)

The problem might come from the timezone. Try to send some dummy log to see if the part without the timezone is correctly matched

All patterns are defined here: http://joda-time.sourceforge.net/apidocs/org/joda/time/format/DateTimeFormat.html

1 Like
7

I used stdin and send some data and i have this result :
{"message":"2017-04-29 00:00:00 +0200\r","@version":"1","@timestamp":"2017-05-02T14:01:02.506Z","host":"HOME"} {"message":"2017-03-29 15:00:00 +0500\r","@version":"1","@timestamp":"2017-05-02T14:02:21.641Z","host":"HOME"} {"message":"2017-03-29 15:01:00 +0500\r","@version":"1","@timestamp":"2017-05-02T14:02:31.201Z","host":"HOME"}

And i'm getting this with the filter :
{"message":"2017-04-29 00:00:00\r","@version":"1","@timestamp":"2017-05-02T14:06:36.545Z","HOME":"CD001603","tags":["_dateparsefailure"]} {"message":"2017-04-29 00:00:00\r","@version":"1","@timestamp":"2017-05-02T14:06:51.880Z","host":"HOME","tags":["_dateparsefailure"]}

8

Try to just send 2017-04-29 00:00:00 (no timezone) and modify your date filter to match this date, and see what happens

Btw, use

stdout {
  codec => rubydebug {} 
}

to see if there is any relevant stacktrace

9

error.png1893×242 5.29 KB

I'm getting this error without timezone.
My code is like this :

10

there's a problem with the \r (probably waiting a \n instead).
Maybe use a file input with this line and a carriage return

input{
  file{
    path => "C:/dummy.log"
    start_position => "beginning"
    sincedb_path => "NUL"
  }
}
11

I resolved the problem of line breaks think to this filter
mutate { strip => "message" }

I'll try to use this with my first code

12

cannot convert instance of class org.jruby.RubyoObject to class java.lang.string

This indicates that the problem is that the field you're trying to parse already is a timestamp. I recently reported this in an issue (Date filter fails to parse timestamps · Issue #95 · logstash-plugins/logstash-filter-date · GitHub). As a workaround you can use the mutate filter's convert option to turn the field into a string.

1 Like
13

Thank you Magnus, I found my hapiness with the mutate filter

14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.