Timestamp - cant figure out

Elastic Stack Logstash
1

Hi ,
I have a web log entry like below:

JINK|Information|0|05-17-2017 16:00:44:56|[JNK]|GET|/Services/portal/v1/CheckToken|End|System.Web.Http.Action|ReflectedHttpActionDescriptor.ExecuteAsync()|Action returned 'True'|26

I am not sure what timestamp this represents. 05-17-2017 16:00:44:56. To me looks like millisecond in the end. How do i convert it to @timestamp ? Should i be using below construct ?

match => ["logtimestamp", "MM-dd-yyyy hh:mm:ss:SSS"]

2

Probably, but it's hard for us to know for sure what your log files mean. But use "HH" for the hours, not "hh".

3

Hi Magnus,

I am getting below error in elasticsearch log server on import:

Caused by: java.lang.IllegalArgumentException: Invalid format: "05-17-2017 16:10:09:76" is malformed at "-17-2017 16:10:09:76"
at org.joda.time.format.DateTimeParserBucket.doParseMillis(DateTimeParserBucket.java:187) ~[joda-time-2.9.5.jar:2.9.5]
at org.joda.time.format.DateTimeFormatter.parseMillis(DateTimeFormatter.java:826) ~[joda-time-2.9.5.jar:2.9.5]
at org.elasticsearch.index.mapper.DateFieldMapper$DateFieldType.parse(DateFieldMapper.java:243) ~[elasticsearch-5.4.0.jar:5.4.0]
at org.elasticsearch.index.mapper.DateFieldMapper.parseCreateField(DateFieldMapper.java:468) ~[elasticsearch-5.4.0.jar:5.4.0]

seems there is problem in date parser. I tried with HH as well, still same issue.

4

Works fine for me:

$ cat test.config 
input { stdin { } }
output { stdout { codec => rubydebug } }
filter {
  date {
    match => ["message", "MM-dd-yyyy HH:mm:ss:SSS"]
  }
}
$ echo '05-17-2017 16:00:44:56' | /opt/logstash/bin/logstash -f test.config
Settings: Default pipeline workers: 8
Pipeline main started
{
       "message" => "05-17-2017 16:00:44:56",
      "@version" => "1",
    "@timestamp" => "2017-05-17T14:00:44.560Z",
          "host" => "lnxolofon"
}
Pipeline main has been shutdown
stopping pipeline {:id=>"main"}

Just note that Logstash doesn't parse the milliseconds correctly. I'm not sure why.

5

Thanks for the test magnus.
Is this a known issue ?
Is there a way around or do you suggest dropping milliseconds ?

6

What version of Logstash are you using? Are you using a date filter like in the example Magnus provided? What does an event look like if you output it to stdout with a rubydebug codec?

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.