Logstash date filter won't parse my date field

Elastic Stack Logstash
1

Hi :smile:

TL;DR

logstash won't parse date field; date field parsed as string...

I'm trying to parse my date field using logstash date filter

but with no success...
looks like it is not override the @timestamp field and also when using other target field it is written as a string field and not a date field.

my input + filter looks like this:

input {
beats {
port => 5044
codec => "json"
}
}
filter {

date {
match => [ "time" , "ISO8601"]
#target => "parsed_date" # when using this the "parsed_date" field creates as type of string
}
}

and my log line (fired by filebeat) looks something like this:
{ "time":"2017-06-20T16:22:24.4233978Z","id":"025c4c1c65","type":"Error","message":"some message" }

when looking in logstash logs there is no errors at all relating to this plugin.
also there is nothing in the tag of the indexed document telling there was a parsing error or something like that.
am I missing something ?

thanks in advance for any help :slight_smile:

2

Please show what you get in ES for the sample log line above. Copy/paste from the JSON tab of Kibana's Discover view.

when using other target field it is written as a string field and not a date field.

That's expected. It's mostly ES that determines the type of fields.

3

Hi, here is the a document json:

  {
      "_index": "filebeat-***",
      "_type": "log",
      "_id": "AVzyzoBDxsbYSTAmagCY",
      "_version": 1,
      "_score": null,
      "_source": {
        "exception": {
          "stack": "****",
          "source": {
            "code": {
              "file": "*****",
              "line": 12,
              "column": 17
            },
            "method": "IfNull",
            "assembly": "LogAndThrow",
            "type": "***"
          },
          "type": "ServiceException",
          "message": "***"
        },
        "data": {
          "source_version": "4.11.0.0",
          "source_name": "****"
        },
        "offset": 815808416,
        "input_type": "log",
        "source": "Service.log",
        "type": "log",
        "message": "Uncaught exception",
        "tags": [
          "beats_input_codec_json_applied"
        ],
        "@timestamp": "2017-06-29T07:43:46.025Z",
        "@version": "1",
        "beat": {
          "hostname": "*****",
          "name": "*****",
          "version": "5.3.0"
        },
        "host": "****",
        "time": "2017-06-29T07:43:44.5908205Z",
        "id": "93136d271e"
      },
      "fields": {
        "@timestamp": [
          1498722226025
        ]
      },
      "sort": [
        1498722226025
      ]
    }

thanks for replying :slight_smile:

4

Hmm, this looks correct. I don't know what's going on.

5

image.png1052×152 5.21 KB

why the time field in middle does not match the left column and right column ?

6

OK I think I get it, Kibana adjust the timezone to the browser timezone does it ?

there is a way to change that ?

7

OK I think I get it, Kibana adjust the timezone to the browser timezone does it ?

Yes.

there is a way to change that ?

Yes, there's a Kibana setting for that .

8

Thanks it works

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.