How to change data type for date field?

Elastic Stack Logstash
1

Hello,

Is there anyone around who got a 'date' field as 'string'? Anyway to change the data type to 'date'? I need something like this:

{
"data" : {
"properties" : {
"date": {
"type" : "date",
"format" : "yyyy-MM-dd HH:mm:ss"
},
"status": {"type" : "string"},
"group": {"type" : "string"},
"ip": {"type" : "string"},
"username":{"type" : "string"},
"category":{"type" : "string"},
"url":{"type" : "string"}
}
}
}

Unfortunately, my online searches did not help much. Below is an extract of the 'grok filter' I am using:

match => [ "message", "%{TIMESTAMP_ISO8601:logdate} %{DATA} %{WORD:status} %{USERNAME:group} %{DATA} %{IPV4:ip} %{DATA} %{USERNAME} %{SPACE} %{USERNAME:category} %{URI:url} %{WORD:method}" ]

Most probably, the 'TIMESTAMP' data type isn't correct? Any advice please.

Regards,

Ish

2

Use the date filter after your Grok. So...

date {
	match => [ "logdate", "ISO8601" ]
	target => "logdate"
}
1 Like
3

Hi,

Below is the config :

filter {
if [type] == "ufdb" {

            grok {
                    match => [  "message", "%{TIMESTAMP_ISO8601:logdate} %{DATA} %{WORD:status} %{DATA} %{USERNAME:group} %{DATA} %{IPV4:ip} %{DATA} %{USERNAME} %{DATA} %{USERNAME:category} %{DATA} %{URI:url} %{WORD:method}" ]

                    }


            grok {
                    match => [  "message", "%{TIMESTAMP_ISO8601:logdate} %{DATA} %{WORD:status} %{USERNAME:group} %{DATA} %{IPV4:ip} %{DATA} %{USERNAME} %{SPACE} %{USERNAME:category} %{URI:url} %{WORD:method}" ]
            }
            date {
      match => [ "logdate", "ISO8601" ]
      target => "logdate"
    }

}

below is the mapping that I get :

http://192.168.1.10:9200/logstash-2015.09.09/ufdb/_mapping

    }
      },
      "ip": {
        "type": "string",
        "norms": {
          "enabled": false
        },
        "fields": {
          "raw": {
            "type": "string",
            "index": "not_analyzed",
            "ignore_above": 256
          }
        }
      },
      "logdate": {
        "type": "string",      <------ "we need to set this type to date"
        "norms": {
          "enabled": false
        },
        "fields": {

The match don’t seem to work. Any ideas ?

4

I changed the 'match' as follows and it worked:

date {
      match => [ "logdate", "ISO8601", "YYYY-MM-dd HH:mm:ss" ]
      target => "logdate"
      locale => "en"
    }

The format was actually wrong.

1 Like
5

I have almost identical situation like you, but your solution did not solve my issue.