Failed to parse date field yyyy/MM/dd HH:mm:ss [SOLVED]

Elastic Stack Logstash
1

Hi,

I'm trying to make a date filter for my servers. And i have an error.

Here is the configuration of the shipper :

input {
        file {
                path => ["/var/log/squid/cache.log"]
                type => "squidservice_mobylette"
                start_position => end
                stat_interval => 1
                tags => ["serveur_proxy", "squidservice_mobylette", "squidservice", "mobylette", "RES"]
                codec => plain {
                        charset => "ISO-8859-1"
                }
        }
}
output {
        redis {
                host => ["195.221.2.55","195.221.2.56","195.221.2.57"]
                data_type => "list"
                key => "logstash"
        }
}

Everything is correct to me. The shipper send the log to my cluster. I saw that with tcpdump.

Here is the configuration of Logstash on my cluster ELK :

input {
    redis {
           host => "127.0.0.1"
           port => 6379
           data_type => "list"
           type => "redis-input"
           key => "logstash"
    }

    beats {
            port => 5045
            type => "filebeat-input"
            congestion_threshold => 300
    }
}
output {
    if "_grokparsefailure" not in [tags] {
            if "tango" in [tags] {
                    elasticsearch {
                            hosts => ["10.1.101.1", "10.1.102.1", "10.1.103.1"]
                            index => "logstash-tango-%{+YYYY.MM.dd}"
                    }
            }
            if "nginx" in [tags] {
                    elasticsearch {
                            hosts => ["10.1.101.1", "10.1.102.1", "10.1.103.1"]
                            index => "logstash-isg-%{+YYYY.MM.dd}"
                    }
            }
            if "scarlette" in [tags] {
                    elasticsearch {
                            hosts => ["10.1.101.1", "10.1.102.1", "10.1.103.1"]
                            index => "logstash-scarlette-%{+YYYY.MM.dd}"
                    }
            }
            if "serveur_owncloud" in [tags] {
                    elasticsearch {
                            hosts => ["10.1.101.1", "10.1.102.1", "10.1.103.1"]
                            index => "logstash-owncloud-%{+YYYY.MM.dd}"
                    }
            }
            if "brouette" in [tags] or "poussette" in [tags] {
                    elasticsearch {
                            hosts => ["10.1.101.1", "10.1.102.1", "10.1.103.1"]
                            index => "logstash-mta-%{+YYYY.MM.dd}"
                    }
            }
            if "serveur_proxy" in [tags] or "serveur_dns" in [tags] {
                    elasticsearch {
                            hosts => ["10.1.101.1", "10.1.102.1", "10.1.103.1"]
                            index => "logstash-proxydns-%{+YYYY.MM.dd}"
                    }
            }
    }
}

Here is my filter :

filter {
        if "squidservice" in [tags] {
                grok {
                        patterns_dir => ["/etc/logstash/patterns.conf"]
                        match => ["message","%{SQUIDATE:squid_timestamp} kid1\| %{GREEDYDATA:content}"]
                        add_tag => "groked"
                }
                date {
                        match => ["squidate", "yyyy\/MM\/dd HH:mm:ss"]
                        timezone => "Europe/Paris"
                }
        }
}

The log i want to parse is like this :

2016/11/03 11:16:30 kid1| ctx: enter level  0: 'http://hits-i.iubenda.com/write?db=hits1'
2016/11/03 11:16:30 kid1| NOTICE: found double content-length header
2016/11/03 11:24:27 kid1| ctx: exit level  0
2016/11/03 11:24:27 kid1| Logfile: opening log stdio:/var/log/squid/netdb.state
2016/11/03 11:24:27 kid1| Logfile: closing log stdio:/var/log/squid/netdb.state
2016/11/03 11:24:27 kid1| NETDB state saved; 0 entries, 0 msec

Logstash on my cluster return the folowing error :

"error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [squid_timestamp]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: "2016/11/03 11:24:27" is malformed at "/11/03 11:24:27""}}}}, :level=>:warn}

I don't know why Logstash can't parse my log date... Do you have an idea ?

2

I believe there is no need to escape the forward slash, so match => ["squidate", "yyyy/MM/dd HH:mm:ss"] should work

3

The error still occur with the modification

4

I suggest adding a stdout { codec => rubydebug } at the start of the output section, to look at the event structure before it is sent to ES

5

Here is the event structure

{
                "message" => "2016/11/03 13:52:14 kid1| urlParse: URL too large (8860 bytes)",
               "@version" => "1",
             "@timestamp" => "2016-11-03T12:52:15.071Z",
                   "type" => "squidservice_mobylette",
                   "tags" => [
            [0] "serveur_proxy",
            [1] "squidservice_mobylette",
            [2] "squidservice",
            [3] "mobylette",
            [4] "RES",
            [5] "groked"
        ],
                   "host" => "mobylette",
                   "path" => "/var/log/squid/cache.log",
        "squid_timestamp" => "2016/11/03 13:52:14",
                "content" => "urlParse: URL too large (8860 bytes)"
    }
6

in your mapping, what type is squid_timestamp?
When you use the date filter, by default it parses the time in the match field, but writes the formatted time in @timestamp, to change this you can use the target option (which defaults to @timestamp) https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html#plugins-filters-date-target

My guess is that elasticsearch thinks that squid_timestamp is a date field, but you are sending plain text to it

7

I'm not very familiar with the mapping, so i let the default configuration with the Logstash index.

I already have multiple date filter. And they work fine. The only things that change between date filter that works and this one is the format of the date

this one work fine :

            date {
                    match => ["squiddate", "UNIX"]
                    timezone => "Europe/Paris"
            }

             date {
                    match => ["namedservice_timestamp", "dd-MMM-yyyy HH:mm:ss.SSS"]
                    timezone => "Europe/Paris"
            }

            date {
                    match => ["squidguard_timestamp", "yyyy-MM-dd HH:mm:ss"]
                    timezone => "Europe/Paris"
            }

But not this one :

           date {
                    match => ["squidate", "yyyy/MM/dd HH:mm:ss"]
                    timezone => "Europe/Paris"
            }

And for the others, i didn't change the default mapping...

8

I resolved my issue, by change the date format of my field before the date filter.

I replaced all forward slashes with dashs

So my original format is :

yyyy/MM/dd HH:mm:ss

And after this filter :

mutate {
    gsub => [
        # replace all forward slashes with underscore
        "squid_timestamp", "/", "-"
    ]
}

the format parsed by the date filter is :

yyyy-MM-dd HH:mm:ss

This resolved my issue.
Thanks for the help :slight_smile:

2 Likes
9

The debug stdout { codec => rubydebug } helped me to find this solution was needed:

mutate {
remove_field => [ "time" ]
}