Kibana all time _dateparsefailure

Hello. I Have _dateparsefailure in Kibana, but in logstash, I think, it's ok
Can you help my ?

echo "gmail.video.google.com;10.111.13.232;31/Jul/2018:17:11:59 +0300;200;0.006;13820;GET /online/js/final.css?v=22e0aff8dsfdsfdsfdsf57e789146d01373aa2705b048e3d6e3f HTTP/1.1;0.006;127.0.0.1:8010;200;-;CmpF+ltgbip5i3sSA5siAg==;Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" | /usr/share/logstash/bin/logstash -e 'input { stdin {} } filter {grok {match =>{"message" => "%{DATA:VHOSTtmp};%{IP:remote_addr};%{HTTPDATE:timestamp};%{NUMBER:status};%{DATA:request_time_ms};%{NUMBER:bytes_sent};%{DATA:request_method} %{DATA:request_uri} %{DATA:protocol_version};%{DATA:upstream_response_time_ms};%{DATA:upstream_addr};%{DATA:upstream_status};%{DATA:http_x_forwarded_for};%{DATA:cookie_CID};%{GREEDYDATA:http_user_agent}"}} date {match => [ "timestamp","dd/MMM/yyyy:HH:mm:ss +0300" ]}}'
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[INFO ] 2018-07-31 18:17:43.817 [main] scaffold - Initializing module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configuration"}
[INFO ] 2018-07-31 18:17:43.835 [main] scaffold - Initializing module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/configuration"}
[WARN ] 2018-07-31 18:17:44.868 [LogStash::Runner] multilocal - Ignoring the 'pipelines.yml' file because modules or command line options are specified
[INFO ] 2018-07-31 18:17:45.256 [LogStash::Runner] runner - Starting Logstash {"logstash.version"=>"6.2.2"}
[INFO ] 2018-07-31 18:17:45.562 [Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}
[INFO ] 2018-07-31 18:17:47.042 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] pipeline - Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>2, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50}
[INFO ] 2018-07-31 18:17:47.341 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] pipeline - Pipeline started succesfully {:pipeline_id=>"main", :thread=>"#<Thread:0x3c2c29cd@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:246 run>"}
[INFO ] 2018-07-31 18:17:47.388 [Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22] agent - Pipelines running {:count=>1, :pipelines=>["main"]}
{
"@timestamp" => 2018-07-31T14:11:59.000Z,
"http_user_agent" => "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36",
"upstream_addr" => "127.0.0.1:8010",
"request_method" => "GET",
"http_x_forwarded_for" => "-",
"upstream_response_time_ms" => "0.006",
"@version" => "1",
"message" => "gmail.video.google.com;10.111.13.232;31/Jul/2018:17:11:59 +0300;200;0.006;13820;GET /online/js/final.css?v=22e0aff8dsfdsfdsfdsf57e789146d01373aa2705b048e3d6e3f HTTP/1.1;0.006;127.0.0.1:8010;200;-;CmpF+ltgbip5i3sSA5siAg==;Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36",
"remote_addr" => "10.111.13.232",
"status" => "200",
"request_uri" => "/online/js/final.css?v=22e0aff8dsfdsfdsfdsf57e789146d01373aa2705b048e3d6e3f",
"upstream_status" => "200",
"cookie_CID" => "CmpF+ltgbip5i3sSA5siAg==",
"timestamp" => "31/Jul/2018:17:11:59 +0300",
"host" => "NSTB-Logstash",
"request_time_ms" => "0.006",
"bytes_sent" => "13820",
"protocol_version" => "HTTP/1.1",
"VHOSTtmp" => "gmail.video.google.com"
}
[INFO ] 2018-07-31 18:17:47.873 [[main]-pipeline-manager] pipeline - Pipeline has terminated {:pipeline_id=>"main", :thread=>"#<Thread:0x3c2c29cd@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:246 run>"}

"_dateparsefailure" is a problem with date conversion in the logstash configuration file.

problem in data filter, I suppose, but -

input
31/Jul/2018:17:11:59 +0300
date {match => [ "timestamp","dd/MMM/yyyy:HH:mm:ss +0300" ]}}
output
"timestamp" => "31/Jul/2018:17:11:59 +0300",
"@timestamp" => 2018-07-31T14:11:59.000Z,

Here it's ok

With Z options same story
Logstash - ok, Kibana - error

echo "31/Jul/2018:17:11:59 +0300" | /usr/share/logstash/bin/logstash -e 'input { stdin {} } filter {date {match => [ "message","dd/MMM/yyyy:HH:mm:ss Z" ]}}'

{
"message" => "31/Jul/2018:17:11:59 +0300",
"host" => "NSTB-Logstash",
"@timestamp" => 2018-07-31T14:11:59.000Z,
"@version" => "1"
}

That is working just fine. 17:11 +0300 is 14:11 UTC.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.