Apache Error logs timestamp

Hi

I tried to get the pattern for apache error timestamp but all failed

The timestamp is used in elasticsearch is when it is indexed not the time of the error

e.g.
[Sun Feb 05 00:32:24.992868 2017] [mpm_winnt:notice] [pid 8776:tid 168] AH00354: Child: Starting 150 worker threads., Child: Starting 150 worker threads

and when I search for timestamp in the index pattern I find this:

Multiple timestamp with different types: date and string

###########################################################

This is the logstash config filter part

###########################################################
filter {
if [type] == "syslog"
{
mutate { add_tag => "syslog_tag" }
grok
{
match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:[%{POSINT:syslog_pid}])?: %{GREEDYDATA:syslog_message}" }
add_field => [ "received_at", "%{@timestamp}" ]
add_field => [ "received_from", "%{host}" ]
}
syslog_pri { }
date
{
match => [ "syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss" ]
}
}
#The winlogbeat send it as wineventlog and logstash put it automatically in winlogevent index

No need for special configuration here

####################################################################################

if [type] == "wineventlog"

{

grok

{

match => { "TimeCreated", "Date(%{NUMBER:timestamp})" }

}

date

{

match => [ "timestamp", "UNIX_MS" ]

}

}

####################################################################################
if [type] == "apache_access"
{
grok
{
match => { "message" => ["%{COMBINEDAPACHELOG}", "%{IPORHOST:clientip} %{NOTSPACE:ident} %{NOTSPACE:auth} [%{HTTPDATE:timestamp}] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion}))" %{NOTSPACE:response} (?:%{NOTSPACE:bytes})" ] }
}
date
{
match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
}
}
if [type] == "apache_error"
{
grok
{
match => { "message" => "[(?%{DAY:day} %{MONTH:month} %{MONTHDAY} %{TIME} %{YEAR})] [%{WORD:module}:%{LOGLEVEL:loglevel}] [pid %{NUMBER:pid}:tid %{NUMBER:tid}]( (%{POSINT:proxy_errorcode})%{DATA:proxy_errormessage}:)?( [client %{IPORHOST:client}:%{POSINT:clientport}])? %{DATA:errorcode}: %{GREEDYDATA:message}" }
}
date
{
match => [ "timestamp" , "EEE MMM dd HH:mm:ssssss yyyy" ]
}
}
if [type] == "apache_sslrequest"
{
grok
{
match => { "message" => "[%{HTTPDATE:timestamp}] %{IPORHOST:client} %{NOTSPACE:protocol} %{NOTSPACE:cipher} "(%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" }
}
date
{
match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
}
}
}

improve your question please.

  1. I think your field named "timestamp" is not been recognized as one, you need to see if the date in your logs is really a TimeStamp that is following a specific pattern.
  2. To use the Date in your logs as the "Time" Field in Kibana, you will need to Identify the timestamp field in the moment that you create your index in kibana management ui.

This is the date from the apache error log

[Sun Feb 05 00:32:24.992868 2017]

And when I create the index, I chose the timestamp field to be used but still no luck

I believe what you're asking is how to update the @timestamp to the date time your error occurred. The reason you have multiple timestamp is because you declared a timestamp variable.

Take a look at the discussion below:

Hi Jimmy,

I declared a different variable "mytimestamp" and used the date filter to assign it to the @timestamp but still with no luck.

From what I see in the discussions and documentation, this should be straight forward !

    if [type] == "apache_error"
    {
            grok
            {
                    match => { "message" => "\[(?<mytimestamp>%{DAY:day} %{MONTH:month} %{MONTHDAY} %{TIME} %{YEAR})\] \[%{WORD:module}:%{LOGLEVEL:loglevel}\] \[pid %{NUMBER:pid}:tid %{NUMBER:tid}\]( \(%{POSINT:proxy_errorcode}\)%{DATA:proxy_errormessage}:)?( \[client %{IPORHOST:client}:%{POSINT:clientport}\])? %{DATA:errorcode}: %{GREEDYDATA:message}" }
            }
            date
            {
                    match => [ "mytimestamp" , "EEE MMM dd HH:mm:ssssss yyyy" ]
            }
    }

Please show an example of an event that has been processed with those filters. Please copy/paste from Kibana's JSON tab instead of posting a screenshot.

Here it is

{
"_index": "apache-2017.04.11",
"_type": "apache_error",
"_id": "AVtdmOC-wb2d0_A3HR5q",
"_score": null,
"_source": {
"offset": 19187,
"module": "core",
"input_type": "log",
"pid": "1056",
"source": "C:\xampp\apache\logs\error.log",
"message": [
"[Mon Feb 27 10:59:12.930694 2017] [core:notice] [pid 1056:tid 528] AH00094: Command line: 'c:\\xampp\\apache\\bin\\httpd.exe -d C:/xampp/apache'",
"Command line: 'c:\\xampp\\apache\\bin\\httpd.exe -d C:/xampp/apache'"
],
"type": "apache_error",
"tid": "528",
"tags": [
"beats_input_codec_plain_applied",
"_dateparsefailure"
],
"@timestamp": "2017-04-11T15:18:54.132Z",
"month": "Feb",
"mytimestamp": "Mon Feb 27 10:59:12.930694 2017",
"loglevel": "notice",
"@version": "1",
"beat": {
"hostname": "services",
"name": "services",
"version": "5.2.2"
},
"host": "services",
"fields": {
"logtype": "apache"
},
"day": "Mon",
"errorcode": "AH00094"
},
"fields": {
"@timestamp": [
1491923934132
]
},
"highlight": {
"source": [
"@kibana-highlighted-field@C@/kibana-highlighted-field@:\@kibana-highlighted-field@xampp@/kibana-highlighted-field@\@kibana-highlighted-field@apache@/kibana-highlighted-field@\@kibana-highlighted-field@logs@/kibana-highlighted-field@\@kibana-highlighted-field@error.log@/kibana-highlighted-field@"
]
},
"sort": [
1491923934132
]
}

The _dateparsefailure tag indicates that the date filter failed. Read Logstash's log to find out why.

I made the logstash logging as debug and this is what I got, I do not see any error

@timestamp"=>2017-04-12T08:43:30.624Z !!

======================================================================
LOG

[2017-04-12T10:43:00,859][DEBUG][logstash.filters.date ] config LogStash::Filters::Date/@tag_on_failure = ["_dateparsefailure"]
[2017-04-12T10:43:00,859][DEBUG][org.logstash.filters.DateFilter] Date filter with format=EEE MMM dd HH:mm:ss,SSS yyyy, locale=null, timezone=null built as org.logstash.filters.parser.JodaParser

[2017-04-12T10:43:31,078][DEBUG][logstash.pipeline ] filter received {"event"=>{"@timestamp"=>2017-04-12T08:43:30.624Z, "offset"=>168, "@version"=>"1", "input_type"=>"log", "beat"=>{"hostname"=>"services", "name"=>"services", "version"=>"5.2.2"}, "host"=>"services", "source"=>"C:\xampp\apache\logs\error.log", "message"=>"[Thu Oct 13 08:24:05.627280 2016] [ssl:warn] [pid 6772:tid 172] AH01909: www.example.com:443:0 server certificate does NOT include an ID which matches the server name", "fields"=>{"logtype"=>"apache"}, "type"=>"apache_error", "tags"=>["beats_input_codec_plain_applied"]}}

[2017-04-12T10:43:31,078][DEBUG][logstash.filters.grok ] Running grok filter {:event=>2017-04-12T08:43:30.624Z services [Thu Oct 13 08:24:05.627280 2016] [ssl:warn] [pid 6772:tid 172] AH01909: www.example.com:443:0 server certificate does NOT include an ID which matches the server name}

[2017-04-12T10:43:31,078][DEBUG][logstash.filters.grok ] Event now: {:event=>2017-04-12T08:43:30.624Z services [Thu Oct 13 08:24:05.627280 2016] [ssl:warn] [pid 6772:tid 172] AH01909: www.example.com:443:0 server certificate does NOT include an ID which matches the server name,www.example.com:443:0 server certificate does NOT include an ID which matches the server name}

[2017-04-12T10:43:31,084][DEBUG][logstash.pipeline ] output received {"event"=>{"offset"=>168, "module"=>"ssl", "input_type"=>"log", "pid"=>"6772", "source"=>"C:\xampp\apache\logs\error.log", "message"=>["[Thu Oct 13 08:24:05.627280 2016] [ssl:warn] [pid 6772:tid 172] AH01909: www.example.com:443:0 server certificate does NOT include an ID which matches the server name", "www.example.com:443:0 server certificate does NOT include an ID which matches the server name"], "type"=>"apache_error", "tid"=>"172", "tags"=>["beats_input_codec_plain_applied", "_dateparsefailure"], "@timestamp"=>2017-04-12T08:43:30.624Z, "loglevel"=>"warn", "@version"=>"1", "beat"=>{"hostname"=>"services", "name"=>"services", "version"=>"5.2.2"}, "host"=>"services", "fields"=>{"logtype"=>"apache"}, "errorcode"=>"AH01909", "timestamp"=>"Thu Oct 13 08:24:05.627280 2016"}}

I found the issue.

It is because of the date format in the date filter

I used "," instead of "." and use SSS intead of SSSSSS

And in case you do not need these milli or micro seconds you can use the gsub to remove the 6 or 3 numbers after the . before you give it to the date filter

--

Thanks all for your help

            mutate
            {
                   gsub => ["timestamp", "\.\d{6}", ""]
            }




            date
            {
                    match => [ "timestamp" , "EEE MMM dd HH:mm:ss.SSSSSS yyyy" ]
            }

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.