Hi All ,
When I am indexing data from Apache access.log , current system date ( Actual date ) is using for indexing. Actually I am indexing 2014 year logs . But these logs are indexing as in today's log . I want to index these data with actual date in log .
Using ELK 5.4.1
Indexing log from access.log
Sample data:
198.0.200.105 - - [14/Jan/2014:09:36:50 -0800] "GET /svds.com/rockandroll HTTP/1.1" 301 241 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36"
198.0.200.105 - - [14/Jan/2014:09:36:50 -0800] "GET /svds.com/rockandroll/ HTTP/1.1" 200 8301 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36"
198.0.200.105 - - [14/Jan/2014:09:36:51 -0800] "GET /svds.com/rockandroll/js/libs/modernizr-2.6.2.min.js HTTP/1.1" 200 8768 "http://www.svds.com/rockandroll/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36"
Filter is :
input{ beats
{ port => "5044"
type => "log"}
}
filter {
if [type] == "log" {
grok {
match => { "message" => "%{HOSTNAME:vhost} - - [%{HTTPDATE:timestamp}] "%{WORD:Method} %{DATA:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response:int} %{NUMBER:response:int} %{GREEDYDATA:Details}"}
}
mutate {
remove_field => [ "message" ]
}
}
}
output { stdout{ codec => rubydebug }
elasticsearch { hosts => ["localhost:9200"] }
The Out put format in Kibana is attached
@timestamp November 28th 2017, 16:38:56.636 ( Today time )
t timestamp 21/Oct/2014:23:19:21 -0700 ( Actual Date )
What to do for actual date comes in @timestamp
For example @timestamp = 21/Oct/2014:23:19:u7121: