I upgrade my ELK stack from 2.4 to 5.0. after the upgrade all my event log indices are saved as random words. like
UcA4wqnpQs2D4AUdBpuRwg
U6a2PE9mTduqAGfbDLT22g
ZzOUMMSRRPukjy6muHRS2w
Before the upgrade it was winlogbeat- (date of the event received)
my logstash config file as follows,
input {
beats {
port => 5000
type => "filebeat"
}
beats {
port => 5001
type => "winlogbeat"
}
}
Filebeat filter
filter {
#ignore log comments
if [message] =~ "^#" {
drop {}
}
grok {
#patterns_dir => "./patterns"
match => {"message" => "%{TIMESTAMP_ISO8601:timestamp} %{IPORHOST:servername1} %{IPORHOST:serverip} %{WORD:verb} %{NOTSPACE:request1} %{GREEDYDATA:request2} %{NUMBER:port} %{IPORHOST:clientip} %{NOTSPACE:protocol} %{NOTSPACE:querystring} %{IPORHOST:servername} %{NOTSPACE:agent} %{NOTSPACE:referrer} %{NUMBER:response} %{NUMBER:sub_response} %{NUMBER:sc_status} %{NUMBER:responsetime}"}
}
date {
match => [ "timestamp", "yyyy-MM-dd HH:mm:ss" ]
locale => "en"
}
}
Second filter
#filter {
if "_grokparsefailure" in [tags] {
} else {
# on success remove the message field to save space
mutate {
# remove_field => ["message", "timestamp", "servername", "servername1"]
#}
}
#}
output {
elasticsearch {
hosts => ["172..........:9200"]
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
Now I can't even use curator to clean up old indices. Because their is no detectable patterns in the indices folders.