Indices folder names are random


#1

I upgrade my ELK stack from 2.4 to 5.0. after the upgrade all my event log indices are saved as random words. like
UcA4wqnpQs2D4AUdBpuRwg
U6a2PE9mTduqAGfbDLT22g
ZzOUMMSRRPukjy6muHRS2w

Before the upgrade it was winlogbeat- (date of the event received)
my logstash config file as follows,

input {
beats {
port => 5000
type => "filebeat"
}
beats {
port => 5001
type => "winlogbeat"
}
}

Filebeat filter

filter {
#ignore log comments
if [message] =~ "^#" {
drop {}
}
grok {
#patterns_dir => "./patterns"

match => {"message" => "%{TIMESTAMP_ISO8601:timestamp} %{IPORHOST:servername1} %{IPORHOST:serverip} %{WORD:verb} %{NOTSPACE:request1} %{GREEDYDATA:request2} %{NUMBER:port} %{IPORHOST:clientip} %{NOTSPACE:protocol} %{NOTSPACE:querystring} %{IPORHOST:servername} %{NOTSPACE:agent} %{NOTSPACE:referrer} %{NUMBER:response} %{NUMBER:sub_response} %{NUMBER:sc_status} %{NUMBER:responsetime}"}
}
date {
match => [ "timestamp", "yyyy-MM-dd HH:mm:ss" ]
locale => "en"
}
}

Second filter

#filter {

if "_grokparsefailure" in [tags] {

} else {

# on success remove the message field to save space

mutate {

# remove_field => ["message", "timestamp", "servername", "servername1"]
#}

}

#}

output {
elasticsearch {
hosts => ["172..........:9200"]
manage_template => false
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}
Now I can't even use curator to clean up old indices. Because their is no detectable patterns in the indices folders.


(Mark Walkom) #2

Curator doesn't use directory names, it uses the APIs.


#3

then how can I use curator to clean up my old indices for logstash 5.0


(Mark Walkom) #4

It's the same as it ever was - https://www.elastic.co/guide/en/elasticsearch/client/curator/current/index.html


#5

But why folder names are in gibberish.previously it was recognizable. :confused:


(Mark Walkom) #6

It doesn't matter, you shouldn't be looking at the file system to remove things. Use the APIs, like curator :slight_smile:


#7

Thanx warkolm. I will try with curator API then :slight_smile:


(system) #8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.