is this my else condition which cause logstash-* index ?
It sounds like you effectively have this:
output {
if [type] == "OnsuranceAppLog" {
elasticsearch {
hosts => ["localhost:9200"]
index => "onsurance-%{+YYYY.MM.dd}"
}
} else {
elasticsearch {
hosts => ["localhost:9200"]
}
stdout { codec => rubydebug }
}
if [type] == "iis" {
elasticsearch {
hosts => ["localhost:9200"]
index => "iis-%{+YYYY.MM.dd}"
}
} else {
elasticsearch {
hosts => ["localhost:9200"]
}
stdout { codec => rubydebug }
}
}
In that case yes, the else block is the problem. All messages will reach it.
another question is, how can i tell ES to keep index of only last 30 days? and delete everything which is older than 30 days. I know you can fire a query to ES but is there any Setting which i can set once and it does the magic?
There's no setting but the Curator program can do this for you.