Continuing the discussion from Multiple Elasticsearch Indices in Logstash output:
Running logstash 1.5.4, elasticsearch 1.4.5 and kibana 4.1.2
Ive set up this output configuration for logstash but I'm not seeing the different indices being created. Am I missing something?
output{
if [source] =~ "-dev" {
elasticsearch {
host => "localhost"
index => "dev-%{_type}-%{+YYYY.MM.dd}"
}
} else if [source] =~ "-staging" {
elasticsearch {
host => "localhost"
index => "staging-%{_type}-%{+YYYY.MM.dd}"
}
} else if [source] =~ "*-prod" {
elasticsearch {
host => "localhost"
index => "prod-%{_type}-%{+YYYY.MM.dd}"
}
} else {
elasticsearch {
host => "localhost"
}
}
}
All my records have the "source" field. but i only see them in the "logstash-" index. What's going wrong?
The reason for doing this is I have one field which I'd like to visualize in kibana but while it's available in the logstash- index, kibana's discover tab always says "this field is not indexed thus unavailable for visualization and search". Most of my log entries dont include my desired field, so I thought if I split up the logs in this way, the field I'm interested wouldn't be swamped by non-existent entries.