An index per host

[vman]

if [host] in ["host1", "host2", "host3" , "host4" , "host5"] {
elasticsearch {
hosts => ["localhost:9200"]
...
index => "host1-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
} else {
elasticsearch {
hosts => ["localhost:9200"]
...
index => "host2-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
} else {
elasticsearch {
hosts => ["localhost:9200"]
...
index => "host3-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
} else {
elasticsearch {
hosts => ["localhost:9200"]
...
index => "host4-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
} else {
elasticsearch {
hosts => ["localhost:9200"]
...
index => "host5-%{+YYYY.MM.dd}"
document_type => "%{[@metadata][type]}"
}
}

Can this be achieved?

[warkolm (Mark Walkom)]

If you have the hostname as a field:value pair, why not just use that as a variable in the index name?

[Christian_Dahlqvist]

You can not have conditionals within a plugin, but can do something like this:

if [host] in ["host1", "host2", "host3" , "host4" , "host5"] {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "%{[host]}-%{+YYYY.MM.dd}"
  }
} else {
  elasticsearch {
    hosts => ["localhost:9200"]
     index => "default-%{+YYYY.MM.dd}"
  }
}

Having lots of small indices and shards in a cluster can however be very inefficient and cause performance problems, so you should ask yourself if this really is necessary. If you still decide to to this, which I generally would not recommend, you probably want to reduce the number ion primary shards per index and consider using monthly rather than daily indices if you intend to keep your data for an extended period.