Logstash 2.1.1 : Routing output to multiple clusters

I have a need to route different message types to different elasticsearch clusters. I realise I could do this by having different "elasticsearch" output blocks separated by "if" conditionals. However, that is wasteful and long winded...
What I'd like to do is set a variable containing the list of hosts for the cluster corresponding to that message type. Is this possible? If so, how would you recommend I do this?


The correct way to do this is using conditionals. A single Elasticsearch output can only send data to a single cluster.

You could try to use:


With a value from your message.

hosts => ["my-cluster-%{cluster}:9200"]

PS: I didn't try it

In addition to what other folks have suggested, you can use the translate filter to look up a message type in a table and obtain a hostname. That way you don't have to reload the Logstash configuration when you want to update the mapping table.

Is the translate yaml dictionary file read once or each time the filter is applied?

The YAML file is periodically read. See the documentation.