Logstash 2.1.1 : Routing output to multiple clusters


I have a need to route different message types to different elasticsearch clusters. I realise I could do this by having different "elasticsearch" output blocks separated by "if" conditionals. However, that is wasteful and long winded...
What I'd like to do is set a variable containing the list of hosts for the cluster corresponding to that message type. Is this possible? If so, how would you recommend I do this?


(Christian Dahlqvist) #2

The correct way to do this is using conditionals. A single Elasticsearch output can only send data to a single cluster.

(Thomas Decaux) #3

You could try to use:


With a value from your message.

hosts => ["my-cluster-%{cluster}:9200"]

PS: I didn't try it

(Magnus Bäck) #4

In addition to what other folks have suggested, you can use the translate filter to look up a message type in a table and obtain a hostname. That way you don't have to reload the Logstash configuration when you want to update the mapping table.


Is the translate yaml dictionary file read once or each time the filter is applied?

(Magnus Bäck) #6

The YAML file is periodically read. See the documentation.

(system) #7