# Advice on dynamically configuring logstash 6.2 output plugin
**URL:** https://discuss.elastic.co/t/advice-on-dynamically-configuring-logstash-6-2-output-plugin/119486
**Category:** Logstash
**Created:** [February 12, 2018, 2:45pm UTC](https://discuss.elastic.co/t/advice-on-dynamically-configuring-logstash-6-2-output-plugin/119486 "2018-02-12T14:45:51Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)
#### Post date: [February 12, 2018, 2:45pm UTC](https://discuss.elastic.co/t/advice-on-dynamically-configuring-logstash-6-2-output-plugin/119486/1 "2018-02-12T14:45:51Z")
Hi Team,
What is the recommended way to dynamically tell logstash to which elasticsearch hosts logstash should connect to?
I have several hosts where I run logstash. However for some hosts I need the data to get to cluster A and for some hosts I need the data to be sent to cluster B.
What I used to do prior to 6.x is dynamically generate the output hosts (from a self-configured central config where I can pick corresponding cluster nodes excluding master dedicated ones) and append it to logstash.conf using the --config.string option. However, in 6.x [--config.string is no longer appended](https://www.elastic.co/guide/en/logstash/current/breaking-changes.html#_command_line_interface_behavior) so I was wondering maybe I have been doing it the non-optimum way all this time.
In 6.x, I can probably combine logstash.conf and --config.string into either a long string or write the same string into a file then passing it to bin/logstash. However, I was wondering if there is another better way?
Cheers,
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 12, 2018, 3:35pm UTC](https://discuss.elastic.co/t/advice-on-dynamically-configuring-logstash-6-2-output-plugin/119486/2 "2018-02-12T15:35:40Z")
I can think of a couple of options:
- Maintain different sets of configuration files for different hosts.
- Use a configuration management tool (Ansible, Chef, Puppet, ...) to generate the configuration files with a templating language that for example supports variable substitution.
- Use Logstash's support for referencing environment variables ([https://www.elastic.co/guide/en/logstash/current/environment-variables.html](https://www.elastic.co/guide/en/logstash/current/environment-variables.html)).
---
### Author:  [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)
#### Post date: [February 12, 2018, 3:53pm UTC](https://discuss.elastic.co/t/advice-on-dynamically-configuring-logstash-6-2-output-plugin/119486/3 "2018-02-12T15:53:10Z")
Thanks Magnus for your suggestions.
I think I like the environment variables approach the best as it would allow me to have one central config that is dynamic.
Maintaining different sets of configs could be a bit cumbersome just cos there would be quite a few sets to manually maintain.
I am using ansible already but I am trying to avoid "extra couplings" involved with config generations.
Cheers!
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 12, 2018, 9:04pm UTC](https://discuss.elastic.co/t/advice-on-dynamically-configuring-logstash-6-2-output-plugin/119486/4 "2018-02-12T21:04:21Z")
> I am using ansible already but I am trying to avoid "extra couplings" involved with config generations.
And how are you going to differentiate the environment variables between hosts?
---
### Author:  [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)
#### Post date: [February 12, 2018, 11:44pm UTC](https://discuss.elastic.co/t/advice-on-dynamically-configuring-logstash-6-2-output-plugin/119486/5 "2018-02-12T23:44:00Z")
Hey Magnus,
Yeah, luckily for me, the hosts already have the metadata that I need readily available on them. My wrapper start up script will expose the metadata via env vars that will be referenced inside logstash.conf.
---
### Author:  [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)
#### Post date: [February 19, 2018, 1:01pm UTC](https://discuss.elastic.co/t/advice-on-dynamically-configuring-logstash-6-2-output-plugin/119486/6 "2018-02-19T13:01:57Z")
Hi Team,
There's an open ticket for making array env vars work: [https://github.com/elastic/logstash/issues/6366](https://github.com/elastic/logstash/issues/6366), do you know if this has been implemented?
I tried several ways of exporting an env var as a list but logstash isn't treating the env var as an array but a string instead, eg:
```
export ES_HOSTS='host1,host2'
```
Then that would be referenced here:
```
output {
elasticsearch {
hosts => ["${ES_HOSTS}"]
}
}
```
Also tried `export ES_HOSTS='["host1", "host2"]'` pass that to `hosts => "${ES_HOSTS}"` but this messes up the syntax by the looks of it.
If this hasn't been implemented, as a workaround, would it be a good idea to use ruby to generate a list from the env var and then save it into a meta field that would be referenced in the output plugin (granted that that ruby code would run on each event)?
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 19, 2018, 3:04pm UTC](https://discuss.elastic.co/t/advice-on-dynamically-configuring-logstash-6-2-output-plugin/119486/7 "2018-02-19T15:04:44Z")
> There's an open ticket for making array env vars work: [Logstash 2.4.1/5.0.x: Environments Variables parsed as Lists/Arrays for Logstash Configuration · Issue #6366 · elastic/logstash · GitHub](https://github.com/elastic/logstash/issues/6366), do you know if this has been implemented?
Since the issue is open I have no reason to believe that it has been implemented.
> If this hasn't been implemented, as a workaround, would it be a good idea to use ruby to generate a list from the env var and then save it into a meta field that would be referenced in the output plugin (granted that that ruby code would run on each event)?
I'm pretty sure the `hosts` option doesn't parse `%{field}` references, in which case your proposal won't work.
---
### Author:  [@ld\_pvl](https://discuss.elastic.co/u/ld_pvl)
#### Post date: [February 19, 2018, 3:30pm UTC](https://discuss.elastic.co/t/advice-on-dynamically-configuring-logstash-6-2-output-plugin/119486/8 "2018-02-19T15:30:20Z")
That's unfortunate.
Any rough timeline for when that ticket is going to be done - just wondering cos it's been open for a year and I just wanna gauge whether I should wait or start on a different approach now.
I really hope the feature gets done soon as filebeat has it: [https://www.elastic.co/guide/en/beats/filebeat/current/using-environ-vars.html](https://www.elastic.co/guide/en/beats/filebeat/current/using-environ-vars.html) and thought logstash would also have it.
---
### Author:  [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)
#### Post date: [February 19, 2018, 3:48pm UTC](https://discuss.elastic.co/t/advice-on-dynamically-configuring-logstash-6-2-output-plugin/119486/9 "2018-02-19T15:48:47Z")
> Any rough timeline for when that ticket is going to be done
I have no idea.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 19, 2018, 3:48pm UTC](https://discuss.elastic.co/t/advice-on-dynamically-configuring-logstash-6-2-output-plugin/119486/10 "2018-03-19T15:48:58Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.