[logstash 5.3][plugin output elasticsearch] Using env variable in conf file

[fares (fares oueslati)]

Hello,

I'm trying to use env var in my conf file

elasticsearch{
...
hosts => ${ES_HOST}
}

My var contains host:port type value, ie localhost:9200
Got this error
ERROR logstash.agent - Cannot load an invalid configuration {:reason=>"bad URI(is not URI?):

Thanks for your help

[magnusbaeck (Magnus Bäck)]

I don't think the hosts option is subject to such expansions. Supporting event-dependent %{fieldname} expansions would be a bad idea, and since they go along with ${envname} expansions you don't get the latter either.

[fares (fares oueslati)]

Thanks @magnusbaeck for your reply.

Could you please elaborate why hosts option is not subject to such expansions ?

Since we could do so in 2.x logstash version and in other ouptut plugins like Kafka in logstash 5.3 ?

Thank you

[magnusbaeck (Magnus Bäck)]

Could you please elaborate why hosts option is not subject to such expansions ?

Because %{fieldname} expansions in hosts could mean that Logstash would have to connect to a different host for every event. There are probably ways to solve that in a reasonable way but I suspect nobody has wanted to spend the time doing that.

Since we could do so in 2.x logstash version

Oh? That's surprising.

and in other ouptut plugins like Kafka in logstash 5.3 ?

Which option for the kafka output?

[fares (fares oueslati)]

Hello Magnus,

Which option for the kafka output?

Here is the kafka output in my logstash 5.3 conf
I'm setting an env variable KAFKA_SERVERS="host1:9092,host2:9092,host3:9092"

 kafka{
                        codec=>"json"
                        topic_id=>"tpc_ex"
                        bootstrap_servers=>"${KAFKA_SERVERS}"
                }

It works well.