Passing in complex array as environment variables?

[sgreszcz (Stephen Greszczyszyn)]

Hi there,

I'm using the Logstash SNMP input to poll a list of servers with a list of OIDs. We are deploying Logstash in Docker using Ansible, and we try to maintain the code using environment variables passed through to the Logstash configurations.

From the documentation I can build a host array and OID array. Is there any way to build this as an environment variable and pass it into Logstash?

input {
  snmp {
    get => ["1.3.6.1.2.1.1.1.0", "1.3.6.1.2.1.1.3.0", "1.3.6.1.2.1.1.5.0"]
    hosts => [{host => "udp:127.0.0.1/161" community => "public"}, {host => "udp:192.168.0.1/161" community => "private"}]
  }
}

[Rahul_Kumar4 (Rahul Kumar)]

@sgreszcz

Is there any way to build this as an environment variable and pass it into Logstash?

If you are using Ansible playbook for Logstash configuration, you could assign variables in your playbook file with the host and OID values and then do variable substitution using Jinja (which Logstash supports). See here

[sgreszcz (Stephen Greszczyszyn)]

Hi there @Rahul_Kumar4, thanks for your response!

Yes, that is how I'm doing it to substitute single variables, but you are saying that I could substitute a more complex string as a variable and logstash will recognise it as an array type?
For example this is all working:

ansible:

# SNMP Environment Variables
                SNMP_INTERVAL: "{{snmp_interval}}"
                SNMP_SECURITY_LEVEL: "{{snmp_security_level}}"
                SNMP_USER: "{{snmp_user}}"
                SNMP_AUTH_PROTOCOL: "{{snmp_auth_protocol}}"
                SNMP_AUTH_PASS: "{{snmp_auth_pass}}"
                SNMP_PRIV_PROTOCOL: "{{snmp_priv_protocol}}"
                SNMP_PRIV_PASS: "{{snmp_priv_pass}}"

logstash:

input {
	snmp {
		interval => "${SNMP_INTERVAL}"
		security_level => "${SNMP_SECURITY_LEVEL}"
		security_name => "${SNMP_USER}"
		auth_protocol => "${SNMP_AUTH_PROTOCOL}"
		auth_pass => "${SNMP_AUTH_PASS}"
		priv_protocol => "${SNMP_PRIV_PROTOCOL}"
		priv_pass => "${SNMP_PRIV_PASS}"
}
}

So now I need to figure out how to pass something like this structure from ansible using a jinja template?

logstash:

		hosts => [
		  {host => "udp:host1/161" version => "3"},
		  {host => "udp:host2/161" version => "3"},
		  {host => "udp:host3/161" version => "3"},
          {host => "udp:host4/161" version => "3"}]

[Rahul_Kumar4 (Rahul Kumar)]

Yeap.

In your playbook declare a variable and pass the hosts as array of strings in YAML format. Something like below:

name_of_hosts_var:
- host1 (these should be strings)
- host2
- more hosts

(I can't get the indentation to work correctly in this editor but those hosts strings need to be indented correctly in YML)

and then in the logstash pipeline you can try

hosts => {{ name_of_hosts_var | to_json }}

That should give hosts as an array of comma separated strings.