Environment variable interpolation in Logstash conf files

Elastic Stack Logstash
1

We run logstash on k8s with centralized pipeline management, and I'm attempting to pass credentials into the containers as env vars. This has worked for everything but the azure_eventhubs input plugin. Debug logging seems to indicate that the SAS_KEY env var in this plugin:

azure_event_hubs {
      codec => "json"
      config_mode => "advanced"
      threads => 5
      decorate_events => true
      storage_connection => "DefaultEndpointsProtocol=https;AccountName=<account_name>;AccountKey=${STORAGE_KEY};EndpointSuffix=core.windows.net"
      event_hubs => [
        {"paas" => {
          event_hub_connection => "Endpoint=sb://<event_hub_namespace>.servicebus.windows.net/;SharedAccessKeyName=collector;SharedAccessKey=${SAS_KEY};EntityPath=paas"
          consumer_group => "logstash"
          storage_container => "azurelogscentralus-paas"
        }}
      ]
      add_field => {
        "[fields][logsource]" => "azure-paas"
      }
    }

is not interpolated, while the STORAGE_KEY var is. Does interpolation not go into nested structures like this?

2

The underlying cause for this ended up being that password configuration variables are passed to the interpolation function unwrapped, and since Password is not a String, the interpolation returned immediately.

I filed a PR to unwrap the password, interpolate, and return the rewrapped object, would appreciate it whenever someone from the elastic team has time to take a look.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.