Cannot access logstash environmental variables

[seanziee (Kofi)]

I installed logstash via Docker and I keep getting the error when I am trying to access an environmental variable locally on the machine. I've tried adding it in /etc/environment and /etc/default/logstash both getting the same error. Any idea where to store this variable on the docker installation?

I'm using the variable in an input in logstash called ${Timestamp}

Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Cannot evaluate ${Timestamp}. Replacement variable Timestamp is not defined in a Logstash secret store or as an Environment entry and there is no default value given.", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/util/substitution_variables.rb:50:in `block in replace_placeholders'", "org/jruby/RubyString.java:2541:in `gsub'", "/usr/share/logstash/logstash-core/lib/logstash/util/substitution_variables.rb:35:in `replace_placeholders'", "/usr/share/logstash/logstash-core/lib/logstash/util/substitution_variables.rb:23:in `deep_replace'", "/usr/share/logstash/logstash-core/lib/logstash/config/mixin.rb:82:in `block in config_init'", "org/jruby/RubyHash.java:1343:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/config/mixin.rb:81:in `config_init'", "/usr/share/logstash/logstash-core/lib/logstash/inputs/base.rb:60:in `initialize'", "org/logstash/plugins/PluginFactoryExt.java:233:in `plugin'", "org/logstash/plugins/PluginFactoryExt.java:166:in `plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:71:in `plugin'", "(eval):8:in `'", "org/jruby/RubyKernel.java:994:in `eval'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:49:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:90:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:38:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:309:in `block in converge_state'"]}

[seanziee (Kofi)]

What I'm trying to do in the end is I have a node script that finds the most recent Timestamp in my SQL db so then I can import only new ES documents to that SQL db. Once the node script runs, it needs to save that timestamp locally for LS to access to use as a filter.

[seanziee (Kofi)]

When I ran logstash as sudo -u root /usr/share/logstash/bin/logstash then it could access /etc/environment folder with the timestamp in there. Problem solved.