How to use environment variable in pipelines.yml? (Windows)

Elastic Stack Logstash
1

Hi,

as dev environment for the elastic stack I am using my laptop which runs windows.
Since I have different paths and different operating systems on my laptop and on the live systems, I would like to use environment variables to configure paths. In logstash.yml it works fine with accessing envrionment variables using follwoing syntax: ${ENV_VAR_NAME}

But using this in pipelines.yml it does not work.

snippet of pipelines.yml:

- pipeline.id: rdautoorders
  pipeline.workers: 1
  #pipeline.output.workers: 1
  queue.type: memory
  path.config: "${PIPELINE_BASE_PATH}/rdautoorders"

when starting logstash, I got following output:

C:\work\elastic>logstash_dev.cmd -t
C:\work\elastic>SET STACK_VERSION=6.4.1
C:\work\elastic>SET HOME=c:\work\elastic\logstash
C:\work\elastic>set JAVA_HOME=c:\work\elastic\java
C:\work\elastic>SET ES_HOST=localhost
C:\work\elastic>SET ES_PORT=9200
C:\work\elastic>SET REDIS_HOST=localhost
C:\work\elastic>SET REDIS_PORT=6379
C:\work\elastic>SET LS_NODE_NAME=DEV_NODE
C:\work\elastic>SET PIPELINE_BASE_PATH=C:\work\elastic\logstash\config\redis_1.1.0\pipelines
C:\work\elastic>SET LS_CONF_PATH=redis_1.1.0
C:\work\elastic>SET GLOBAL_GROK_PATTERN_DIR=c:\work\elastic\logstash\redis_1.1.0\patterns
C:\work\elastic>echo C:\work\elastic\logstash\config\redis_1.1.0\pipelines
C:\work\elastic\logstash\config\redis_1.1.0\pipelines

C:\work\elastic>call c:\work\elastic\logstash\logstash-6.4.1\bin\logstash.bat --path.settings=c:\work\elastic\logstash\config\redis_1.1.0\config -t
Sending Logstash logs to c:/work/elastic/logstash/logstash-6.4.1/logs which is now configured via log4j2.properties
[2018-10-09T22:04:11,301][INFO ][logstash.config.source.local.configpathloader] No config files found in path {:path=>"C:/work/elastic/${PIPELINE_BASE_PATH}/rdautoorders"}
[2018-10-09T22:04:11,312][ERROR][logstash.config.sourceloader] No configuration found in the configured sources.
Configuration OK
[2018-10-09T22:04:11,316][INFO ][logstash.runner          ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash

Any Ideas? Would be nice If I could use the identical config on the different stages and configure only by environment variables.

Thanks, Andreas

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.