# Environment variables not replaced in external configuration file **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [March 20, 2018, 6:06am UTC](https://discuss.elastic.co/t/environment-variables-not-replaced-in-external-configuration-file/124690 "2018-03-20T06:06:49Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![oetzi](https://avatars.discourse-cdn.com/v4/letter/o/3ec8ea/32.png) [@oetzi](https://discuss.elastic.co/u/oetzi)\ **Post date:** [March 20, 2018, 6:06am UTC](https://discuss.elastic.co/t/environment-variables-not-replaced-in-external-configuration-file/124690/1 "2018-03-20T06:06:49Z")
I face the problem that environment variables in external config files are not replaced. The external config files are imported to the filebeat.yml file via the filebeat.config.prospectors option. My external config file contains some environment variables. These are not replaced. If I copy the contents of the external yml-file into the file filebeat.yml (option filebeat.prospectors) everything works fine. Does anyone have any idea why this may be or am I doing something wrong? **Not working:** _filebeat.yml_ ```auto path.home: ${FILEBEAT_HOME:?You need to set the FILEBEAT_HOME environment variable} path.config: ${INSTANCE_HOME:?You need to set the INSTANCE_HOME environment variable} path.logs: ${LOGS_PATH:?You need to set the LOGS_PATH environment variable} path.data: ${INSTANCE_HOME:?You need to set the INSTANCE_HOME environment variable}/data custom.env: ${CUSTOM_ENV:?You need to set the CUSTOM_ENV environment variable} custom.env.enabled: ${CUSTOM_ENV_ENABLED:?You need to set the CUSTOM_ENV_ENABLED environment variable} filebeat.config.prospectors: enabled: true path: configs/*.yml .... further configuration options ommited ``` _config/example1.yml_ ```auto - type: log paths: - '/all_logs/${custom.env}_Nodes/*/appl_logs/*.log' exclude_files: ['\.gz$'] encoding: us-ascii multiline.pattern: '^[0-9]{2}[/\.:][0-9]{2}[/\.:][0-9]{2,4}' multiline.negate: true multiline.match: after enabled: ${custom.env.enabled} fields: custom.env: ${custom.env} fields_under_root: true ignore_older: 48h close_inactive: 24h clean_inactive: 72h ``` **Working:** _filebeat.yml_ ```auto path.home: ${FILEBEAT_HOME:?You need to set the FILEBEAT_HOME environment variable} path.config: ${INSTANCE_HOME:?You need to set the INSTANCE_HOME environment variable} path.logs: ${LOGS_PATH:?You need to set the LOGS_PATH environment variable} path.data: ${INSTANCE_HOME:?You need to set the INSTANCE_HOME environment variable}/data custom.env: ${CUSTOM_ENV:?You need to set the CUSTOM_ENV environment variable} custom.env.enabled: ${CUSTOM_ENV_ENABLED:?You need to set the CUSTOM_ENV_ENABLED environment variable} filebeat.prospectors: - type: log paths: - "/all_logs/${custom.env}_Nodes/*/appl_logs/*.log" exclude_files: ['\.gz$'] encoding: us-ascii multiline.pattern: '^[0-9]{2}[/\.:][0-9]{2}[/\.:][0-9]{2,4}' multiline.negate: true multiline.match: after enabled: ${custom.env.enabled:false} fields: custom.env: ${custom.env} fields_under_root: true ignore_older: 48h close_inactive: 24h clean_inactive: 72h ``` I'm using Version 6.2.2 on Linux. Environment variables are passed to filebeat via -E command line option: "./filebeat -E CUSTOM\_ENV=exampe1 -E CUSTOM\_ENV\_ENABLED=true ..." ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [March 21, 2018, 7:11am UTC](https://discuss.elastic.co/t/environment-variables-not-replaced-in-external-configuration-file/124690/2 "2018-03-21T07:11:21Z")
Could you try what happens if you set custom.env inside `example1.yml`? BTW I reformatted your post a bit to make it more readable. ---
**Author:** ![oetzi](https://avatars.discourse-cdn.com/v4/letter/o/3ec8ea/32.png) [@oetzi](https://discuss.elastic.co/u/oetzi)\ **Post date:** [March 21, 2018, 7:37am UTC](https://discuss.elastic.co/t/environment-variables-not-replaced-in-external-configuration-file/124690/3 "2018-03-21T07:37:34Z")
Unfortunatelly this doesn't work either. Here's my modified example1.yml: ```auto custom.env: ${CUSTOM_ENV:?You need to set the CUSTOM_ENV environment variable} custom.env.enabled: ${CUSTOM_ENV_ENABLED:?You need to set the CUSTOM_ENV_ENABLED environment variable} - type: log paths: - '/all_logs/${custom.env}_Nodes/*/appl_logs/*.log' exclude_files: ['\.gz$'] encoding: us-ascii multiline.pattern: '^[0-9]{2}[/\.:][0-9]{2}[/\.:][0-9]{2,4}' multiline.negate: true multiline.match: after enabled: ${custom.env.enabled} fields: custom.env: ${custom.env} fields_under_root: true ignore_older: 48h close_inactive: 24h clean_inactive: 72h ``` In the log i see the following message: ```auto 2018-03-21T08:31:19.548+0100 ERROR instance/beat.go:667 Exiting: 1 error: invalid config: yaml: line 3: did not find expected key ``` I think it references the following line: ```auto - '/all_logs/${custom.env}_Nodes/*/appl_logs/*.log' ``` ---
**Author:** ![pierhugues](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/pierhugues/32/48383_2.png) [@pierhugues](https://discuss.elastic.co/u/pierhugues)\ **Post date:** [March 26, 2018, 2:05pm UTC](https://discuss.elastic.co/t/environment-variables-not-replaced-in-external-configuration-file/124690/4 "2018-03-26T14:05:13Z")
Hello @oetzi, This is a bug it and this scenario should be supported in external config, I was able to reproduce it with a simple test case and I have created [https://github.com/elastic/beats/issues/6660](https://github.com/elastic/beats/issues/6660) to follow up. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 23, 2018, 2:05pm UTC](https://discuss.elastic.co/t/environment-variables-not-replaced-in-external-configuration-file/124690/5 "2018-04-23T14:05:47Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.