# Specifying complex objects in environment variable doesn't work with Logstash output

**URL:** https://discuss.elastic.co/t/specifying-complex-objects-in-environment-variable-doesnt-work-with-logstash-output/142824
**Category:** Beats
**Tags:** filebeat
**Created:** [August 2, 2018, 11:34pm UTC](https://discuss.elastic.co/t/specifying-complex-objects-in-environment-variable-doesnt-work-with-logstash-output/142824 "2018-08-02T23:34:23Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![Alsheh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alsheh/32/36799_2.png) [@Alsheh](https://discuss.elastic.co/u/Alsheh)
#### Post date: [August 2, 2018, 11:34pm UTC](https://discuss.elastic.co/t/specifying-complex-objects-in-environment-variable-doesnt-work-with-logstash-output/142824/1 "2018-08-02T23:34:23Z")

</div>

Hi, I tried to follow the example provided in the filebeat docs to [specify complex objects in environment variables](https://www.elastic.co/guide/en/beats/filebeat/6.3/using-environ-vars.html#_specify_complex_objects_in_environment_variables) (array of LS hosts in my case) for logstash output but filebeat exits with the following error message:

```auto
Exiting: error initializing publisher: missing field accessing 'output.logstash.hosts' (source:'filebeat.yml')

```

Env variable is defined in docker-compose.yml:

```auto
LS_HOSTS="logstash:5044,logstash2:5044"

```

filebeat.yml:

```auto
...
output.logstash:
  hosts: '${LS_HOSTS}'

```

Is this feature exclusive for Elasticsearch output, a bug, or am I doing something wrong here?

Thanks!

---

<div class="post-metadata">

### Author: ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)
#### Post date: [August 3, 2018, 7:06am UTC](https://discuss.elastic.co/t/specifying-complex-objects-in-environment-variable-doesnt-work-with-logstash-output/142824/2 "2018-08-03T07:06:04Z")

</div>

Your config seems correct. Do you get the same error when running `filebeat` on the host without any containerization?

---

<div class="post-metadata">

### Author: ![Alsheh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alsheh/32/36799_2.png) [@Alsheh](https://discuss.elastic.co/u/Alsheh)
#### Post date: [August 3, 2018, 5:03pm UTC](https://discuss.elastic.co/t/specifying-complex-objects-in-environment-variable-doesnt-work-with-logstash-output/142824/3 "2018-08-03T17:03:46Z")

</div>

I get the same error without containerization:

Defining env variable:

```auto
export LS_HOSTS="logstash:5044,logstash2:5044"

```

filebeat.yml

```auto
...
output.logstash:
  hosts: '${LS_HOSTS}'
  loadbalance: true
  index: filebeat

```

Logs after running Filebeat

```auto
➜ filebeat-6.3.2-linux-x86_64 ✗ sudo ./filebeat -e -c filebeat.yml -d "publish"
...
2018-08-03T16:57:56.794Z	INFO	instance/beat.go:225	Setup Beat: filebeat; Version: 6.3.2
2018-08-03T16:57:56.795Z	INFO	instance/beat.go:275	filebeat stopped.
2018-08-03T16:57:56.795Z	ERROR	instance/beat.go:691	Exiting: error initializing publisher: missing field accessing 'output.logstash.hosts' (source:'filebeat.yml')
Exiting: error initializing publisher: missing field accessing 'output.logstash.hosts' (source:'filebeat.yml')

```

---

<div class="post-metadata">

### Author: ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)
#### Post date: [August 6, 2018, 9:05am UTC](https://discuss.elastic.co/t/specifying-complex-objects-in-environment-variable-doesnt-work-with-logstash-output/142824/4 "2018-08-06T09:05:53Z")

</div>

Could you please share the full configuration file? It's possible that there is a whitespace error which messes up the parsing.

---

<div class="post-metadata">

### Author: ![Alsheh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alsheh/32/36799_2.png) [@Alsheh](https://discuss.elastic.co/u/Alsheh)
#### Post date: [August 6, 2018, 6:48pm UTC](https://discuss.elastic.co/t/specifying-complex-objects-in-environment-variable-doesnt-work-with-logstash-output/142824/5 "2018-08-06T18:48:30Z")

</div>

@kvch, here is a simplified version of the config file that produces the same error:

```auto
filebeat.inputs:
- type: log
  paths:
    - /var/log/*.log

output.logstash:
  hosts: '${LS_HOSTS}'

```

env variable:

```auto
export LS_HOSTS="10.45.3.2:9220,10.45.3.1:9230"

```

```auto
➜ filebeat-6.3.2 sudo ./filebeat -e -c filebeat.yml -d "publish"
...
2018-08-06T11:35:54.023-0700	INFO	instance/beat.go:225	Setup Beat: filebeat; Version: 6.3.2
2018-08-06T11:35:54.024-0700	INFO	instance/beat.go:275	filebeat stopped.
2018-08-06T11:35:54.024-0700	ERROR	instance/beat.go:691	Exiting: error initializing publisher: missing field accessing 'output.logstash.hosts' (source:'filebeat.yml')
Exiting: error initializing publisher: missing field accessing 'output.logstash.hosts' (source:'filebeat.yml')

```

Filebeat doesn't seems to be able to read env variable because I get the same error message when I simply put an empty string in the host field but works as expected when I hardcode the host in the config file! Also, I am doing the testing without any containerization and I run Filebeat in the same terminal session where I defined the the env variable.

---

<div class="post-metadata">

### Author: ![Alsheh](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alsheh/32/36799_2.png) [@Alsheh](https://discuss.elastic.co/u/Alsheh)
#### Post date: [August 6, 2018, 10:50pm UTC](https://discuss.elastic.co/t/specifying-complex-objects-in-environment-variable-doesnt-work-with-logstash-output/142824/6 "2018-08-06T22:50:55Z")

</div>

Problem solved!

I used `sudo` when I ran filebeat which does not preserve env variables for security purposes. [This post helped](https://discuss.elastic.co/t/env-variables-in-not-working-in-custom-fields-in-filebeat/101374/4?u=alsheh).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [September 3, 2018, 10:50pm UTC](https://discuss.elastic.co/t/specifying-complex-objects-in-environment-variable-doesnt-work-with-logstash-output/142824/7 "2018-09-03T22:50:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
