Filebeat dashboard creation via Docker errors on Kibana host

[claflico]

Trying to setup an ELK:6.0.1 stack and *beat:6.0.1 via CentOS 7, Docker 17.09, & Rancher 1.6.12 for a presentation I'm giving this week at a DevOps MeetUp. Am trying to set everything up without having to create custom images or mounting custom config files like I usually have to.

Using the following guides:

elastic/beats-docker

https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-configuration.html
https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-template.html
https://www.elastic.co/guide/en/beats/filebeat/current/load-kibana-dashboards.html

After loading the template I use the following docker-compose config to try and load the dashboard:
filebeat-dashboard:
image: docker.elastic.co/beats/filebeat:6.0.1
environment:
output.logstash.enabled: 'false'
output.elasticsearch.hosts: '["elasticsearch:9200"]'
setup.kibana.host: kibana:5601
depends_on:
- filebeat-template
- kibana
command:
- setup
- --dashboards
labels:
io.rancher.container.start_once: 'true'
io.rancher.container.hostname_override: container_name
io.rancher.container.pull_image: always

When I start the container in Rancher I get the following:
Exiting: Error importing Kibana dashboards: fail to create the Kibana loader: Error creating Kibana client: fail to get the Kibana version:HTTP GET request to /api/status fails: fail to execute the HTTP GET request: Get http://localhost:5601/api/status: dial tcp 127.0.0.1:5601: getsockopt: connection refused. Response: .

Only way I was able to get it to work was create a custom filebeat.yml file on the host and mount it inside the container which is what I am trying to avoid:
filebeat.config:
prospectors:
path: ${path.config}/prospectors.d/.yml
reload.enabled: false
modules:
path: ${path.config}/modules.d/.yml
reload.enabled: false
processors:
#- add_cloud_metadata: #added for forum formatting
output.elasticsearch:
hosts: ['elasticsearch:9200']
setup.kibana:
host: "kibana:5601"

filebeat-dashboard:
image: docker.elastic.co/beats/filebeat:6.0.1
environment:
output.logstash.enabled: 'false'
output.elasticsearch.hosts: '["elasticsearch:9200"]'
setup.kibana.host: kibana:5601
depends_on:
- filebeat-template
- kibana
volumes:
- /mnt/docker/config/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro
command:
- setup
- --dashboards
labels:
io.rancher.container.start_once: 'true'
io.rancher.container.hostname_override: container_name
io.rancher.container.pull_image: always

So specifying "setup.kibana.host: kibana:5601" as an environment variable appears to have no effect.

[exekias (Carlos Pérez Aradros)]

Hi,

You could use -E flag to override configuration settings, so passing -E setup.kibana.host="kibana:5601" should work.

Best regards

[claflico]

Thanks but I already have that specified in my environment via the compose file:

environment:
output.logstash.enabled: 'false'
output.elasticsearch.hosts: '["elasticsearch:9200"]'
setup.kibana.host: kibana:5601

When I run "env" from within the non-setup, long-running container I see setup.kibana.host as an environment variable so it is at least getting set. I can't verify it in the setup container since it exits as soon as the script fails.

[exekias (Carlos Pérez Aradros)]

Sorry I didn't explain myself well, the -E parameter is for filebeat, it allows you to override default settings