Filebeat fails with connecting to Kibana over GET 127.0.0.1:5601

[makrellen]

Description of problem

I'm trying to get Filebeat to work on my Ubuntu VPS via docker-compose and I've set it up behind my caddy-reverse proxy and configured everything as I should according to the Filebeat docs, but I keep getting the same errror attached below. All my other docker-containers behind the reverse-proxy works fine, so my stack-setup is working.

error:

Exiting: error connecting to Kibana: fail to get the Kibana version: HTTP GET request to http://localhost:5601/api/status fails: fail to execute the HTTP GET request: Get "http://localhost:5601/api/status": dial tcp 127.0.0.1:5601: connect: connection refused. Response: .

docker-compose.yml

version: '3.8'

services:
filebeat:
image: docker.elastic.co/beats/filebeat:7.12.0
container_name: filebeat
expose:
- 5601
volumes:
- /home/ubuntu/filebeat/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro
- /var/run/docker.sock:/var/run/docker.sock
- /var/lib/docker/containers:/var/lib/docker/containers:ro
networks:
- main
restart: always

networks:
main:
external: true

Filebeat.yml

filebeat.config:
modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: true

setup.dashboards.enabled: true
setup.kibana.host: "http://localhost:5601"

filebeat.inputs:

• type: docker
  containers.ids: '*'

output.console:
pretty: true

Caddyfile reverse proxy config

filebeat.mydomain.me {
reverse_proxy filebeat:5601
}

[legoguy1000 (Alex)]

What port is the reverse proxy listening on? Is that running in docker too?

[makrellen]

Caddy is running as docker too on the same docker-network as every other docker container.
Its listening on 80:80 and 443:443

[legoguy1000 (Alex)]

Also is the error you posted from Filebeat? Just to confirm, is that your entire docker-compose? If so, it looks like you are only running filebeat, you don't have a kibana container. You appear to be exposing port 5601 on the Filebeat container.

[makrellen]

Yes the error comes from the filebeat container when running docker-compose up -d and then docker logs filebeat to read the logs of the container.

Yes its also the entire docker-compose. My suspicsion is that Kibana is not running and not part of the Filebeat image, but I understood it as Kibana is built in to Filebeat, so one could view the logs in a dashboard

[legoguy1000 (Alex)]

no kibana is its own application. its available in its own container kibana | Docker @ Elastic.

[makrellen]

I'm still stuck with the same problem, but when I enter the kibana frontend through the browser I see this message on the page: Kibana server is not ready yet.

This is my new docker-compose file

version: '3.8'

services:
filebeat:
image: docker.elastic.co/beats/filebeat:7.12.0
container_name: filebeat
volumes:
- /home/ubuntu/filebeat/filebeat.yml:/usr/share/filebeat/filebeat.yml
- /var/run/docker.sock:/var/run/docker.sock
- /var/lib/docker/containers:/var/lib/docker/containers:ro
depends_on:
- kibana
networks:
- main
restart: always

kibana:
image: docker.elastic.co/kibana/kibana:7.12.0
container_name: kibana
expose:
- 5601
networks:
- main
restart: always

networks:
main:
external: true

The error

2021-04-08T18:49:10.054Z ERROR instance/beat.go:971 Exiting: error connecting to Kibana: fail to get the Kibana version: HTTP GET request to http://0.0.0.0:5601/api/status fails: fail to execute the HTTP GET request: Get "http://0.0.0.0:5601/api/status": dial tcp 0.0.0.0:5601: connect: connection refused. Response: .

Exiting: error connecting to Kibana: fail to get the Kibana version: HTTP GET request to http://0.0.0.0:5601/api/status fails: fail to execute the HTTP GET request: Get "http://0.0.0.0:5601/api/status": dial tcp 0.0.0.0:5601: connect: connection refused. Response: .

filebeat.yml

filebeat.config:
modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false

setup.dashboards.enabled: true
setup.kibana.host: "0.0.0.0:5601"

filebeat.inputs:

• type: docker
  containers.ids: '*'

output.console:
pretty: true

[legoguy1000 (Alex)]

Do u have a elasicsearch container/server running?? that's where teh data is actually stored. Kibana is just a UI for it.