Docker Beat with Docker ELK

Elastic Stack Beats
1

Hi,
i have a docker container that is running filebeat container in on one host with an webpp an his logfile, and the docker ELK on other.

i configure filebeat.yml :

filebeat.prospectors:
- input_type: log
  paths:
    - /logs/*.log 
output.logstash:
  hosts: ["172.18.0.4:5000"]

IP address is the ip of the container that run logstash

docker ps: 
CONTAINER ID        IMAGE                     COMMAND                  CREATED             STATUS              PORTS                                            NAMES
b725172b6bf7        filebeat_filebeat         "filebeat -e"            8 minutes ago       Up 8 minutes        5000/tcp, 5044/tcp                               filebeat_filebeat_1
f7c1df5b872c        dockerelk_logstash        "/usr/local/bin/do..."   About an hour ago   Up About an hour    5044/tcp, 0.0.0.0:5000->5000/tcp, 9600/tcp       dockerelk_logstash_1
9be1eb9a3599        dockerelk_kibana          "/bin/sh -c /usr/l..."   About an hour ago   Up About an hour    0.0.0.0:5601->5601/tcp                           dockerelk_kibana_1
7fc7459e68aa        dockerelk_elasticsearch   "/bin/bash bin/es-..."   About an hour ago   Up About an hour    0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp   dockerelk_elasticsearch_1
64c46f119cb9        phpmyadmin/phpmyadmin     "/run.sh phpmyadmin"     26 hours ago        Up 26 hours         0.0.0.0:8080->80/tcp                             vtiger_vtiger_phpmyadmin_1

logstash has 5000:5000 port mapped and 5044 only exposed
With docker inspect i took the ip of logstash that i use in the filebeat.yml

In the filebeat container i run the command:

./filebeat -e -c filebeat.yml -d "publish"

and i have this response:

2017/09/13 10:12:42.605212 output.go:109: DBG  output worker: publish 2039 events
2017/09/13 10:13:12.492870 metrics.go:39: INFO Non-zero metrics in the last 30s: filebeat.harvester.open_files=1 filebeat.harvester.running=1 filebeat.harvester.started=1 libbeat.publisher.published_events=2039
2017/09/13 10:13:12.605776 single.go:140: ERR Connecting error publishing events (retrying): dial tcp 172.18.0.4:5000: i/o timeout
2017/09/13 10:13:42.492704 metrics.go:34: INFO No non-zero metrics in the last 30s
2017/09/13 10:13:43.606680 single.go:140: ERR Connecting error publishing events (retrying): dial tcp 172.18.0.4:5000: i/o timeout
2017/09/13 10:14:12.492595 metrics.go:34: INFO No non-zero metrics in the last 30s
2017/09/13 10:14:15.607513 single.go:140: ERR Connecting error publishing events (retrying): dial tcp 172.18.0.4:5000: i/o timeout

in the logstash/pipiline/logstash.conf with this config:

input {
	tcp {
		port => 5000
	}
}

output {
	elasticsearch {
		hosts => "elasticsearch:9200"
	}
}

but in filebeat has error and of course logstash don't pass anything to logstash...
Please help me

1 Like
2

localhost inside of the Filebeat container refers to the container itself and not the parent host running the container. You either need to

  • specify the IP/hostname of the docker host machine (this is possible because you have forwarded port 5000 from the LS container)
  • or link the filebeat and logstash containers such that you can directly communicate to the LS container from the FB container (this is probably how LS, ES, and Kibana are all communicating)
3

BTW You might want to have a look at the elastic/stack-docker demo project. It uses docker-compose to run the whole Elastic stack.

1 Like
4

sorry this is an old screenshot

5

i saw it i have the elk stack up

an i installed filebeat on vtiger host to load the log file through Logstash

6

This topic was automatically closed after 21 days. New replies are no longer allowed.