# Docker\[ELK+Filebeat\] =\> No logs

**URL:** <https://discuss.elastic.co/t/docker-elk-filebeat-no-logs/40946>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [February 4, 2016, 10:05am UTC](https://discuss.elastic.co/t/docker-elk-filebeat-no-logs/40946 "2016-02-04T10:05:50Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ragnar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ragnar/32/7610_2.png) [@Ragnar](https://discuss.elastic.co/u/Ragnar)\
**Post date:** [February 4, 2016, 10:05am UTC](https://discuss.elastic.co/t/docker-elk-filebeat-no-logs/40946/1 "2016-02-04T10:05:50Z")

</div>

Hi everyone,

I'm learning how to use ELK so I decided to do my own Hello world.  
I planned to make some graph with my syslog/auth.log

I also decided to put every services in a Docker's container.

I set up everything the way I found online but I still cannot get any data in ES.

I give you my conf files to, please, help me find why.

docker-compose.yml:

```
elasticsearch:
  image: elasticsearch:latest
  command: elasticsearch -Des.network.host=0.0.0.0
  ports:
    - "9200:9200"
    - "9300:9300"
logstash:
  image: logstash:latest
  command: logstash -f /etc/logstash/conf.d/logstash.conf
  volumes:
    - ./logstash/config:/etc/logstash/conf.d
  ports:
    - "5000:5000"
  links:
    - elasticsearch
kibana:
  build: kibana/
  volumes:
    - ./kibana/config/kibana.yml:/opt/kibana/config/kibana.yml
  ports:
    - "5601:5601"
  links:
    - elasticsearch

```

logstash.conf:

```
input {
        beats {
                port => 5000
                type => "logs"
        }
}

filter {
        if [type] == "syslog" {
                grok {
                        match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
                        add_field => ["received_at", "%{@timestamp}"]
                        add_field => ["received_from", "%{host}"]
                }
                syslog_pri { }
                date {
                        match => ["syslog_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]
                }
        }
}

output {
        elasticsearch {
                hosts => "elasticsearch:9200"
        }
}

```

filebeat.yml:

```
filebeat:
  # List of prospectors to fetch data.
  prospectors:
    # Each - is a prospector. Below are the prospector specific configurations
    -
      # Paths that should be crawled and fetched. Glob based paths.
      # For each file found under this path, a harvester is started.
      paths:
        - "/var/log/*.log"
        - "/var/log/syslog"
      # - c:\programdata\elasticsearch\logs\*

      # Type of the files. Based on this the way the file is read is decided.
      # The different types cannot be mixed in one prospector
      #
      # Possible options are:
      # * log: Reads every line of the log file (default)
      # * stdin: Reads the standard in
      input_type: log

      document_type: syslog

output:
  logstash:
    hosts:["localhost:5000"]

```

Everything is setup on the same machine.  
I look at the conf files many time and I cannot figure it out.  
I think filebeat don't send correctly the data to logstash. I'm new to this and I don't know how to debug this by myself.

Thanks.

---

<div class="post-metadata">

**Author:** ![Ragnar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ragnar/32/7610_2.png) [@Ragnar](https://discuss.elastic.co/u/Ragnar)\
**Post date:** [February 4, 2016, 11:18am UTC](https://discuss.elastic.co/t/docker-elk-filebeat-no-logs/40946/2 "2016-02-04T11:18:44Z")

</div>

I put forward in the resolution of my problem.

`docker ps -a` shows me that filebeat exit after running.  
So obviously I don't have anything to index.

I'm looking forward this way.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [February 4, 2016, 4:54pm UTC](https://discuss.elastic.co/t/docker-elk-filebeat-no-logs/40946/3 "2016-02-04T16:54:47Z")

</div>

any log output from filebeat? Have you tried to start filebeat with debug logging enalbed?

---

<div class="post-metadata">

**Author:** ![Ragnar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ragnar/32/7610_2.png) [@Ragnar](https://discuss.elastic.co/u/Ragnar)\
**Post date:** [February 5, 2016, 10:04am UTC](https://discuss.elastic.co/t/docker-elk-filebeat-no-logs/40946/4 "2016-02-05T10:04:16Z")

</div>

I'm trying to find where are the filebeat log in the document and also how to get it from a docker container. I think I have to mount a volume to save the data from filebeat into my server. Because of the volatile type of docker data.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [February 5, 2016, 12:06pm UTC](https://discuss.elastic.co/t/docker-elk-filebeat-no-logs/40946/5 "2016-02-05T12:06:05Z")

</div>

Start filebeat with `-e` flag to have filebeat log to stderr. You can use `docker logs <container_id/name>` to show all filebeat log output.

---

<div class="post-metadata">

**Author:** ![Ragnar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ragnar/32/7610_2.png) [@Ragnar](https://discuss.elastic.co/u/Ragnar)\
**Post date:** [February 5, 2016, 12:13pm UTC](https://discuss.elastic.co/t/docker-elk-filebeat-no-logs/40946/6 "2016-02-05T12:13:19Z")

</div>

Thank you !

I got :

`Loading config file error: Failed to read /etc/filebeat/filebeat.yml: read /etc/filebeat/filebeat.yml: is a directory. Exiting.`

How can be this path a directory ?

I launch my service with :

`sudo docker run -d -v filebeat.yml:/etc/filebeat/filebeat.yml --name filebeat prima/filebeat -e`

---

<div class="post-metadata">

**Author:** ![Ragnar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ragnar/32/7610_2.png) [@Ragnar](https://discuss.elastic.co/u/Ragnar)\
**Post date:** [February 5, 2016, 5:45pm UTC](https://discuss.elastic.co/t/docker-elk-filebeat-no-logs/40946/7 "2016-02-05T17:45:05Z")

</div>

Ok I moved forward.

The main reason why file exited after launch was because of a typo in the conf file.

yml files are like python, they use "white space indentation".

Instead of :

```
output:
  logstash:
    hosts:["localhost:5000"]

```

I had to add a space like this

```
output:
  logstash:
    hosts : ["localhost:5000"]

```

now all my services are in their containers. But I still don't have any output data.

Even with a `nc localhost 5000 < /var/log/syslog`

 ![](https://us1.discourse-cdn.com/elastic/original/2X/b/b82a48533785e06cd12faf5fa44bfde189c6798c.png)

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [February 9, 2016, 7:53am UTC](https://discuss.elastic.co/t/docker-elk-filebeat-no-logs/40946/8 "2016-02-09T07:53:07Z")

</div>

Can you share some log debug output from filebeat, means running it with `-e -d "*"`

---

<div class="post-metadata">

**Author:** ![Ragnar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ragnar/32/7610_2.png) [@Ragnar](https://discuss.elastic.co/u/Ragnar)\
**Post date:** [February 11, 2016, 1:10pm UTC](https://discuss.elastic.co/t/docker-elk-filebeat-no-logs/40946/9 "2016-02-11T13:10:46Z")

</div>

Thanks @ruflin

The only suspect log is :  
`WARN DNS lookup failure "logstash": lookup logstash on 8.8.8.8:53: no such host`

for the all logs : [http://pastebin.com/kJjeMUbs](http://pastebin.com/kJjeMUbs)

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [February 11, 2016, 1:47pm UTC](https://discuss.elastic.co/t/docker-elk-filebeat-no-logs/40946/10 "2016-02-11T13:47:38Z")

</div>

What is strange is that in your config you have localhost for LS and here it tries to look up "logstash" host?

---

<div class="post-metadata">

**Author:** ![diginoise](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/diginoise/32/9487_2.png) [@diginoise](https://discuss.elastic.co/u/diginoise)\
**Post date:** [April 29, 2016, 2:48pm UTC](https://discuss.elastic.co/t/docker-elk-filebeat-no-logs/40946/11 "2016-04-29T14:48:42Z")

</div>

Why not remove **logstash** from the equation since Filebeat can write natively to Elasticsearch?

That would help no narrow the problem.

Docker when containers are linked uses the destination container name as a hostname, but you don't seem to link any containers to **logstash** container.

BTW, Which container does **Filebeat** run in? It is not clear to me...

Also your logs in **Pastebin** don't show anything abnormal. The log files watched are too old for Filebeat to care (\> 24h) or the files didn't change within last 24h.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 5, 2017, 9:52pm UTC](https://discuss.elastic.co/t/docker-elk-filebeat-no-logs/40946/12 "2017-07-05T21:52:39Z")

</div>