# Filebeat send not all log files

**URL:** <https://discuss.elastic.co/t/filebeat-send-not-all-log-files/158073>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [November 24, 2018, 8:45pm UTC](https://discuss.elastic.co/t/filebeat-send-not-all-log-files/158073 "2018-11-24T20:45:14Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![forthgate](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/forthgate/32/45305_2.png) [@forthgate](https://discuss.elastic.co/u/forthgate)\
**Post date:** [November 24, 2018, 8:45pm UTC](https://discuss.elastic.co/t/filebeat-send-not-all-log-files/158073/1 "2018-11-24T20:45:14Z")

</div>

I got this Filebeat config:

I got this Filebeat config:

```
filebeat.prospectors:
- input_type: log
  paths:
- /var/lib/docker/containers/*/*.log
  document_type: docker
  json.message_key: log
output.elasticsearch:
  hosts: ["localhost:9200"]
setup.kibana:
  host: "localhost"
processors:
- add_docker_metadata:
host: "unix:///var/run/docker.sock"
match_fields: ["system.process.cgroup.id"]
match_pids: ["process.pid", "process.ppid"]
match_source: true
match_source_index: 4
match_short_id: true

```

So as you can see i try to grab all logs from docker containers. Actually i can see only logs only one container, other logs (i got 3 containers) not sending to logstash. Where is my mistake?

---

<div class="post-metadata">

**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\
**Post date:** [November 26, 2018, 8:19am UTC](https://discuss.elastic.co/t/filebeat-send-not-all-log-files/158073/2 "2018-11-26T08:19:26Z")

</div>

The indentation of the configuration seems to be off:

```auto
filebeat.prospectors:
- input_type: log
  enabled: true
  paths:
    - /var/lib/docker/containers/*/*.log
  document_type: docker
  json.message_key: log

output.elasticsearch:
  hosts: ["localhost:9200"]

setup.kibana:
  host: "localhost"

processors:
- add_docker_metadata:
    host: "unix:///var/run/docker.sock"
    match_fields: ["system.process.cgroup.id"]
    match_pids: ["process.pid", "process.ppid"]
    match_source: true
    match_source_index: 4
    match_short_id: true

```

Also the `log` input is not enabled. If the input is not enabled it is not supposed to read events. Have you tried using the `docker` input instead? [https://www.elastic.co/guide/en/beats/filebeat/6.5/filebeat-input-docker.html](https://www.elastic.co/guide/en/beats/filebeat/6.5/filebeat-input-docker.html)  
It contains Docker specific options. It is superior compared to `log` when it comes to Docker logs.

You mentioned that you are sending to Logstash. But in the configuration you have shared the output is Elasticsearch. Are you sure that it is the configuration you are runnning?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 24, 2018, 8:19am UTC](https://discuss.elastic.co/t/filebeat-send-not-all-log-files/158073/3 "2018-12-24T08:19:32Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.