# Filebeat too many open files using Docker autodiscover
**URL:** https://discuss.elastic.co/t/filebeat-too-many-open-files-using-docker-autodiscover/205596
**Category:** Beats
**Tags:** docker, filebeat
**Created:** [October 29, 2019, 6:29am UTC](https://discuss.elastic.co/t/filebeat-too-many-open-files-using-docker-autodiscover/205596 "2019-10-29T06:29:32Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@CpuID](https://discuss.elastic.co/u/CpuID)
#### Post date: [October 29, 2019, 6:29am UTC](https://discuss.elastic.co/t/filebeat-too-many-open-files-using-docker-autodiscover/205596/1 "2019-10-29T06:29:33Z")
Seeing this on a machine that runs short lived containers every few minutes, seems to run out of file descriptors due to filebeat hanging onto connections to the Docker daemon socket? File descriptor leak?
Takes a day or two to get to that point, noticed the file handles count increasing steadily throughout.
```auto
root@system:/var/log/filebeat# lsof -n 2>&1 | grep -i filebeat | grep "type=STREAM" | wc -l
73512
root@system:/var/log/filebeat# lsof -n 2>&1 | grep -i filebeat | grep "type=STREAM" | tail -n 5
filebeat 949 32596 root 1019u unix 0xffff8817d69eb400 0t0 52295907 type=STREAM
filebeat 949 32596 root 1020u unix 0xffff88180019f800 0t0 52291899 type=STREAM
filebeat 949 32596 root 1021u unix 0xffff880bfbb6f000 0t0 52291020 type=STREAM
filebeat 949 32596 root 1022u unix 0xffff880b64fd4800 0t0 52265676 type=STREAM
filebeat 949 32596 root 1023u unix 0xffff880b51642800 0t0 52287359 type=STREAM
```
Gets to the point where it can't even rotate out its own logfile at `/var/log/filebeat/filebeat`...
Latest log entry:
```auto
root@system:/var/log/filebeat# tail -n 10 filebeat.1
2019-10-28T07:37:05.434-0400 ERROR [autodiscover] cfgfile/list.go:96 Error creating runner from config: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.22/containers/json?limit=0: dial unix /var/run/docker.sock: socket: too many open files
2019-10-28T07:37:05.434-0400 ERROR [autodiscover] cfgfile/list.go:96 Error creating runner from config: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.22/containers/json?limit=0: dial unix /var/run/docker.sock: socket: too many open files
2019-10-28T07:37:05.435-0400 ERROR [autodiscover] cfgfile/list.go:96 Error creating runner from config: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.22/containers/json?limit=0: dial unix /var/run/docker.sock: socket: too many open files
2019-10-28T07:37:05.435-0400 ERROR [autodiscover] cfgfile/list.go:96 Error creating runner from config: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.22/containers/json?limit=0: dial unix /var/run/docker.sock: socket: too many open files
2019-10-28T07:37:05.436-0400 ERROR [autodiscover] cfgfile/list.go:96 Error creating runner from config: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.22/containers/json?limit=0: dial unix /var/run/docker.sock: socket: too many open files
2019-10-28T07:37:05.436-0400 ERROR [autodiscover] cfgfile/list.go:96 Error creating runner from config: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.22/containers/json?limit=0: dial unix /var/run/docker.sock: socket: too many open files
2019-10-28T07:37:05.437-0400 ERROR [autodiscover] cfgfile/list.go:96 Error creating runner from config: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.22/containers/json?limit=0: dial unix /var/run/docker.sock: socket: too many open files
2019-10-28T07:37:05.437-0400 ERROR [autodiscover] cfgfile/list.go:96 Error creating runner from config: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.22/containers/json?limit=0: dial unix /var/run/docker.sock: socket: too many open files
2019-10-28T07:37:05.437-0400 ERROR [autodiscover] cfgfile/list.go:96 Error creating runner from config: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.22/containers/json?limit=0: dial unix /var/run/docker.sock: socket: too many open files
2019-10-28T07:37:05.438-0400 ERROR [autodiscover] cfgfile/list.go:96 Error creating runner from config: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.22/containers/json?limit=0: dial unix /var/run/docker.sock: socket: too many open files
```
Tested on 6.8.3 + 7.4.1 so far. Config snippet on 6.8.3:
```auto
filebeat.autodiscover:
providers:
- type: docker
cleanup_timeout: 30s
templates:
- condition:
regexp:
docker.container.name: ".*"
config:
- type: docker
containers.ids:
- "${data.docker.container.id}"
processors:
- add_docker_metadata: ~
- decode_json_fields:
fields: ["message"]
target: "message_json"
.....
output:
redis:
hosts: ["ourredisserver"]
port: 6379
key: "logstash-somesuffix"
datatype: "list"
timeout: 5
reconnect_interval: 1
```
---
### Author:  [@CpuID](https://discuss.elastic.co/u/CpuID)
#### Post date: [October 29, 2019, 11:01pm UTC](https://discuss.elastic.co/t/filebeat-too-many-open-files-using-docker-autodiscover/205596/2 "2019-10-29T23:01:27Z")
Steps to reproduce: [https://github.com/CpuID/filebeat-docker-socket-leak](https://github.com/CpuID/filebeat-docker-socket-leak)
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 26, 2019, 11:01pm UTC](https://discuss.elastic.co/t/filebeat-too-many-open-files-using-docker-autodiscover/205596/3 "2019-11-26T23:01:32Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.