# Harvester not harvesting Autodiscover **URL:** \ **Category:** Beats\ **Tags:** docker, filebeat\ **Created:** [February 5, 2021, 1:54pm UTC](https://discuss.elastic.co/t/harvester-not-harvesting-autodiscover/263392 "2021-02-05T13:54:40Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Alexis\_Oviedo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexis_oviedo/32/77531_2.png) [@Alexis\_Oviedo](https://discuss.elastic.co/u/Alexis_Oviedo)\ **Post date:** [February 5, 2021, 1:54pm UTC](https://discuss.elastic.co/t/harvester-not-harvesting-autodiscover/263392/1 "2021-02-05T13:54:40Z")
I'm trying to use filebeat docker but it's not harvesting files. Autodiscover seems to work because found the logs I want but it's not harvesting them. This is my filebeat.yml: > filebeat.autodiscover: > > providers: > > ``` > - type: docker > > templates: > > - condition: > > contains: > > docker.container.image: nginx > > config: > > - type: container > > paths: > > - /var/lib/docker/containers/${data.docker.container.id}/*.log > > exclude_lines: ["^\\s+[\\-`('.|_]"] # drop asciiart lines > > ``` > > output.elasticsearch: > > hosts: '192.168.141.109:30920' and my output log: > 2021-02-05T13:48:23.936Z INFO instance/beat.go:645 Home path: [/usr/share/filebeat] Config path: [/usr/share/filebeat] Data path: [/usr/share/filebeat/data] Logs path: [/usr/share/filebeat/logs] > 2021-02-05T13:48:23.939Z INFO instance/beat.go:653 Beat ID: 8531f8b3-2e5e-4b76-b228-ac638bc39a69 > 2021-02-05T13:48:23.940Z INFO [seccomp] seccomp/seccomp.go:124 Syscall filter successfully installed > 2021-02-05T13:48:23.940Z INFO [beat] instance/beat.go:981 Beat info {"system\_info": {"beat": {"path": {"config": "/usr/share/filebeat", "data": "/usr/share/filebeat/data", "home": "/usr/share/filebeat", "logs": "/usr/share/filebeat/logs"}, "type": "filebeat", "uuid": "8531f8b3-2e5e-4b76-b228-ac638bc39a69"}}} > 2021-02-05T13:48:23.940Z INFO [beat] instance/beat.go:990 Build info {"system\_info": {"build": {"commit": "aacf9ecd9c494aa0908f61fbca82c906b16562a8", "libbeat": "7.10.2", "time": "2021-01-12T23:11:24.000Z", "version": "7.10.2"}}} > 2021-02-05T13:48:23.940Z INFO [beat] instance/beat.go:993 Go runtime info {"system\_info": {"go": {"os":"linux","arch":"amd64","max\_procs":2,"version":"go1.14.12"}}} > 2021-02-05T13:48:23.943Z INFO [beat] instance/beat.go:997 Host info {"system\_info": {"host": {"architecture":"x86\_64","boot\_time":"2021-02-05T10:36:38Z","containerized":true,"name":"247dcdf397d7","ip":["127.0.0.1/8","172.17.0.3/16"],"kernel\_version":"5.4.0-53-generic","mac":["02:42:ac:11:00:03"],"os":{"family":"redhat","platform":"centos","name":"CentOS Linux","version":"7 (Core)","major":7,"minor":9,"patch":2009,"codename":"Core"},"timezone":"UTC","timezone\_offset\_sec":0,"id":"3d458769eadb4ca1ac689c5da40af2bf"}}} > 2021-02-05T13:48:23.944Z INFO [beat] instance/beat.go:1026 Process info {"system\_info": {"process": {"capabilities": {"inheritable":["chown","dac\_override","fowner","fsetid","kill","setgid","setuid","setpcap","net\_bind\_service","net\_raw","sys\_chroot","mknod","audit\_write","setfcap"],"permitted":null,"effective":null,"bounding":["chown","dac\_override","fowner","fsetid","kill","setgid","setuid","setpcap","net\_bind\_service","net\_raw","sys\_chroot","mknod","audit\_write","setfcap"],"ambient":null}, "cwd": "/usr/share/filebeat", "exe": "/usr/share/filebeat/filebeat", "name": "filebeat", "pid": 20, "ppid": 19, "seccomp": {"mode":"filter","no\_new\_privs":true}, "start\_time": "2021-02-05T13:48:23.400Z"}}} > 2021-02-05T13:48:23.944Z INFO instance/beat.go:299 Setup Beat: filebeat; Version: 7.10.2 > 2021-02-05T13:48:23.944Z INFO [index-management] idxmgmt/std.go:184 Set output.elasticsearch.index to 'filebeat-7.10.2' as ILM is enabled. > 2021-02-05T13:48:23.944Z INFO eslegclient/connection.go:99 elasticsearch url: [http://192.168.141.109:30920](http://192.168.141.109:30920) > 2021-02-05T13:48:23.945Z INFO [publisher] pipeline/module.go:113 Beat name: 247dcdf397d7 > 2021-02-05T13:48:23.947Z INFO [monitoring] log/log.go:118 Starting metrics logging every 30s > 2021-02-05T13:48:23.947Z INFO instance/beat.go:455 filebeat start running. > 2021-02-05T13:48:23.958Z INFO memlog/store.go:119 Loading data file of '/usr/share/filebeat/data/registry/filebeat' succeeded. Active transaction id=0 > 2021-02-05T13:48:23.959Z INFO memlog/store.go:124 Finished loading transaction log file for '/usr/share/filebeat/data/registry/filebeat'. Active transaction id=0 > 2021-02-05T13:48:23.960Z INFO [registrar] registrar/registrar.go:109 States Loaded from registrar: 0 > 2021-02-05T13:48:23.961Z INFO [crawler] beater/crawler.go:71 Loading Inputs: 0 > 2021-02-05T13:48:23.961Z INFO [crawler] beater/crawler.go:108 Loading and starting Inputs completed. Enabled inputs: 0 > 2021-02-05T13:48:24.002Z INFO [autodiscover] autodiscover/autodiscover.go:113 Starting autodiscover manager > 2021-02-05T13:48:24.004Z INFO log/input.go:157 Configured paths: [/var/lib/docker/containers/67e21b763a8c830c504933afb302c2c1237997b5883f2cfd69d75cf37ff254c7/_.log] > 2021-02-05T13:48:24.005Z INFO log/input.go:157 Configured paths: [/var/lib/docker/containers/67e21b763a8c830c504933afb302c2c1237997b5883f2cfd69d75cf37ff254c7/_.log] > 2021-02-05T13:48:53.965Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cgroup":{"cpu":{"id":"247dcdf397d729708a6c44e2113988babd67fdb774dc6e02d0a8ba4c30f6c756"},"cpuacct":{"id":"247dcdf397d729708a6c44e2113988babd67fdb774dc6e02d0a8ba4c30f6c756"},"memory":{"id":"247dcdf397d729708a6c44e2113988babd67fdb774dc6e02d0a8ba4c30f6c756"}},"cpu":{"system":{"ticks":130,"time":{"ms":133}},"total":{"ticks":290,"time":{"ms":297},"value":290},"user":{"ticks":160,"time":{"ms":164}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":10},"info":{"ephemeral\_id":"43af8240-de5b-4074-b307-8d223898ade4","uptime":{"ms":30154}},"memstats":{"gc\_next":16240032,"memory\_alloc":14824312,"memory\_total":44533136,"rss":67768320},"runtime":{"goroutines":32}},"filebeat":{"harvester":{"open\_files":0,"running":0}},"libbeat":{"config":{"module":{"running":1,"starts":1}},"output":{"type":"elasticsearch"},"pipeline":{"clients":1,"events":{"active":0}}},"registrar":{"states":{"current":0}},"system":{"cpu":{"cores":2},"load":{"1":0.53,"15":0.34,"5":0.55,"norm":{"1":0.265,"15":0.17,"5":0.275}}}}}} ---
**Author:** ![kvch](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kvch/32/72058_2.png) [@kvch](https://discuss.elastic.co/u/kvch)\ **Post date:** [February 5, 2021, 4:06pm UTC](https://discuss.elastic.co/t/harvester-not-harvesting-autodiscover/263392/2 "2021-02-05T16:06:21Z")
Could you please share debug logs of Filebeat? ---
**Author:** ![Alexis\_Oviedo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexis_oviedo/32/77531_2.png) [@Alexis\_Oviedo](https://discuss.elastic.co/u/Alexis_Oviedo)\ **Post date:** [February 8, 2021, 8:50am UTC](https://discuss.elastic.co/t/harvester-not-harvesting-autodiscover/263392/3 "2021-02-08T08:50:25Z")
Here you are [Filebeat Debug Log](https://anonymousfiles.io/ztBdXok2/) ---
**Author:** ![Alexis\_Oviedo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexis_oviedo/32/77531_2.png) [@Alexis\_Oviedo](https://discuss.elastic.co/u/Alexis_Oviedo)\ **Post date:** [February 8, 2021, 10:25am UTC](https://discuss.elastic.co/t/harvester-not-harvesting-autodiscover/263392/4 "2021-02-08T10:25:04Z")
I tried same config on same machine but not dockerizing filebeat and it worked. I launch docker like this --volume="/var/lib/docker/containers:/var/lib/docker/containers:ro" --volume="/var/run/docker.sock:/var/run/docker.sock:ro" maybe am I missing any volume? ---
**Author:** ![Alexis\_Oviedo](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/alexis_oviedo/32/77531_2.png) [@Alexis\_Oviedo](https://discuss.elastic.co/u/Alexis_Oviedo)\ **Post date:** [February 8, 2021, 11:10am UTC](https://discuss.elastic.co/t/harvester-not-harvesting-autodiscover/263392/5 "2021-02-08T11:10:24Z")
I got it, was a problem with permissions on docker logs. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 8, 2021, 1:11pm UTC](https://discuss.elastic.co/t/harvester-not-harvesting-autodiscover/263392/6 "2021-03-08T13:11:17Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.