# Syslog harvester does not start. Running Filebeat 7.3.1 on Docker

**URL:** https://discuss.elastic.co/t/syslog-harvester-does-not-start-running-filebeat-7-3-1-on-docker/201398
**Category:** Beats
**Tags:** docker, filebeat
**Created:** [September 27, 2019, 12:57pm UTC](https://discuss.elastic.co/t/syslog-harvester-does-not-start-running-filebeat-7-3-1-on-docker/201398 "2019-09-27T12:57:37Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![glauber.ferreira](https://avatars.discourse-cdn.com/v4/letter/g/b2d939/32.png) [@glauber.ferreira](https://discuss.elastic.co/u/glauber.ferreira)
#### Post date: [September 27, 2019, 12:57pm UTC](https://discuss.elastic.co/t/syslog-harvester-does-not-start-running-filebeat-7-3-1-on-docker/201398/1 "2019-09-27T12:57:37Z")

</div>

I am running Elastic Stack 7.3.1 on Docker (Elastic, Kibana, Metricbeat, Filebeat). Filebeat harvesting are being started successfully for Docker container files. But, harvesting for Syslog module are not being started.

This is my filebeat log file. There are four lines starting with _Harvester started for file_; all of them related to docker containers. But there is not a line indicating the start of harvester for syslog files.

```
2019-09-27T12:00:23.788Z INFO instance/beat.go:606 Home path: [/usr/share/filebeat] Config path: [/usr/share/filebeat] Data path: [/usr/share/filebeat/data] Logs path: [/usr/share/filebeat/logs]
2019-09-27T12:00:23.789Z INFO instance/beat.go:614 Beat ID: ef9dd9a2-06ad-4358-88e8-cd91c34a9a08
2019-09-27T12:00:23.799Z INFO [seccomp] seccomp/seccomp.go:124 Syscall filter successfully installed
2019-09-27T12:00:23.835Z INFO instance/beat.go:292 Setup Beat: filebeat; Version: 7.3.1
2019-09-27T12:00:23.835Z INFO [index-management] idxmgmt/std.go:178 Set output.elasticsearch.index to 'filebeat-7.3.1' as ILM is enabled.
2019-09-27T12:00:23.836Z INFO elasticsearch/client.go:170 Elasticsearch url: https://192.168.1.6:9200
2019-09-27T12:00:23.836Z INFO [publisher] pipeline/module.go:97 Beat name: vm-07
2019-09-27T12:00:23.902Z INFO beater/filebeat.go:92 Enabled modules/filesets: system (syslog), ()
2019-09-27T12:00:23.902Z INFO instance/beat.go:421 filebeat start running.
2019-09-27T12:00:23.902Z INFO [monitoring] log/log.go:118 Starting metrics logging every 30s
2019-09-27T12:00:23.903Z INFO registrar/migrate.go:104 No registry home found. Create: /usr/share/filebeat/data/registry/filebeat
2019-09-27T12:00:23.908Z INFO registrar/migrate.go:112 Initialize registry meta file
2019-09-27T12:00:23.910Z INFO registrar/registrar.go:108 No registry file found under: /usr/share/filebeat/data/registry/filebeat/data.json. Creating a new registry file.
2019-09-27T12:00:23.911Z INFO registrar/registrar.go:145 Loading registrar data from /usr/share/filebeat/data/registry/filebeat/data.json
2019-09-27T12:00:23.911Z INFO registrar/registrar.go:152 States Loaded from registrar: 0
2019-09-27T12:00:23.911Z INFO crawler/crawler.go:72 Loading Inputs: 1
2019-09-27T12:00:23.918Z INFO log/input.go:148 Configured paths: [/var/log/messages* /var/log/syslog*]
2019-09-27T12:00:23.918Z INFO input/input.go:114 Starting input of type: log; ID: 6766254637019430523
2019-09-27T12:00:23.918Z INFO crawler/crawler.go:106 Loading and starting Inputs completed. Enabled inputs: 1
2019-09-27T12:00:23.918Z WARN [cfgwarn] docker/docker.go:57 BETA: The docker autodiscover is beta
2019-09-27T12:00:23.918Z INFO cfgfile/reload.go:171 Config reloader started
2019-09-27T12:00:23.919Z INFO cfgfile/reload.go:226 Loading of config files completed.
2019-09-27T12:00:23.938Z INFO [autodiscover] autodiscover/autodiscover.go:105 Starting autodiscover manager
2019-09-27T12:00:23.941Z INFO log/input.go:148 Configured paths: [/var/lib/docker/containers/c3bfe253216cdda7bd2fb3712be1b3cd2baac4918fc41d5c0b71f4975fcb0e2d/*-json.log]
2019-09-27T12:00:23.941Z INFO input/input.go:114 Starting input of type: container; ID: 9980403068538753444
2019-09-27T12:00:23.942Z INFO log/harvester.go:253 Harvester started for file: /var/lib/docker/containers/c3bfe253216cdda7bd2fb3712be1b3cd2baac4918fc41d5c0b71f4975fcb0e2d/c3bfe253216cdda7bd2fb3712be1b3cd2baac4918fc41d5c0b71f4975fcb0e2d-json.log
2019-09-27T12:00:23.963Z INFO log/input.go:148 Configured paths: [/var/lib/docker/containers/19b2a9c435da1aacadf891900fd2848d103f89d694324c439ab2f145d4c6cc45/*-json.log]
2019-09-27T12:00:23.963Z INFO log/input.go:148 Configured paths: [/var/lib/docker/containers/19b2a9c435da1aacadf891900fd2848d103f89d694324c439ab2f145d4c6cc45/*-json.log]
2019-09-27T12:00:23.964Z INFO elasticsearch/client.go:170 Elasticsearch url: https://192.168.1.6:9200
2019-09-27T12:00:23.982Z INFO elasticsearch/client.go:743 Attempting to connect to Elasticsearch version 7.3.1
2019-09-27T12:00:24.006Z INFO input/input.go:114 Starting input of type: container; ID: 16974832933917144666
2019-09-27T12:00:24.006Z INFO input/input.go:114 Starting input of type: container; ID: 908475651821469795
2019-09-27T12:00:24.006Z INFO log/input.go:148 Configured paths: [/var/lib/docker/containers/d0c0c3a9b3066eb8393a110ca8019c9b0760eb2e076744662065ba295c4b0913/*-json.log]
2019-09-27T12:00:24.006Z INFO input/input.go:114 Starting input of type: container; ID: 15115359509441093605
2019-09-27T12:00:24.007Z INFO log/harvester.go:253 Harvester started for file: /var/lib/docker/containers/19b2a9c435da1aacadf891900fd2848d103f89d694324c439ab2f145d4c6cc45/19b2a9c435da1aacadf891900fd2848d103f89d694324c439ab2f145d4c6cc45-json.log
2019-09-27T12:00:24.007Z INFO log/harvester.go:253 Harvester started for file: /var/lib/docker/containers/19b2a9c435da1aacadf891900fd2848d103f89d694324c439ab2f145d4c6cc45/19b2a9c435da1aacadf891900fd2848d103f89d694324c439ab2f145d4c6cc45-json.log
2019-09-27T12:00:24.007Z INFO log/harvester.go:253 Harvester started for file: /var/lib/docker/containers/d0c0c3a9b3066eb8393a110ca8019c9b0760eb2e076744662065ba295c4b0913/d0c0c3a9b3066eb8393a110ca8019c9b0760eb2e076744662065ba295c4b0913-json.log
2019-09-27T12:00:24.147Z INFO pipeline/output.go:95 Connecting to backoff(elasticsearch(https://192.168.1.6:9200))
2019-09-27T12:00:24.161Z INFO elasticsearch/client.go:743 Attempting to connect to Elasticsearch version 7.3.1
2019-09-27T12:00:24.206Z INFO [index-management] idxmgmt/std.go:252 Auto ILM enable success.
2019-09-27T12:00:24.206Z INFO [index-management.ilm] ilm/std.go:134 do not generate ilm policy: exists=true, overwrite=false
2019-09-27T12:00:24.206Z INFO [index-management] idxmgmt/std.go:265 ILM policy successfully loaded.
2019-09-27T12:00:24.208Z INFO [index-management] idxmgmt/std.go:300 Write alias successfully generated.
2019-09-27T12:00:24.225Z INFO pipeline/output.go:105 Connection to backoff(elasticsearch(https://192.168.1.6:9200)) established

```

Anyone can help me? I don't know what I am missing.

...

---

<div class="post-metadata">

### Author: ![glauber.ferreira](https://avatars.discourse-cdn.com/v4/letter/g/b2d939/32.png) [@glauber.ferreira](https://discuss.elastic.co/u/glauber.ferreira)
#### Post date: [September 27, 2019, 12:59pm UTC](https://discuss.elastic.co/t/syslog-harvester-does-not-start-running-filebeat-7-3-1-on-docker/201398/2 "2019-09-27T12:59:21Z")

</div>

Last line of log...

`2019-09-27T12:00:53.911Z INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":950,"time":{"ms":958}},"total":{"ticks":6360,"time":{"ms":6375},"value":6360},"user":{"ticks":5410,"time":{"ms":5417}}},"handles":{"limit":{"hard":1048576,"soft":1048576},"open":13},"info":{"ephemeral_id":"c05f9d31-a89b-4f0c-9f80-72dd28101ae6","uptime":{"ms":30137}},"memstats":{"gc_next":58904112,"memory_alloc":39445224,"memory_total":826252240,"rss":104927232},"runtime":{"goroutines":74}},"filebeat":{"events":{"active":4119,"added":46483,"done":42364},"harvester":{"open_files":4,"running":4,"started":4}},"libbeat":{"config":{"module":{"running":0},"reloads":1},"output":{"events":{"acked":42360,"active":50,"batches":849,"total":42410},"read":{"bytes":9735265},"type":"elasticsearch","write":{"bytes":80293951}},"pipeline":{"clients":5,"events":{"active":4119,"filtered":4,"published":46476,"retry":50,"total":46483},"queue":{"acked":42360}}},"registrar":{"states":{"current":3,"update":42364},"writes":{"success":852,"total":852}},"system":{"cpu":{"cores":2},"load":{"1":0.35,"15":0.18,"5":0.17,"norm":{"1":0.175,"15":0.09,"5":0.085}}}}}}`

The configured paths for Syslog module are correct. The files exist.

```
ls -l /var/log/syslog*
-rw-r----- 1 syslog adm 2870704 set 27 09:34 /var/log/syslog
-rw-r----- 1 syslog adm 21769097 set 27 06:25 /var/log/syslog.1
-rw-r----- 1 syslog adm 977882 set 26 06:25 /var/log/syslog.2.gz
-rw-r----- 1 syslog adm 977450 set 25 06:25 /var/log/syslog.3.gz

```

This is my filebeat configuration file:

```
filebeat.modules:
- module: system
# Syslog
  syslog:
    enabled: true
# Authorization logs
  auth:
    enabled: false
filebeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true
filebeat.config:
  modules:
    path: ${path.config}/modules.d/*.yml
    reload.enabled: false
output.elasticsearch:
  hosts: '${ELASTICSEARCH_HOSTS:}'
  username: '${ELASTICSEARCH_USERNAME:}'
  password: '${ELASTICSEARCH_PASSWORD:}'
  ssl.certificate_authorities: '${ELASTICSEARCH_SSL_CERTIFICATE_AUTHORITIES:}'
setup.template.enabled: false
setup.ilm.check_exists: false
setup.ilm.overwrite: false
```

---

<div class="post-metadata">

### Author: ![glauber.ferreira](https://avatars.discourse-cdn.com/v4/letter/g/b2d939/32.png) [@glauber.ferreira](https://discuss.elastic.co/u/glauber.ferreira)
#### Post date: [September 30, 2019, 7:58pm UTC](https://discuss.elastic.co/t/syslog-harvester-does-not-start-running-filebeat-7-3-1-on-docker/201398/3 "2019-09-30T19:58:38Z")

</div>

Solved. I need to mount syslog host files in order to docker container access them. See [https://discuss.elastic.co/t/file-beats-in-docker-how-to-access-host-logs/127313/2](https://discuss.elastic.co/t/file-beats-in-docker-how-to-access-host-logs/127313/2)

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 28, 2019, 7:58pm UTC](https://discuss.elastic.co/t/syslog-harvester-does-not-start-running-filebeat-7-3-1-on-docker/201398/4 "2019-10-28T19:58:40Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
