# FileBeat not harvesting logs **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [December 1, 2017, 12:11am UTC](https://discuss.elastic.co/t/filebeat-not-harvesting-logs/109854 "2017-12-01T00:11:38Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![david2](https://avatars.discourse-cdn.com/v4/letter/d/a87d85/32.png) [@david2](https://discuss.elastic.co/u/david2)\ **Post date:** [December 1, 2017, 12:11am UTC](https://discuss.elastic.co/t/filebeat-not-harvesting-logs/109854/1 "2017-12-01T00:11:38Z")
Hi, I am using filebeat and in particular the apache module to send the logs to ElasticSearch and it doesn't appear that the logs are being sent to ElasticSearch and are not appearing in Kibana. I have followed this page : [https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-apache2.html](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-apache2.html) And run the following command : ./filebeat setup -e and the apache templates appear in kibana. 2017-11-29T10:45:54+10:00 DBG Disable stderr logging 2017-11-29T10:45:54+10:00 INFO Home path: [/u01/filebeat/current] Config path: [/u01/filebeat/current] Data path: [/u01/filebeat/current/data] Logs path: [/u01/filebeat/current/logs] 2017-11-29T10:45:54+10:00 DBG Beat metadata path: /u01/filebeat/current/data/meta.json 2017-11-29T10:45:54+10:00 INFO Beat UUID: 13f79a8f-e275-4fab-ba86-a5045c24363f 2017-11-29T10:45:54+10:00 INFO Setup Beat: filebeat; Version: 6.0.0 2017-11-29T10:45:54+10:00 DBG Initializing output plugins 2017-11-29T10:45:54+10:00 DBG Processors: 2017-11-29T10:45:54+10:00 INFO Elasticsearch url: [http://cidclx062.qdot.qld.gov.au:9200](http://cidclx062.qdot.qld.gov.au:9200) 2017-11-29T10:45:54+10:00 INFO Metrics logging every 30s 2017-11-29T10:45:54+10:00 DBG start pipeline event consumer 2017-11-29T10:45:54+10:00 INFO Beat name: cidclx047 2017-11-29T10:45:54+10:00 INFO filebeat start running. 2017-11-29T10:45:54+10:00 INFO Registry file set to: /u01/filebeat/current/data/registry 2017-11-29T10:45:54+10:00 INFO Loading registrar data from /u01/filebeat/current/data/registry 2017-11-29T10:45:54+10:00 INFO States Loaded from registrar: 0 2017-11-29T10:45:54+10:00 INFO Loading Prospectors: 1 2017-11-29T10:45:54+10:00 DBG Checking module configs from: /u01/filebeat/current/modules.d/_.yml 2017-11-29T10:45:54+10:00 DBG Load config from file: /u01/filebeat/current/modules.d/apache2.yml 2017-11-29T10:45:54+10:00 DBG Number of module configs found: 1 2017-11-29T10:45:54+10:00 DBG Processors: 2017-11-29T10:45:54+10:00 DBG recursive glob disabled 2017-11-29T10:45:54+10:00 DBG exclude\_files: [(?-s:.)gz(?-m:$)] 2017-11-29T10:45:54+10:00 DBG Prospector with previous states loaded: 0 2017-11-29T10:45:54+10:00 DBG File Configs: [/u01/apache/logs/_ /u01/apache/logs/_] 2017-11-29T10:45:54+10:00 DBG Processors: 2017-11-29T10:45:54+10:00 DBG recursive glob disabled 2017-11-29T10:45:54+10:00 DBG exclude\_files: [(?-s:.)gz(?-m:$)] 2017-11-29T10:45:54+10:00 DBG Prospector with previous states loaded: 0 2017-11-29T10:45:54+10:00 DBG File Configs: [/var/log/apache2/error.log_ /var/log/apache2/error.log\*] 2017-11-29T10:45:54+10:00 INFO Loading and starting Prospectors completed. Enabled prospectors: 0 2017-11-29T10:45:54+10:00 INFO Starting Registrar 2017-11-29T10:45:54+10:00 INFO Config reloader started 2017-11-29T10:45:54+10:00 DBG Scan for new config files 2017-11-29T10:45:54+10:00 DBG Load config from file: /u01/filebeat/current/modules.d/apache2.yml 2017-11-29T10:45:54+10:00 DBG Number of module configs found: 1 2017-11-29T10:45:54+10:00 DBG Remove module from stoplist: 20118039972280951 2017-11-29T10:45:54+10:00 DBG Add module to startlist: 20118039972280951 2017-11-29T10:45:54+10:00 DBG Processors: 2017-11-29T10:45:54+10:00 DBG recursive glob disabled 2017-11-29T10:45:54+10:00 DBG exclude\_files: [(?-s:.)gz(?-m:$)] 2017-11-29T10:45:54+10:00 DBG Prospector with previous states loaded: 0 2017-11-29T10:45:54+10:00 DBG File Configs: [/u01/apache/logs/\* /u01/apache/logs/_] 2017-11-29T10:45:54+10:00 DBG Processors: 2017-11-29T10:45:54+10:00 DBG recursive glob disabled 2017-11-29T10:45:54+10:00 DBG exclude\_files: [(?-s:.)gz(?-m:$)] 2017-11-29T10:45:54+10:00 DBG Prospector with previous states loaded: 0 2017-11-29T10:45:54+10:00 DBG File Configs: [/var/log/apache2/error.log_ /var/log/apache2/error.log\*] 2017-11-29T10:45:54+10:00 INFO Starting 1 runners ... 2017-11-29T10:45:54+10:00 INFO Elasticsearch url: [http://cidclx062.qdot.qld.gov.au:9200](http://cidclx062.qdot.qld.gov.au:9200) 2017-11-29T10:45:54+10:00 DBG ES Ping(url=http://cidclx062.qdot.qld.gov.au:9200) 2017-11-29T10:45:54+10:00 DBG Ping status code: 200 2017-11-29T10:45:54+10:00 INFO Connected to Elasticsearch version 5.6.4 2017-11-29T10:45:54+10:00 DBG Required processors: [{user\_agent ingest-user-agent} {geoip ingest-geoip}] 2017-11-29T10:45:54+10:00 DBG GET [http://cidclx062.qdot.qld.gov.au:9200/\_nodes/ingest](http://cidclx062.qdot.qld.gov.au:9200/_nodes/ingest) 2017-11-29T10:45:54+10:00 DBG GET [http://cidclx062.qdot.qld.gov.au:9200/\_ingest/pipeline/filebeat-6.0.0-apache2-access-default](http://cidclx062.qdot.qld.gov.au:9200/_ingest/pipeline/filebeat-6.0.0-apache2-access-default) 2017-11-29T10:45:54+10:00 DBG Pipeline filebeat-6.0.0-apache2-access-default already loaded 2017-11-29T10:45:54+10:00 DBG Required processors: [] 2017-11-29T10:45:54+10:00 DBG GET [http://cidclx062.qdot.qld.gov.au:9200/\_ingest/pipeline/filebeat-6.0.0-apache2-error-pipeline](http://cidclx062.qdot.qld.gov.au:9200/_ingest/pipeline/filebeat-6.0.0-apache2-error-pipeline) 2017-11-29T10:45:54+10:00 DBG Pipeline filebeat-6.0.0-apache2-error-pipeline already loaded 2017-11-29T10:45:54+10:00 INFO Starting prospector of type: log; id: 241050076287732636 2017-11-29T10:45:54+10:00 INFO Starting prospector of t 2017-12-01T10:04:24+10:00 INFO Non-zero metrics in the last 30s: beat.memstats.gc\_next=4194304 beat.memstats.memory\_alloc=1511672 beat.memstats.memory\_total=141713328 filebeat.harvester.open\_files=0 filebeat.harvester.running=0 libbeat.config.module.running=1 libbeat.pipeline.clients=4 libbeat.pipeline.events.active=0 registrar.states.current=0 I can't work out what the problem is. Any help is appreciated. Thanks ype: log; id: 12452675827382683975 ---
**Author:** ![david2](https://avatars.discourse-cdn.com/v4/letter/d/a87d85/32.png) [@david2](https://discuss.elastic.co/u/david2)\ **Post date:** [December 1, 2017, 1:01am UTC](https://discuss.elastic.co/t/filebeat-not-harvesting-logs/109854/2 "2017-12-01T01:01:28Z")
Hi, I discovered this problem but now have a new problem, It was the directory was configured wrong. Now I am getting this error. 2017-12-01T10:59:52+10:00 DBG Skipping directory: /u01/apache2/logs/dev.corp.qdot.qld.gov.au-http 2017-12-01T10:59:52+10:00 DBG Skipping directory: /u01/apache2/logs/dev.corp.qdot.qld.gov.au-https 2017-12-01T10:59:52+10:00 DBG Skipping directory: /u01/apache2/logs/dev.cssr.qdot.qld.gov.au-http 2017-12-01T10:59:52+10:00 DBG Skipping directory: /u01/apache2/logs/dev.cssr.qdot.qld.gov.au-https 2017-12-01T10:59:52+10:00 DBG Skipping directory: /u01/apache2/logs/dev.imd.qdot.qld.gov.au-http Any ideas ? ---
**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\ **Post date:** [December 3, 2017, 10:14am UTC](https://discuss.elastic.co/t/filebeat-not-harvesting-logs/109854/3 "2017-12-03T10:14:34Z")
Could you share your config and version that you are using? The above means these files are skipped, as they are directories and not files, so can't be tailed. ---
**Author:** ![david2](https://avatars.discourse-cdn.com/v4/letter/d/a87d85/32.png) [@david2](https://discuss.elastic.co/u/david2)\ **Post date:** [December 4, 2017, 1:24am UTC](https://discuss.elastic.co/t/filebeat-not-harvesting-logs/109854/4 "2017-12-04T01:24:22Z")
Hi, We are running version 6.0 of filebeat. _filebeat.yml_ filebeat.prospectors: # Each - is a prospector. Most options can be set at the prospector level, so # you can use different prospectors for various configurations. # Below are the prospector specific configurations. - type: log _apache2.yml_ - module: apache2 ---
**Author:** ![david2](https://avatars.discourse-cdn.com/v4/letter/d/a87d85/32.png) [@david2](https://discuss.elastic.co/u/david2)\ **Post date:** [December 4, 2017, 2:15am UTC](https://discuss.elastic.co/t/filebeat-not-harvesting-logs/109854/5 "2017-12-04T02:15:39Z")
Hi, I have resolved the issue, I needed to change /u01/apache2/logs/\* to /u01/apache2/logs/\*\*/\*. Thanks. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 1, 2018, 2:16am UTC](https://discuss.elastic.co/t/filebeat-not-harvesting-logs/109854/6 "2018-01-01T02:16:00Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.