I am new to filebeat and have set up the syslog files of our servers to be sent to logstash by filebeat. I am now trying to get apache logs into logstash but it does not seem to be reading the log.
When starting up filebeat this is the debug info:
sudo service filebeat restart
- Restarting Sends log files to Logstash or directly to Elasticsearch. filebeat 2016/01/18 06:45:06.457697 beat.go:97: DBG Initializing output plugins
2016/01/18 06:45:06.457773 geolite.go:24: INFO GeoIP disabled: No paths were set under output.geoip.paths
2016/01/18 06:45:06.520679 outputs.go:111: INFO Activated logstash as output plugin.
2016/01/18 06:45:06.521348 publish.go:198: DBG create output worker: 0x0, 0x0
2016/01/18 06:45:06.523384 publish.go:235: DBG No output is defined to store the topology. The server fields might not be filled.
2016/01/18 06:45:06.523684 publish.go:249: INFO Publisher name: myserver
2016/01/18 06:45:06.524167 async.go:95: DBG create bulk processing worker (interval=1s, bulk size=200)
2016/01/18 06:45:06.524334 beat.go:107: INFO Init Beat: filebeat; Version: 1.0.1
When I look at the filebeat log it looks like this:
2016-01-18T15:45:06+09:00 DBG scan path /var/log/apache/.log
2016-01-18T15:45:06+09:00 DBG No pending prospectors. Finishing setup
2016-01-18T15:45:06+09:00 INFO All prospectors initialised with 1 states to persist
2016-01-18T15:45:06+09:00 INFO Starting Registrar
2016-01-18T15:45:06+09:00 INFO Start sending events to output
2016-01-18T15:45:09+09:00 DBG Flushing spooler because of timemout. Events flushed: 0
2016-01-18T15:45:14+09:00 DBG Flushing spooler because of timemout. Events flushed: 0
2016-01-18T15:45:16+09:00 DBG Start next scan
2016-01-18T15:45:16+09:00 DBG scan path /var/log/apache/.log
2016-01-18T15:45:21+09:00 DBG Flushing spooler because of timemout. Events flushed: 0
The access logs have definitely been written to during this period.
The apache prospector looks like this:
Ignore files which are older then 24 hours
Additional fields which can be freely defined
Any feedback would be appreciated.