Filebeat stops harvesting (error file didn't change) but tail -f is ok!


(Hong Zhou) #1

i'm pretty sure the monitor log file is busy (check by tail -f xxxx.log)

but the filebeat stops harvesting and didn't send any newline log .

i restart the filebeat and still not working .

then start debug mode with ( filebeat -e -d "*") which show file didn't change :sob:

this strange thing happen more than one time and suddenly appear

i google for answer but still confusing

any suggestion will appreciated :relaxed:

2016/09/21 06:35:44.896153 prospector.go:275: DBG Check file for harvesting: /app/bkgadm/trc/21/jsonTXNTRC.lst
2016/09/21 06:35:44.896172 registrar.go:175: DBG Same file as before found. Fetch the state.
2016/09/21 06:35:44.896178 prospector.go:418: DBG Update existing file for harvesting: /app/bkgadm/trc/19/jsonTXNTRC.lst
2016/09/21 06:35:44.896185 prospector.go:465: DBG Not harvesting, file didn't change: /app/bkgadm/trc/19/jsonTXNTRC.lst
2016/09/21 06:35:44.896190 prospector.go:418: DBG Update existing file for harvesting: /app/bkgadm/trc/20/jsonTXNTRC.lst
2016/09/21 06:35:44.896194 prospector.go:465: DBG Not harvesting, file didn't change: /app/bkgadm/trc/20/jsonTXNTRC.lst
2016/09/21 06:35:44.896198 prospector.go:418: DBG Update existing file for harvesting: /app/bkgadm/trc/21/jsonTXNTRC.lst
2016/09/21 06:35:44.896202 prospector.go:465: DBG Not harvesting, file didn't change: /app/bkgadm/trc/21/jsonTXNTRC.lst
2016/09/21 06:35:45.160333 reader.go:138: DBG End of file reached: /app/bkgadm/trc/21/jsonTXNTRC.lst; Backoff now.
2016/09/21 06:35:45.890674 reader.go:138: DBG End of file reached: /var/log/testjson.log; Backoff now.
2016/09/21 06:35:45.892728 reader.go:138: DBG End of file reached: /app/bkgadm/trc/20/jsonTXNTRC.lst; Backoff now.
2016/09/21 06:35:45.892733 reader.go:138: DBG End of file reached: /app/bkgadm/trc/19/jsonTXNTRC.lst; Backoff now.
2016/09/21 06:35:46.160515 reader.go:138: DBG End of file reached: /app/bkgadm/trc/21/jsonTXNTRC.lst; Backoff now.
2016/09/21 06:35:47.160716 reader.go:138: DBG End of file reached: /app/bkgadm/trc/21/jsonTXNTRC.lst; Backoff now.
2016/09/21 06:35:47.389941 spooler.go:97: DBG Flushing spooler because of timeout. Events flushed: 0
2016/09/21 06:35:47.890803 reader.go:138: DBG End of file reached: /var/log/testjson.log; Backoff now.
2016/09/21 06:35:47.892842 reader.go:138: DBG End of file reached: /app/bkgadm/trc/20/jsonTXNTRC.lst; Backoff now.
2016/09/21 06:35:47.892845 reader.go:138: DBG End of file reached: /app/bkgadm/trc/19/jsonTXNTRC.lst; Backoff now.
2016/09/21 06:35:48.160889 reader.go:138: DBG End of file reached: /app/bkgadm/trc/21/jsonTXNTRC.lst; Backoff now.
2016/09/21 06:35:49.161079 reader.go:138: DBG End of file reached: /app/bkgadm/trc/21/jsonTXNTRC.lst; Backoff now.
2016/09/21 06:35:49.896303 prospector.go:185: DBG Start next scan
2016/09/21 06:35:49.896333 prospector.go:261: DBG scan path /var/log/testjson.log
2016/09/21 06:35:49.896348 prospector.go:275: DBG Check file for harvesting: /var/log/testjson.log
2016/09/21 06:35:49.896360 registrar.go:175: DBG Same file as before found. Fetch the state.

uname -ra

Linux app22.xxxxx.local 2.6.32-573.12.1.el6.x86_64 #1 SMP Tue Dec 15 21:19:08 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
[root@app22 filebeat]#

filebeat version 1.3.0 (amd64)

[root@app22 filebeat]# cat filebeat.yml | sed '/#/d' | grep -v "^$"
filebeat:
prospectors:
-
paths:
- /var/log/testjson.log
- /app/bkgadm/trc/*/jsonTXNTRC.lst
encoding: gbk
input_type: log
scan_frequency: 5s
registry_file: /var/lib/filebeat/registry
output:
logstash:
hosts: ["172.16.12.63:5044"]
shipper:
logging:
files:
[root@app22 filebeat]#


(system) #2

This topic was automatically closed after 21 days. New replies are no longer allowed.