Hi all,
I have a filebeat configured to consume log files using filestream
and send them to kibana like this:
filebeat.inputs:
- type: filestream
id: my-stream
enabled: true
paths:
- C:\my-app\logs\*.jsonl
parsers:
- ndjson:
keys_under_root: true
add_error_key: true
The source log files are written to every 10 minutes by my-app
application (imagine periodic job).
When filebeat service is started all log lines currently in the files are successfuly sent to kibana.
However after about 7-8 mins I can see in filebeat log (filebeat-20230426.ndjson) the folloving:
//formatted log line
{
"log.level": "info",
"@timestamp": "2023-04-26T11:05:14.058+0200",
"log.logger": "input.filestream",
"log.origin": {
"file.name": "filestream/input.go",
"file.line": 321
},
"message": "Reader was closed. Closing.",
"service.name": "filebeat",
"id": "my-stream",
"source_file": "filestream::my-stream::native::25296896-229669-3598722556",
"path": "C:\\my-app\\logs\\log-struct-20230426.jsonl",
"state-id": "native::25296896-229669-3598722556",
"ecs.version": "1.6.0"
}
and since that point no further log lines are sent to kibana (I waited for hours and could see new log lines in the source log file).
If I restart the filebeat service then all new log lines are sent to kibana, but again after 7-8 minutes the Reader was closed. Closing.
message shows up in filebeat log and no further log lines are processed. What am I missing?
This is a fresh filebeat installation, not a migration from previous log
input. However if I do the same with (now deprecated) log
input:
filebeat.inputs:
- type: log
enabled: true
paths:
- C:\my-app\logs\*.jsonl
json.keys_under_root: true
json.add_error_key: true
all the log lines (including new ones coming in every 10 mins) are floating to kibana without issues.
useful info:
source log file is written by my-app using serilog
Filebeat version: 8.7.0
OS: Windows Server 2016 Datacenter (issue is reproducible on all 3 servers runnig my-app)