The symptom is that I can't see the real time log in kibana.
I changed filebeat to debug mode, and I found out that the filebeat publishes old log event.
If you see my log below, filebeat publishes the log of April 29 07:38 at April 29 14:29.
It is about 7:00 hour delay in publishing the event in the filebeat side.
The log file size is about 1.6G.
Here is the filebeat conf.
-
type: log
enabled: true
paths:- /**/_coresystem.log
- /**/_technical.log
multiline.pattern: '^####'
multiline.negate: true
multiline.match: after
tail_files: true
close_rename: true
close_timeout: 24h
exclude_lines: ['<INFO>']
log of filebeat:
2019-04-29T16:29:58.818-0400 DEBUG [publish] pipeline/processor.go:275 Publish event: {
"@timestamp": "2019-04-29T20:29:58.818Z",
"@metadata": {
"beat": "filebeat",
"type": "doc",
"version": "6.2.2"
},
"prospector": {
"type": "log"
},
"beat": {
"name": "xxxxx",
"hostname": "xxxxx",
"version": "6.2.2"
},
"source": "xxxxx",
"offset": 619412645,
"message": "####\u003c2019/04/29 07:37:34.143 -0400\u003e xxxxx: value\u003e$"
}