Filebeat publishes old event of log so I can't see real time log in kibana

The symptom is that I can't see the real time log in kibana.
I changed filebeat to debug mode, and I found out that the filebeat publishes old log event.
If you see my log below, filebeat publishes the log of April 29 07:38 at April 29 14:29.
It is about 7:00 hour delay in publishing the event in the filebeat side.

The log file size is about 1.6G.

Here is the filebeat conf.

  • type: log
    enabled: true

    • /**/_coresystem.log
    • /**/_technical.log

    multiline.pattern: '^####'
    multiline.negate: true
    multiline.match: after
    tail_files: true
    close_rename: true
    close_timeout: 24h
    exclude_lines: ['<INFO>']

log of filebeat:

2019-04-29T16:29:58.818-0400 DEBUG [publish] pipeline/processor.go:275 Publish event: {
"@timestamp": "2019-04-29T20:29:58.818Z",
"@metadata": {
"beat": "filebeat",
"type": "doc",
"version": "6.2.2"
"prospector": {
"type": "log"
"beat": {
"name": "xxxxx",
"hostname": "xxxxx",
"version": "6.2.2"
"source": "xxxxx",
"offset": 619412645,
"message": "####\u003c2019/04/29 07:37:34.143 -0400\u003e xxxxx: value\u003e$"

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.