If filebeat is deployed with single instance then no issue is observed in multiline handling. If Filebeat is deployed with daemonset then all events does not follow proper multiline pattern thus results in individual events. Is there any bug reported?

Elastic Stack Beats
1

If filebeat is deployed with single instance then no issue is observed in multiline handling. If Filebeat is deployed with daemonset then all events does not follow proper multiline pattern thus results in individual events. Is there any bug reported ?

2

Issues and bug reports are handled through the beats github repo: Issues · elastic/beats · GitHub

Can you provide any details on how filebeat is deployed and configured?

3

Filebeat config looks like this

filebeat-container.yml: |2
filebeat.inputs:

  • type: log
    paths: [/opt/abc/myfile.log]
    multiline.pattern: [1]
    multiline.negate: true
    multiline.match: after
    output.kafka:
    hosts: ${KAFKA_HOST}
    topic: myfilelogs
    compression: snappy
    max_message_bytes: 1000000
    logging.level: debug

Log file -
[00:00:37]
hsdhdjn : xmt/rcv/%loss = 23/23/0%, min/avg/max = 0.249/0.412/2.69
[00:00:44]
240************** : xmt/rcv/%loss = 23/23/0%, min/avg/max = 0.249/0.316/0.390
[00:00:51]
240************** : xmt/rcv/%loss = 24/24/0%, min/avg/max = 0.260/0.3**/0.5**
[00:00:58]
240************** : xmt/rcv/%loss = 23/23/0%, min/avg/max = 0.264/0.322/0.412
[00:01:05]
240************** : xmt/rcv/%loss = 23/23/0%, min/avg/max = 0.242/0.372/0.859

output on kafka when using deployment -

{
"@timestamp": "2022-08-30T06:15:51.300Z",
"@metadata": {
"beat": "filebeat",
"type": "_doc",
"version": "8.2.3"
},
"log": {
"offset": 518535,
"file": {
"path": "/home/obf/abc/myfile.log"
},
"flags": [
"multiline"
]
},
"input": {
"type": "log"
},
"ecs": {
"version": "8.0.0"
},
"host": {
"name": "abc-network-tool-network***-0"
},
"agent": {
"version": "8.2.3",
"ephemeral_id": "",
"id": "2dc778780",
"name": "-0",
"type": "filebeat"
},
"message": "[00:01:05]\n240***** : xmt/rcv/%loss = 23/23/0%, min/avg/max = 0.242/0.372/0.859"
}
output on kafka when using daemonset -

{
"@timestamp": "2022-08-30T06:15:51.300Z",
"@metadata": {
"beat": "filebeat",
"type": "_doc",
"version": "8.2.3"
},
"log": {
"offset": 518535,
"file": {
"path": "/home/obf/abc/myfile.log"
}
},
"input": {
"type": "log"
},
"ecs": {
"version": "8.0.0"
},
"host": {
"name": "abc-network-tool-network***-0"
},
"agent": {
"version": "8.2.3",
"ephemeral_id": "",
"id": "2dc778780",
"name": "-0",
"type": "filebeat"
},
"message": "[00:00:58]"
}

{
"@timestamp": "2022-08-30T06:15:51.300Z",
"@metadata": {
"beat": "filebeat",
"type": "_doc",
"version": "8.2.3"
},
"log": {
"offset": 518536,
"file": {
"path": "/home/obf/abc/myfile.log"
}
},
"input": {
"type": "log"
},
"ecs": {
"version": "8.0.0"
},
"host": {
"name": "abc-network-tool-network***-0"
},
"agent": {
"version": "8.2.3",
"ephemeral_id": "",
"id": "2dc778780",
"name": "-0",
"type": "filebeat"
},
"message": "240************** : xmt/rcv/%loss = 23/23/0%, min/avg/max = 0.264/0.322/0.412"
}

  1. \d{2}:\d{2}:\d{2} ↩︎

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.