phillipgibb
(phillipgibb@gmail.com)
January 4, 2017, 3:04pm
1
Hi,
I am using FileBeat 5.1.1
I have a portion of a log file:
2 2014-01-01 00:08:27.517 XML.Handler[3]: Received:
<request version="00">
<date>20140101</date>
<time>000827</time>
</request>
2 2014-01-01 00:08:27.564 ConfirmCustomerRequestHandler[0]: POST /services HTTP/1.0
My multiline config is:
multiline:
pattern: '^[0-9]{1}\s[0-9]{4}-[0-9]{2}-[0-9]{2}\s[0-9]{2}:[0-9]{2}:[0-9]{2}\.[0-9]{3}\s'
match: after
negate: true
timeout: 15s
max_lines: 1000
In Kibana I see the 6 lines in different entries with the same timestamp.
I tested this in the Go Playground and it works:
matches line
false 2 2014-01-01 00:08:27.517 XML.Handler[3]: Received:
true <request version="00">
true <date>20140101</date>
true <time>000827</time>
true </request>
false 2 2014-01-01 00:08:27.564 ConfirmCustomerRequestHandler[0]: POST /services HTTP/1.0
Is there something I am missing? If I include the lines matching the pattern then all the multilines get dropped.
thanks
Phillip
andrewkroh
(Andrew Kroh)
January 4, 2017, 4:05pm
2
Double check the indentation in your config file. I tested with:
filebeat.prospectors:
- paths: [input.txt]
multiline:
pattern: '^[0-9]{1}\s[0-9]{4}-[0-9]{2}-[0-9]{2}\s[0-9]{2}:[0-9]{2}:[0-9]{2}\.[0-9]{3}\s'
match: after
negate: true
timeout: 15s
max_lines: 1000
output.console.pretty: true
logging.level: debug
where input.txt
contained the snippit you gave.
The output was:
{
"@timestamp": "2017-01-04T16:02:47.395Z",
"beat": {
"hostname": "macbook",
"name": "macbook",
},
"input_type": "log",
"message": "2 2014-01-01 00:08:27.517 XML.Handler[3]: Received:\n\u003crequest version=\"00\"\u003e\n \u003cdate\u003e20140101\u003c/date\u003e\n \u003ctime\u003e000827\u003c/time\u003e\n\u003c/request\u003e",
"offset": 136,
"source": "input.txt",
"type": "log"
}
{
"@timestamp": "2017-01-04T16:02:47.395Z",
"beat": {
"hostname": "macbook",
"name": "macbook",
},
"input_type": "log",
"message": "2 2014-01-01 00:08:27.564 ConfirmCustomerRequestHandler[0]: POST /services HTTP/1.0",
"offset": 220,
"source": "input.txt",
"type": "log"
}
phillipgibb
(phillipgibb@gmail.com)
January 5, 2017, 6:45am
3
it was the indentation. as much as I had stared at the indentation I could not convince myself that was the issue,
It really was because of all the comments in the yml; you config, without the comments, made it clear to me.
Thanks
system
(system)
Closed
February 2, 2017, 6:45am
4
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.