Hello,
I would like to send application logs from filebeat directly to elasticsearch index.
I'm trying send multiline message but in index I see it as single line.
EmbargoServiceLogger Error: 92 : 3/12/2019 1:46:29 PM##4784##8100##Creation of listner #7 failed. Exception:System.IO.FileNotFoundException: Error reading the Test\To\AllianceManual\Screening\SRC directory.
at System.IO.FileSystemWatcher.StartRaisingEvents()
at EmbargoSvc.AMLEmbargoService.StartFileImport()
my filebeat.yml multiline configuration is following:
multiline.pattern: '^[[:space:]]'
multiline.negate: false
multiline.match: after
Could somebody help me with this?
Thx
shaunak
(Shaunak Kashyap)
March 18, 2019, 9:19pm
2
Hi @Matus_Gajdos ,
Welcome!
I tried your settings and sample message with my Filebeat and I'm seeing a multiline (\n
-delimited) message in Kibana Discover:
What are you seeing in your Kibana Discover? Alternatively, you could just post the complete Elasticsearch document for that log entry over here as well.
Hello @shaunak
here is what I see in kibana
Could you send me your filebeat configuration?
Are you sending data directly to elasticsearch or through logstash?
Thank you.
shaunak
(Shaunak Kashyap)
March 20, 2019, 11:54am
4
So, first, I tested with the console
output, just for debugging purposes. For that, my filebeat.yml
looked like this:
filebeat.inputs:
- type: stdin
enabled: true
multiline.pattern: '^[[:space:]]'
multiline.negate: false
multiline.match: after
output.console:
enabled: true
Using that configuration, when I supplied your sample input on STDIN, I got the following output on STDOUT (in the console):
{"@timestamp":"2019-03-20T11:51:03.693Z","@metadata":{"beat":"filebeat","type":"_doc","version":"8.0.0"},"message":"EmbargoServiceLogger Error: 92 : 3/12/2019 1:46:29 PM##4784##8100##Creation of listner #7 failed. Exception:System.IO.FileNotFoundException: Error reading the Test\\To\\AllianceManual\\Screening\\SRC directory.\n at System.IO.FileSystemWatcher.StartRaisingEvents()\n at EmbargoSvc.AMLEmbargoService.StartFileImport()","input":{"type":"stdin"},"ecs":{"version":"1.0.0"},"host":{"name":"Shaunaks-MBP-2"},"agent":{"version":"8.0.0","type":"filebeat","ephemeral_id":"1bc476cb-af43-440b-8626-61184b65792b","hostname":"Shaunaks-MBP-2","id":"51a8bb52-16b0-4129-a33b-9fba504fe6e8"},"log":{"flags":["multiline"],"offset":0,"file":{"path":""}}}
Note the \n
characters in the message
field.
Next, I changed the output to elasticsearch
and provided the same input via STDIN. In that case I got what I posted earlier in the Kibana screenshot.
Are you sending via Logstash? Could you try the console
output first like I did, just for debugging purposes?
system
(system)
Closed
April 17, 2019, 11:54am
5
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.