Delimiter for FILEBEAT?

Elastic Stack Beats
1

I have been using logstash to ship logs using the following config

  	file {
		path => "/appdir/logs/*\.log\.xml"
		type => "log.xml" 
		delimiter => "</LogRoot>"

I am trying to shift to use Filebeat but i have issues finding a "delimiter" replacement.

Can anyone help me with this?

Thanks

2

in filebeat you can not configure a 'delimiter'. filebeat is splitting up logs by newline.

But you can use filebeat multiline support to match on ^</LogRoot>$.

3

would it work if the log in question is a single line ?

<LogRoot><LogMessage Date="Wed Apr 12 19:36:28 SGT 2017" Count="1" Type="Text" Level="Information"/><LogMessage Date="Wed Apr 12 19:36:36 SGT 2017" Count="2" Type="Text">Session MXPROCESSINGSCRIPT_POOL_A created, PID=104506, NPID=39314, InstallationCode=mxprocessingscript_s01smxu10z1a, nickName=MXPROCESSINGSCRIPT_POOL_A, host=s01smxu10z1a</LogMessage><LogMessage Date="Wed Apr 12 19:37:16 SGT 2017" Count="3" Type="Tree" Level="Information"/><GuiRoot><batch execution="synchronous"><label>BEXT_AUDINSRMSE</label><getJobDetailsResponse><jobID>2239574</jobID><user>BATCHUSER</user><group>MO_EOD_A</group><status>Done</status><date>12/04/17</date><time>19:37:05</time><endTime>19:37:16</endTime><details><element index="0"><label>EXT_AUDINSRMSPM</label></element></details></getJobDetailsResponse></batch></GuiRoot><LogMessage Date="Wed Apr 12 19:37:17 SGT 2017" Count="4" Type="Text" Level="Information"/></LogRoot>

if it doesnt, what would you suggest in this case?

since its a single line, the harvester is open but doesnt send anything to logstash.
In addition, this log file is updated over time : meaning that values are changed within the log file.

4

questions:

  • is the log entry always a single line?
  • do you have multiple entries in a log file?
  • does the line end with a newline character?
5

yes the log entry is always a single line
there is only a single entry in each log file
it does not end with a new line character

thanks ^^

6

filebeat splits lines on newline characters. The splitting pattern is not yet configurable. Feel free to open a feature request on: https://github.com/elastic/beats/issues