Filebeat has problem to record new lines separated

Elastic Stack Beats
1

Hi

I have following lines, and I am using filebeat to publish this data to logstash and then trying to extract values using grok in Logstash filter, untell now every thing is good (new pattern, multiline config , etc...) the problem is the filebeat record my 2 lines bouth in one single line. i tryed to elemenate /n or /t but i could't get results. Look on my config to make a pictue about the problem .
Could you please advise?

my Log-lines:

[10/8/18 13:14:46:537 CEST] 0000007a SystemOut     O CIWEB Warn : [ls89ta(unknown) @ 10.16.60.8] de.abc.response.ImportContactsChoiceList.executeFilter() User with bensl=QJREQ7K not found.
[10/29/18 20:31:54:185 CET] 00000000 ResourceMgrIm E   WSVR0017E: Error encountered binding the J2EE resource, CNMailSession, as mail/CNMailSession from /opt/websphere-80/profiles/n1filenet80a/config/cells/filenet80a/resources.xml
com.ibm.ws.runtime.component.binder.ResourceBindingException: invalid configuration passed to resource binding logic. REASON: No transport or store protocol defined for mail session CNMailSession
	at com.ibm.ws.mail.resource.server.SessionBinder.getBindingObject(SessionBinder.java:223)

my filebeat:

filebeat.inputs:


- type: log
  paths:
    - /home/badr/Dev/oktobertest/giam.log


  multiline.pattern: '\[[0-9]{1}/[0-9]{2}/[0-9]{2}'
  multiline.negate: true
  multiline.match: after
  fields:
   log_type: SystemOut-JVM
   document_type: SystemOut-JVM
   environment: prod


output.logstash:
    hosts: ["localhost:5044"]

filebeat output:

"host": {
    "name": "badr-VirtualBox"
  },
  "source": "/home/badr/Dev/oktobertest/giam.log",
  "offset": 714,
  "message": "[10/8/18 13:14:46:537 CEST] 0000007a SystemOut     O CIWEB Warn : [ls22ta(unknown) @ 10.16.60.8] de.abc.response.ImportContactsChoiceList.executeFilter() User with bensl=QJREQ7K not found.\n[10/29/18 20:31:54:185 CET] 00000000 ResourceMgrIm E   WSVR0017E: Error encountered binding the J2EE resource, CNMailSession, as mail/CNMailSession from /opt/websphere-80/profiles/n1filenet80a/config/cells/filenet80a/resources.xml\ncom.ibm.ws.runtime.component.binder.ResourceBindingException: invalid configuration passed to resource binding logic. REASON: No transport or store protocol defined for mail session CNMailSession\n\tat com.ibm.ws.mail.resource.server.SessionBinder.getBindingObject(SessionBinder.java:223)\n\n\n",
  "prospector": {
    "type": "log"
  },
  "input": {
    "type": "log"
  },
  "fields": {
    "log_type": "SystemOut-JVM",
    "document_type": "SystemOut-JVM",
    "environment": "prod"
  }

if you chek the output you can see that filed message includes all of the 2 lines and not separately as it should be done.

2

The issue here appears to be a bad pattern. The correct pattern for this use case is multiline.pattern: '^\[[0-9]{1,2}/[0-9]{2}/[0-9]{2}'.

Notice how the month is either 1 or 2 digits (not exactly one as in the given example) and that the pattern is anchored to the start of the line with ^.

3

@Andrew_Cholakian1 yes you are right. Thank you, it work now.
That was pattern error.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.