Filebeat multiline parsing problem

Elastic Stack Beats
1

Hi community,

I started to use filebeat to get my tomcats logs into ELK. It works fine for some logs but when I am trying to use multiline pattern, I got troubles.

I followed the documentation to get all lines of one event

Here is the log :

05-Mar-2018 10:57:59.285 INFO [ajp-nio-9080-exec-9] org.apache.catalina.realm.JNDIRealm.authenticate Exception performing authentication. Retrying...
 javax.naming.CommunicationException [Root exception is java.net.SocketException: Connection reset]; remaining name 'dc=xxxx,dc=yyyy,dc=zzzz'
        at com.sun.jndi.ldap.LdapClient.authenticate(LdapClient.java:163)
        at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2788)
        at com.sun.jndi.ldap.LdapCtx.ensureOpen(LdapCtx.java:2696)
        at com.sun.jndi.ldap.LdapCtx.ensureOpen(LdapCtx.java:2670)
        at com.sun.jndi.ldap.LdapCtx.doSearch(LdapCtx.java:1941)
        at com.sun.jndi.ldap.LdapCtx.searchAux(LdapCtx.java:1844)
        at com.sun.jndi.ldap.LdapCtx.c_search(LdapCtx.java:1769)
        at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_search(ComponentDirContext.java:392)
        at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:358)
        at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.search(PartialCompositeDirContext.java:341)
        at javax.naming.directory.InitialDirContext.search(InitialDirContext.java:267)
        at org.apache.catalina.realm.JNDIRealm.getUserBySearch(JNDIRealm.java:1681)
        at org.apache.catalina.realm.JNDIRealm.getUser(JNDIRealm.java:1517)
        at org.apache.catalina.realm.JNDIRealm.getUser(JNDIRealm.java:1465)
        at org.apache.catalina.realm.JNDIRealm.authenticate(JNDIRealm.java:1406)
        at org.apache.catalina.realm.JNDIRealm.authenticate(JNDIRealm.java:1264)
        at org.apache.catalina.realm.CombinedRealm.authenticate(CombinedRealm.java:192)
        at org.apache.catalina.authenticator.FormAuthenticator.authenticate(FormAuthenticator.java:245)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:453)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:617)
        at org.apache.catalina.authenticator.SingleSignOn.invoke(SingleSignOn.java:240)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:518)
        at org.apache.coyote.ajp.AbstractAjpProcessor.process(AbstractAjpProcessor.java:844)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:668)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1521)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1478)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.SocketException: Connection reset
        at java.net.SocketInputStream.read(SocketInputStream.java:189)
        at java.net.SocketInputStream.read(SocketInputStream.java:121)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
        at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
        at com.sun.jndi.ldap.Connection.run(Connection.java:848)
        ... 1 more

Here is the filebeat.yml :

#================================ variable ===================================
path.home: /usr/share/filebeat
path.config: /etc/filebeat
#================================ Logging =====================================
logging.level: debug
logging.to_files: true


#================================ Config  =====================================
#filebeat.config.modules:
#  path: ${path.config}/modules.d/*.yml
#  reload.enabled: false

filebeat.prospectors:
- input_type: log
  document_type: tomcat_log
  paths:
    - /opt/test.out
  exclude_files: [".gz$"]

multiline:
  pattern: '^[[:space:]]+(at|\.{3})\b|^Caused by:'
  negate: false
  match: after


output.elasticsearch:
  hosts: ["DNS-ELK:9200"]
  index: "tomcat-%{+dd.MM.YYYY}"

#setup.template.settings:
#  index.number_of_shards: 3

setup.template.name: "filebeat"
setup.template.pattern: "filebeat-*"

DId I miss something ?

Regards

2

The indentation of the multiline part looks of. It needs to be on the same level as exclude_files for example as it can be defined for each prospector.

3

Hello,

Yes it works now.

Just another question. If I want to send the multiline events to logstash to add tags (e.g. environment), is it possible ? Should I use the same regex in logstash ?

Thanks

Regards

4

You can send your events to LS to do enrichment. I would still recommend you to do the multiline on the beats side as otherwise it could happen that events do not end up in the right order or different servers.

5

Thx for your reply

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.