Which pattern to choose for Stack trace

Elastic Stack Beats
1

Hi

Can somebody please assist me a bit to know which pattern I should use for the following stack trace:

<event>
  <date>2016/06/30 00:00:02</date><severity>ERROR</severity><reqid></reqid><thread>Thread-429878</thread><user>Superuser/Superuser</user><runid>146366100371
8</runid><subsys>JMS</subsys><msgid>JMS0005</msgid><ex>javax.naming.NameNotFoundException</ex>
  <msg>JMS0005: Failed to start JMS consumer(s) for 'ConsumerInfo[destinationJndiName='ap.aia.cataloguetranslation.v1.q' connectionFactoryName='SONIC_QCF' j
ndiName='SONIC_JNDI' transportName='ap.aia.cataloguetranslation.v1']'
APPL0065: javax.naming.NamingException exception
javax.naming.NameNotFoundException: /ap.aia.cataloguetranslation.v1.q not found in the specified context
        at com.actional.jndi.InitialContextValuesCache$CacheEventsImpl.createCacheItem(InitialContextValuesCache.java:198)
        at com.actional.jndi.InitialContextValuesCache$CacheEventsImpl.createCacheItem(InitialContextValuesCache.java:145)
        at com.actional.util.Cache$CachedItem.init(Cache.java:573)
        at com.actional.util.Cache$CachedItem.getDataWithInit(Cache.java:561)
        at com.actional.util.SemaphoreAwareCache.lookup(SemaphoreAwareCache.java:69)
        at com.actional.jndi.InitialContextValuesCache.lookup(InitialContextValuesCache.java:132)
        at com.actional.jndi.InitialContextCache.lookup(InitialContextCache.java:57)
        at com.actional.jms.AbstractJmsMessageListener.init(AbstractJmsMessageListener.java:218)
        at com.actional.jms.IntermediaryJmsMessageListenerFactory.createListener(IntermediaryJmsMessageListenerFactory.java:139)
        at com.actional.jms.JmsConsumerManager$ConsumerInfo$1.init(JmsConsumerManager.java:470)
        at com.actional.jms.ConnectionFacade.doStartConnection(ConnectionFacade.java:118)
        at com.actional.jms.ConnectionFacade.startConnection(ConnectionFacade.java:98)
        at com.actional.jms.JmsConsumerManager$ConsumerInfo.start(JmsConsumerManager.java:502)
        at com.actional.jms.JmsConsumerManager$ConsumerInfo.start(JmsConsumerManager.java:413)
        at com.actional.jms.JmsConsumerManager$ConsumerInfo.start(JmsConsumerManager.java:391)
        at com.actional.jms.JmsConsumerManager$ConsumerInfo$StartListeningTask$1.run(JmsConsumerManager.java:711)

Nested Exception:
javax.naming.NameNotFoundException: /ap.aia.cataloguetranslation.v1.q not found in the specified context
        at com.sonicsw.jndi.mfcontext.MFContext.lookup(MFContext.java:383)
        at javax.naming.InitialContext.lookup(Unknown Source)
        at com.actional.jndi.InitialContextValuesCache$CacheEventsImpl.lookup(InitialContextValuesCache.java:211)
        at com.actional.jndi.InitialContextValuesCache$CacheEventsImpl.createCacheItem(InitialContextValuesCache.java:185)
        at com.actional.jndi.InitialContextValuesCache$CacheEventsImpl.createCacheItem(InitialContextValuesCache.java:145)
        at com.actional.util.Cache$CachedItem.init(Cache.java:573)
        at com.actional.util.Cache$CachedItem.getDataWithInit(Cache.java:561)
        at com.actional.util.SemaphoreAwareCache.lookup(SemaphoreAwareCache.java:69)
        at com.actional.jndi.InitialContextValuesCache.lookup(InitialContextValuesCache.java:132)
        at com.actional.jndi.InitialContextCache.lookup(InitialContextCache.java:57)
        at com.actional.jms.AbstractJmsMessageListener.init(AbstractJmsMessageListener.java:218)
        at com.actional.jms.IntermediaryJmsMessageListenerFactory.createListener(IntermediaryJmsMessageListenerFactory.java:139)
        at com.actional.jms.JmsConsumerManager$ConsumerInfo$1.init(JmsConsumerManager.java:470)
        at com.actional.jms.ConnectionFacade.doStartConnection(ConnectionFacade.java:118)
        at com.actional.jms.ConnectionFacade.startConnection(ConnectionFacade.java:98)
        at com.actional.jms.JmsConsumerManager$ConsumerInfo.start(JmsConsumerManager.java:502)
        at com.actional.jms.JmsConsumerManager$ConsumerInfo.start(JmsConsumerManager.java:413)
        at com.actional.jms.JmsConsumerManager$ConsumerInfo.start(JmsConsumerManager.java:391)
        at com.actional.jms.JmsConsumerManager$ConsumerInfo$StartListeningTask$1.run(JmsConsumerManager.java:711)
</msg>
</event>

This is the part I have to fill in in the filebeat.yml config file:

Mutiline can be used for log messages spanning multiple lines. This is common

  # for Java Stack Traces or C-Line Continuation
  multiline:

    # The regexp Pattern that has to be matched. The example pattern matches all lines starting with [
    pattern: ^<event>

What should I choose for the pattern as I would like to have the ERROR, >javax.naming.NameNotFoundException and the JMS0005 field but only till the lines start with "at"

Can somebody please help me, I'm still looking into it but as filebeat is quiet new to me...

Kind regards

Johnny

2

Your logic should be "if the line does not begin with a timestap, join with the preceding line". I believe the documentation of Logstash's multiline codec contains an example of what you need, and although the configuration keywords differ between Logstash and Filebeat the functionality is basically the same.

3

please format you logs using 3 backticks ('`'). Seems your log is using xml (this has not been clear from original post). With every log entry start with event your multiline config should be similar to:

multiline:
  pattern: '^<event>'
  negate: true

This will capture the complete event. Multiline is not about parsing + filebeat does not directly support xml. Post-processing + parsing is normally done via logstash using the xml and grok filters in logstash.

The JMS0005 line doesn't even start at beginning of line, but right after tag. One maybe can construct a regex to collect some of these lines.

I can not tell if the lines copy'n paster are correctly formatted or if you have introduced some newlines by accident.

Here is a small script to 'multiline concat' a subset of of lines you're interersted in: https://play.golang.org/p/JWRA5W9ztz

Matching multiline config:

multiline:
  pattern: '^[^ \t<]+'
  match: after

The events being generated using this pattern:

<event>

  <date>2016/06/30 00:00:02</date><severity>ERROR</severity><reqid></reqid><thread>Thread-429878</thread><user>Superuser/Superuser</user><runid>1463661003718</runid><subsys>JMS</subsys><msgid>JMS0005</msgid><ex>javax.naming.NameNotFoundException</ex>

  <msg>JMS0005: Failed to start JMS consumer(s) for 'ConsumerInfo[destinationJndiName='ap.aia.cataloguetranslation.v1.q' connectionFactoryName='SONIC_QCF' jndiName='SONIC_JNDI' transportName='ap.aia.cataloguetranslation.v1']'
APPL0065: javax.naming.NamingException exception
javax.naming.NameNotFoundException: /ap.aia.cataloguetranslation.v1.q not found in the specified context

        at com.actional.jndi.InitialContextValuesCache$CacheEventsImpl.createCacheItem(InitialContextValuesCache.java:198)

	
Nested Exception:
javax.naming.NameNotFoundException: /ap.aia.cataloguetranslation.v1.q not found in the specified context

With you being interested in the event starting with <msg> you can use exclude_lines and include_lines filter to only forward those lines you're interested in. Still in logstash you might want to use grok to get rid of the <msg> element. The filters in filebeat are only used to reduce amount of data being send + reduce parsing overhead in logstash.

Given all the meta-data included in one event, I'd consider capturing the complete xml event and use the xml filter in logstash. Fields like severity or user, subsys and ex would be very great to have in kibana for filtering. Use grok if you want to reduce the msg field. Or rather use grok to capture subject from msg, but keep msg in case some dev is interested in stack-trace.

4

OK, thanks a lot for your explanation but it seems the way I wanted to use it is not really possible and I should use the xml fileter in logstash I guess.

Kind regards

Johnny

5

This topic was automatically closed after 21 days. New replies are no longer allowed.