Trying to build grok pattern for impossible logfile fetched with filebeat sent to logstash

I've gotten this logfile from UNICA by IBM. It's impossible :slight_smile:

A few lines of example.

18 feb 2016 10:37:46,292 - ERROR - An error occured while scanning for the next trigger to fire.
org.quartz.JobPersistenceException: Couldn't acquire next trigger: DSRA9110E: Connection is closed. [See nested exception: com.ibm.websphere.ce.cm.ObjectClosedException: DSRA9110E: Connection is closed.]
	at org.quartz.impl.jdbcjobstore.JobStoreSupport.acquireNextTrigger(JobStoreSupport.java:2814)
	at org.quartz.impl.jdbcjobstore.JobStoreSupport$36.execute(JobStoreSupport.java:2757)
	at org.quartz.impl.jdbcjobstore.JobStoreSupport.executeInNonManagedTXLock(JobStoreSupport.java:3788)
	at org.quartz.impl.jdbcjobstore.JobStoreSupport.acquireNextTrigger(JobStoreSupport.java:2753)
	at org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:263)
Caused by: com.ibm.websphere.ce.cm.ObjectClosedException: DSRA9110E: Connection is closed.
	at com.ibm.ws.rsadapter.jdbc.WSJdbcWrapper.createClosedException(WSJdbcWrapper.java:122)
	at com.ibm.ws.rsadapter.jdbc.WSJdbcConnection.activate(WSJdbcConnection.java:2884)
	at com.ibm.ws.rsadapter.jdbc.WSJdbcConnection.prepareStatement(WSJdbcConnection.java:2777)
	at com.ibm.ws.rsadapter.jdbc.WSJdbcConnection.prepareStatement(WSJdbcConnection.java:2745)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
	at java.lang.reflect.Method.invoke(Method.java:611)
	at org.quartz.impl.jdbcjobstore.AttributeRestoringConnectionInvocationHandler.invoke(AttributeRestoringConnectionInvocationHandler.java:73)
	at com.sun.proxy.$Proxy49.prepareStatement(Unknown Source)
	at org.quartz.impl.jdbcjobstore.StdJDBCDelegate.selectTriggerToAcquire(StdJDBCDelegate.java:2911)
	at org.quartz.impl.jdbcjobstore.JobStoreSupport.acquireNextTrigger(JobStoreSupport.java:2771)
	... 4 more
18 feb 2016 11:47:20,401 - ERROR - Multicast receiver thread caught throwable. Cause was Java heap space. Continuing...
18 feb 2016 14:52:43,213 - WARN  - Authentication event AuthenticationSuccessEvent: asm_admin; details: org.springframework.security.ui.WebAuthenticationDetails@fffe3f86: RemoteIpAddress: fe80:0:0:0:f84f:59cd:f8fe:9355; SessionId: uu_qBghhgKtd0Povxp4Yghg

So as I can see it, the only permanent delimiter is the date pattern.

If it throws a JAVA error it put's the dump in the log, the JAVA dump can both be with a WARN and with an ERROR.
If it's a one-line entry, it could have a ; as a delimiter, which, if it has that, has extra information that would be indexable, but if there is not a ; then it's just a message.

So as I see it, there are three distinct patterns.

I was thinking that I could use this grok and see if there is a match:
\A%{MONTHDAY} %{MONTH} %{YEAR} %{TIME} - %{WORD} - %{GREEDYDATA}
This will match the two 'one line' logentries, but fail on the java dump.

First problem: It would be great to distinguish between the two types of 'one line' logentries but I don't know how to make a pattern that puts everything on the rest of the line until a ';' is found into a field?

Second problem: would it be possible to include the java dump into the log entry where it belongs instead of just discarding it?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.