Parsing stacktraces in log files

Elastic Stack Logstash
1

Hello, I'm new to the Elastic Stack and I'm trying to parse the following log-
06:40:13,339 ERROR [org.hornetq.core.client] (Thread-29 (HornetQ-server-HornetQServerImpl::serverUUID=3a64b874-09b1-11e7-af28-998b43be6a2d-774875813)) HQ214016: Failed to create netty connection: java.net.SocketTimeoutException: connect timed out
at java.net.TwoStacksPlainSocketImpl.socketConnect(Native Method) [rt.jar:1.8.0_102]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) [rt.jar:1.8.0_102]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) [rt.jar:1.8.0_102]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) [rt.jar:1.8.0_102]
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:172) [rt.jar:1.8.0_102]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) [rt.jar:1.8.0_102]
at java.net.Socket.connect(Socket.java:589) [rt.jar:1.8.0_102]
at org.jboss.netty.channel.socket.oio.OioClientSocketPipelineSink.connect(OioClientSocketPipelineSink.java:109) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
at org.jboss.netty.channel.socket.oio.OioClientSocketPipelineSink.eventSunk(OioClientSocketPipelineSink.java:69) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
at org.jboss.netty.channel.Channels.connect(Channels.java:634) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
at org.jboss.netty.channel.AbstractChannel.connect(AbstractChannel.java:207) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:229) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
at org.jboss.netty.bootstrap.ClientBootstrap.connect(ClientBootstrap.java:182) [netty-3.6.10.Final-redhat-1.jar:3.6.10.Final-redhat-1]
at org.hornetq.core.remoting.impl.netty.NettyConnector.createConnection(NettyConnector.java:653) [hornetq-core-client-2.3.25.Final-redhat-1.jar:2.3.25.Final-redhat-1]
at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnection(ClientSessionFactoryImpl.java:1257) [hornetq-core-client-2.3.25.Final-redhat-1.jar:2.3.25.Final-redhat-1]
at org.hornetq.core.client.impl.ClientSessionFactoryImpl.getConnectionWithRetry(ClientSessionFactoryImpl.java:1103) [hornetq-core-client-2.3.25.Final-redhat-1.jar:2.3.25.Final-redhat-1]
at org.hornetq.core.client.impl.ClientSessionFactoryImpl.connect(ClientSessionFactoryImpl.java:253) [hornetq-core-client-2.3.25.Final-redhat-1.jar:2.3.25.Final-redhat-1]
at org.hornetq.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:748) [hornetq-core-client-2.3.25.Final-redhat-1.jar:2.3.25.Final-redhat-1]
at org.hornetq.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:704) [hornetq-core-client-2.3.25.Final-redhat-1.jar:2.3.25.Final-redhat-1]
at org.hornetq.core.server.cluster.impl.ClusterConnectionBridge.createSessionFactory(ClusterConnectionBridge.java:150) [hornetq-server-2.3.25.Final-redhat-1.jar:2.3.25.Final-redhat-1]
at org.hornetq.core.server.cluster.impl.BridgeImpl.connect(BridgeImpl.java:942) [hornetq-server-2.3.25.Final-redhat-1.jar:2.3.25.Final-redhat-1]
at org.hornetq.core.server.cluster.impl.BridgeImpl$ConnectRunnable.run(BridgeImpl.java:1160) [hornetq-server-2.3.25.Final-redhat-1.jar:2.3.25.Final-redhat-1]
at org.hornetq.utils.OrderedExecutorFactory$OrderedExecutor$1.run(OrderedExecutorFactory.java:105) [hornetq-core-client-2.3.25.Final-redhat-1.jar:2.3.25.Final-redhat-1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_102]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_102]
at java.lang.Thread.run(Thread.java:745) [rt.jar:1.8.0_102]

My pattern so far is -
%{HOUR:Hour}:%{MINUTE:Min}:%{SECOND:Sec} %{LOGLEVEL:LogLevel} [%{JAVACLASS:Class}]
and it works up until the class name.
I want to put the rest of the message into a Javastacktrace tag. It is a multiline message, and I know I need a multiline codec. I have tried with type "at", "{JAVASTACKTRACEPART} and {TIMESTAMP}. Can someone out there tell me how to properly use multiline parsing to do this? Thanks!Preformatted text

2

What kind of input plugin are you using (file, beats, ...)?

3

I am using filebeat as input.
My new pattern is
%{HOUR:Hour}:%{MINUTE:Min}:%{SECOND:Sec} %{LOGLEVEL:LogLevel} [%{JAVACLASS:Class}] (%{GREEDYDATA:MessageID}) %{GREEDYDATA:Error}\n%{GREEDYDATA:JavaStackTrace}

with the multiline pattern - "^\s"

It is working fine except %{GREEDYDATA:JavaStackTrace} is only accepting the first line.
Any thoughts?

4

Configure multiline on the Filebeat side. I think the Filebeat documentation contains configuration example that are quite close to what your multiline logs look like.

5

A simple (?m) in front of my pattern solved my problem.
Thanks for all your help!

1 Like
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.