Logstash config question


(tei.chae) #1

Hi, guys.

I have some questions about logstash setup.

I want to use logstash to read the logs of our Tomcat through Kibana.

However, due to misconfiguration, the multi-line is not being displayed.

I need your help on this matter.

The following is the log of Tomcat:

2018:11:17 02:39:20.161 WARN  --- [redisMessageListenerContainer-22539] o.s.d.r.l.RedisMessageListenerContainer : Execution of message listener failed, and no ErrorHandler has been set. 
org.springframework.data.redis.serializer.SerializationException: Cannot deserialize; nested exception is org.springframework.core.serializer.support.SerializationFailedException: Failed to deserialize payload. Is the byte array a result of corresponding serialization for DefaultDeserializer?; nested exception is java.io.InvalidClassException: com.ourtomcat.member.vo.MemberVO; local class incompatible: stream classdesc serialVersionUID = local class serialVersionUID = -8722833363164466383
	at org.springframework.data.redis.serializer.JdkSerializationRedisSerializer.deserialize(JdkSerializationRedisSerializer.java:82)
	at org.springframework.data.redis.core.AbstractOperations.deserializeHashValue(AbstractOperations.java:338)
	at org.springframework.data.redis.core.AbstractOperations.deserializeHashMap(AbstractOperations.java:282)
	at org.springframework.data.redis.core.DefaultHashOperations.entries(DefaultHashOperations.java:227)
	at org.springframework.data.redis.core.DefaultBoundHashOperations.entries(DefaultBoundHashOperations.java:102)
	at org.springframework.session.data.redis.RedisOperationsSessionRepository.getSession(RedisOperationsSessionRepository.java:432)
	at org.springframework.session.data.redis.RedisOperationsSessionRepository.onMessage(RedisOperationsSessionRepository.java:519)
	at org.springframework.data.redis.listener.RedisMessageListenerContainer.executeListener(RedisMessageListenerContainer.java:249)
	at org.springframework.data.redis.listener.RedisMessageListenerContainer.processMessage(RedisMessageListenerContainer.java:239)
	at org.springframework.data.redis.listener.RedisMessageListenerContainer$1.run(RedisMessageListenerContainer.java:967)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.springframework.core.serializer.support.SerializationFailedException: Failed to deserialize payload. Is the byte array a result of corresponding serialization for DefaultDeserializer?; nested exception is java.io.InvalidClassException: com.ourtomcat.member.vo.MemberVO; local class incompatible: stream classdesc serialVersionUID = 3092639245606648778, local class serialVersionUID = -8722833363164466383
	at org.springframework.core.serializer.support.DeserializingConverter.convert(DeserializingConverter.java:78)
	at org.springframework.core.serializer.support.DeserializingConverter.convert(DeserializingConverter.java:36)
	at org.springframework.data.redis.serializer.JdkSerializationRedisSerializer.deserialize(JdkSerializationRedisSerializer.java:80)
	... 10 common frames omitted
Caused by: java.io.InvalidClassException: com.ourtomcat.member.vo.MemberVO; local class incompatible: stream classdesc serialVersionUID = 3092639245606648778, local class serialVersionUID = -8722833363164466383
	at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:699)
	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1885)
	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1751)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2042)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
	at org.springframework.core.serializer.DefaultDeserializer.deserialize(DefaultDeserializer.java:70)
	at org.springframework.core.serializer.support.DeserializingConverter.convert(DeserializingConverter.java:73)
	... 12 common frames omitted
2018:11:17 04:48:38.550 INFO  --- [https-jsse-nio-8443-exec-4] com.ourtomcat.LoginInterceptor : Request URI / 
2018:11:17 09:09:19.504 INFO  --- [http-nio-8080-exec-3] o.a.coyote.http11.Http11Processor : Error parsing HTTP request header
 Note: further occurrences of HTTP header parsing errors will be logged at DEBUG level. 
java.lang.IllegalArgumentException: Invalid character found in method name. HTTP method names must be tokens
	at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:428)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:687)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1468)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:748)
2018:11:17 09:42:09.198 INFO  --- [https-jsse-nio-8443-exec-53] com.ourtomcat.LoginInterceptor : Request URI / 

Next is config of filebeat.

- type: log
  enabled: true
  paths:
    - /home/ubuntu/tomcat-log-**-*.log
  tags : ["service"]
  multiline.pattern: '[0-9]{4}-[0-9]{2}-[0-9]{2} [0-9]{2}'
  multiline.negate: true
  multiline.match: after

What settings should I modify here?

I look forward to your kind help.

Thank you.


(tei.chae) #2

I'm very bothered because there's a problem with the multi-line output failing and only a partial message being displayed.


(Christian Dahlqvist) #3

Your multiline config looks for lines starting with e.g. 2018-11-17 02 while your log file has the format 2018:11:17 02. You therefore need to replace the - with: in your multiline statement.


(tei.chae) #4

Hi, Christian_Dahlqvist

If I change the filebeat setting as follows, I wonder if it will not be the configuration I want.

Multiline.pattern: '^[:space:]+(at|.{3})\b|^Caused by:'
Multiline.negate: false
Multiline.match: after

What do you think?

Thank you for your quick and kind reply.


(Christian Dahlqvist) #5

Why did you change it to that and not what I suggested?


(tei.chae) #6

No, I found the reference late, but I heard this suggestion.

I was wondering what it was like.


(tei.chae) #7

finished.

If change the settings in filebeat as shown above, a log pattern will be created.