We have a syserr messages from one of our WebApplication servers it is writing each line with timestamp we want to group these lines depending on their threadID you can see the example log file below. The challenging thing we couldn't manage to solve is file can contain different log messages with the same ThreadID. I mean thread1 is writing logs and then thread2 is writing logs then Thread1 can come and write different logs again we need to create 3 different logs from this. I've tried to use "aggregate" module however its grouping all ThreadID messages as one not creating different lines for the same ThreadID.
Logstash config:
grok {
# was_shortname need to be regex, because numbers and $ can be in the word
match => ["message", "[%{DATA:wastimestamp} %{WORD:tz}] %{BASE16NUM:was_threadID} (?<was_shortname>\b[A-Za-z0-9$]{2,}\b) %{SPACE}%{WORD:was_loglevel}%{SPACE} %{GREEDYDATA:logM}"]
overwrite => [ "message" ]
#tag_on_failure =>
}
grok {
# Extract the WebSphere Response Code
match => ["message", "(?<was_responsecode>[A-Z0-9]{9,10})[:,\s\s]"]
tag_on_failure =>
}
aggregate {
task_id => "%{was_threadID}"
code => "
map['logTime'] = event.get('logdate')
map['logMessage'] = event.get('logM') + '\n';
map['logLevel'] = event.get('was_loglevel');
map['threadID'] = event.get('was_threadID')
"
map_action => "create"
timeout => 1
push_map_as_event_on_timeout => true
}
aggregate {
task_id => "%{was_threadID}"
code => "
map['logMessage'] += event.get('logM') + '\n'
"
# Only aggregates if there was a recent deadlock for this same threadId
map_action => "update"
}
Log example:
| [03.12.2020 09:37:16:582 TRT] 0000015d SystemErr R | java.lang.NullPointerException |
|---|---|
| [03.12.2020 09:37:16:582 TRT] 0000015d SystemErr R | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56) |
| [03.12.2020 09:37:16:582 TRT] 0000015d SystemErr R | at java.lang.reflect.Method.invoke(Method.java:620) |
| [03.12.2020 09:37:16:582 TRT] 0000015d SystemErr R | at org.jboss.seam.util.Reflections.invoke(Reflections.java:22) |
| [03.12.2020 09:37:16:582 TRT] 0000015d SystemErr R | at org.jboss.seam.intercept.RootInvocationContext.proceed(RootInvocationContext.java:32) |
| [03.12.2020 09:37:16:582 TRT] 0000015d SystemErr R | at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:56) |
| [03.12.2020 09:55:43:750 TRT] 00000df0 SystemErr R | at java.lang.reflect.Method.invoke(Method.java:620) |
| [03.12.2020 09:55:43:750 TRT] 00000df0 SystemErr R | at org.jboss.el.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:335) |
| [03.12.2020 09:55:43:750 TRT] 00000df0 SystemErr R | at org.jboss.el.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:280) |
| [03.12.2020 09:55:43:750 TRT] 00000df0 SystemErr R | at org.jboss.el.parser.AstMethodSuffix.getValue(AstMethodSuffix.java:59) |
| [03.12.2020 09:55:43:750 TRT] 00000df0 SystemErr R | at org.jboss.el.parser.AstMethodSuffix.invoke(AstMethodSuffix.java:65) |
| [03.12.2020 09:55:43:751 TRT] 00000df0 SystemErr R | at org.jboss.el.parser.AstValue.invoke(AstValue.java:96) |
| [03.12.2020 09:55:43:751 TRT] 00000df0 SystemErr R | at org.jboss.el.MethodExpressionImpl.invoke(MethodExpressionImpl.java:276) |
| [03.12.2020 10:00:56:242 TRT] 0000015d SystemErr R | at java.lang.reflect.Method.invoke(Method.java:620) |
| [03.12.2020 10:00:56:242 TRT] 0000015d SystemErr R | at org.jboss.seam.util.Reflections.invoke(Reflections.java:22) |
| [03.12.2020 10:00:56:242 TRT] 0000015d SystemErr R | at org.jboss.seam.intercept.RootInvocationContext.proceed(RootInvocationContext.java:32) |
| [03.12.2020 10:00:56:242 TRT] 0000015d SystemErr R | at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:56) |