# Consolidate log lines sa one depending on the threadID information **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [December 8, 2020, 11:16am UTC](https://discuss.elastic.co/t/consolidate-log-lines-sa-one-depending-on-the-threadid-information/257978 "2020-12-08T11:16:44Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![secintic](https://avatars.discourse-cdn.com/v4/letter/s/e68b1a/32.png) [@secintic](https://discuss.elastic.co/u/secintic)\ **Post date:** [December 8, 2020, 11:16am UTC](https://discuss.elastic.co/t/consolidate-log-lines-sa-one-depending-on-the-threadid-information/257978/1 "2020-12-08T11:16:44Z")
We have a syserr messages from one of our WebApplication servers it is writing each line with timestamp we want to group these lines depending on their threadID you can see the example log file below. The challenging thing we couldn't manage to solve is file can contain different log messages with the same ThreadID. I mean thread1 is writing logs and then thread2 is writing logs then Thread1 can come and write different logs again we need to create 3 different logs from this. I've tried to use "aggregate" module however its grouping all ThreadID messages as one not creating different lines for the same ThreadID. Logstash config: grok { # was\_shortname need to be regex, because numbers and $ can be in the word match =\> ["message", "[%{DATA:wastimestamp} %{WORD:tz}] %{BASE16NUM:was\_threadID} (?\\b[A-Za-z0-9$]{2,}\b) %{SPACE}%{WORD:was\_loglevel}%{SPACE} %{GREEDYDATA:logM}"] overwrite =\> ["message"] #tag\_on\_failure =\> } grok { # Extract the WebSphere Response Code match =\> ["message", "(?\[A-Z0-9]{9,10})[:,\s\s]"] tag\_on\_failure =\> } aggregate { task\_id =\> "%{was\_threadID}" code =\> " map['logTime'] = event.get('logdate') map['logMessage'] = event.get('logM') + '\n'; map['logLevel'] = event.get('was\_loglevel'); map['threadID'] = event.get('was\_threadID') " map\_action =\> "create" timeout =\> 1 push\_map\_as\_event\_on\_timeout =\> true } aggregate { task\_id =\> "%{was\_threadID}" code =\> " map['logMessage'] += event.get('logM') + '\n' " # Only aggregates if there was a recent deadlock for this same threadId map\_action =\> "update" } Log example: | [03.12.2020 09:37:16:582 TRT] 0000015d SystemErr R | java.lang.NullPointerException | | --- | --- | | [03.12.2020 09:37:16:582 TRT] 0000015d SystemErr R | at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56) | | [03.12.2020 09:37:16:582 TRT] 0000015d SystemErr R | at java.lang.reflect.Method.invoke(Method.java:620) | | [03.12.2020 09:37:16:582 TRT] 0000015d SystemErr R | at org.jboss.seam.util.Reflections.invoke(Reflections.java:22) | | [03.12.2020 09:37:16:582 TRT] 0000015d SystemErr R | at org.jboss.seam.intercept.RootInvocationContext.proceed(RootInvocationContext.java:32) | | [03.12.2020 09:37:16:582 TRT] 0000015d SystemErr R | at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:56) | | [03.12.2020 09:55:43:750 TRT] 00000df0 SystemErr R | at java.lang.reflect.Method.invoke(Method.java:620) | | [03.12.2020 09:55:43:750 TRT] 00000df0 SystemErr R | at org.jboss.el.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:335) | | [03.12.2020 09:55:43:750 TRT] 00000df0 SystemErr R | at org.jboss.el.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:280) | | [03.12.2020 09:55:43:750 TRT] 00000df0 SystemErr R | at org.jboss.el.parser.AstMethodSuffix.getValue(AstMethodSuffix.java:59) | | [03.12.2020 09:55:43:750 TRT] 00000df0 SystemErr R | at org.jboss.el.parser.AstMethodSuffix.invoke(AstMethodSuffix.java:65) | | [03.12.2020 09:55:43:751 TRT] 00000df0 SystemErr R | at org.jboss.el.parser.AstValue.invoke(AstValue.java:96) | | [03.12.2020 09:55:43:751 TRT] 00000df0 SystemErr R | at org.jboss.el.MethodExpressionImpl.invoke(MethodExpressionImpl.java:276) | | [03.12.2020 10:00:56:242 TRT] 0000015d SystemErr R | at java.lang.reflect.Method.invoke(Method.java:620) | | [03.12.2020 10:00:56:242 TRT] 0000015d SystemErr R | at org.jboss.seam.util.Reflections.invoke(Reflections.java:22) | | [03.12.2020 10:00:56:242 TRT] 0000015d SystemErr R | at org.jboss.seam.intercept.RootInvocationContext.proceed(RootInvocationContext.java:32) | | [03.12.2020 10:00:56:242 TRT] 0000015d SystemErr R | at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:56) | ---
**Author:** ![eubarkana](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eubarkana/32/77317_2.png) [@eubarkana](https://discuss.elastic.co/u/eubarkana)\ **Post date:** [January 4, 2021, 1:54pm UTC](https://discuss.elastic.co/t/consolidate-log-lines-sa-one-depending-on-the-threadid-information/257978/2 "2021-01-04T13:54:27Z")
I am facing the same issue. Appreciate any ideas about how to resolve? ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [February 1, 2021, 3:54pm UTC](https://discuss.elastic.co/t/consolidate-log-lines-sa-one-depending-on-the-threadid-information/257978/3 "2021-02-01T15:54:34Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.