# How to parse WebSphere SystemErr log in logstash

**URL:** https://discuss.elastic.co/t/how-to-parse-websphere-systemerr-log-in-logstash/252323
**Category:** Logstash
**Created:** [October 16, 2020, 9:20am UTC](https://discuss.elastic.co/t/how-to-parse-websphere-systemerr-log-in-logstash/252323 "2020-10-16T09:20:06Z")
**Posts on this page:** 16
**Page:** 1

<div class="post-metadata">

### Author: ![eubarkana](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eubarkana/32/77317_2.png) [@eubarkana](https://discuss.elastic.co/u/eubarkana)
#### Post date: [October 16, 2020, 9:20am UTC](https://discuss.elastic.co/t/how-to-parse-websphere-systemerr-log-in-logstash/252323/1 "2020-10-16T09:20:06Z")

</div>

IBM WebSphere logs include SystemOut and SystemErr logs for a JVM. Each message in SystemOut is enabled with timestamp that can be parsed with logstash. Exceptions in this log has stack trace but does not include timestamp in front so that they can be combined into one message. Sample of SystemOut.log:  
// Sample code  
\</\>  
[02.10.2020 12:00:00:742 TRT] 000000bc SystemOut O Fri Oct 02 12:00:00 TRT 2020 : PusulaQuartzJobListener.END : AsynchronousInvocation(pusulaQueueJob.asyncScan())  
[02.10.2020 12:01:26:224 TRT] 0000005f NGUtil$Server I ASND0002I: Detected server alfa102 started on node valyriaNode01  
[02.10.2020 12:04:04:835 TRT] 000001c0 filter E com.ibm.ws.webcontainer.filter.FilterInstanceWrapper doFilter SRVE8109W: Uncaught exception thrown by filter Seam Filter: java.io.FileNotFoundException: SRVE0190E: File not found: /favicon.ico  
at com.ibm.ws.webcontainer.extension.DefaultExtensionProcessor.\_processEDR(DefaultExtensionProcessor.java:976)  
at com.ibm.ws.webcontainer.extension.DefaultExtensionProcessor.processEDR(DefaultExtensionProcessor.java:957)  
at com.ibm.ws.webcontainer.extension.DefaultExtensionProcessor.handleRequest(DefaultExtensionProcessor.java:485)  
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:136)  
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:97)  
at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:83)  
at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:73)  
at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:73)  
at org.jboss.seam.web.IdentityFilter.doFilter(IdentityFilter.java:40)  
at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)  
at org.jboss.seam.web.MultipartFilter.doFilter(MultipartFilter.java:90)  
at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)  
at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:73)  
at org.jboss.seam.web.RedirectFilter.doFilter(RedirectFilter.java:45)  
at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)  
at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:73)  
at org.ajax4jsf.webapp.BaseXMLFilter.doXmlFilter(BaseXMLFilter.java:206)  
at org.ajax4jsf.webapp.BaseFilter.handleRequest(BaseFilter.java:290)  
at org.ajax4jsf.webapp.BaseFilter.processUploadsAndHandleRequest(BaseFilter.java:388)  
at org.ajax4jsf.webapp.BaseFilter.doFilter(BaseFilter.java:515)  
at org.jboss.seam.web.Ajax4jsfFilter.doFilter(Ajax4jsfFilter.java:56)  
at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)  
at org.jboss.seam.web.LoggingFilter.doFilter(LoggingFilter.java:60)  
at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)  
at org.jboss.seam.servlet.SeamFilter.doFilter(SeamFilter.java:158)  
at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:195)  
at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91)  
at org.jboss.seam.web.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:42)  
\</\>  
Up to here parsing is fine. But when it comes to SystemErr.log each message has timestamp in front even if the stack trace has timestamp. Sample for SystemErr.log:  
Starting of an error message:  
\</\>  
[18.09.2020 08:49:07:557 TRT] 0000016a SystemErr R org.mozilla.javascript.WrappedException: Wrapped java.lang.NullPointerException (Komut[700988]#5)  
[18.09.2020 08:49:07:558 TRT] 0000016a SystemErr R at org.mozilla.javascript.Context.throwAsScriptRuntimeEx(Context.java:1754)  
[18.09.2020 08:49:07:558 TRT] 0000016a SystemErr R at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:148)  
[18.09.2020 08:49:07:558 TRT] 0000016a SystemErr R at org.mozilla.javascript.NativeJavaMethod.call(NativeJavaMethod.java:225)  
[18.09.2020 08:49:07:559 TRT] 0000016a SystemErr R at org.mozilla.javascript.optimizer.OptRuntime.call1(OptRuntime.java:32)  
[18.09.2020 08:49:07:559 TRT] 0000016a SystemErr R at org.mozilla.javascript.gen.Komut\_700988\_\_7.\_c\_f\_1(Komut[700988]:5)  
[18.09.2020 08:49:07:559 TRT] 0000016a SystemErr R at org.mozilla.javascript.gen.Komut\_700988\_\_7.call(Komut[700988])  
[18.09.2020 08:49:07:559 TRT] 0000016a SystemErr R at org.mozilla.javascript.optimizer.OptRuntime.callName0(OptRuntime.java:74)  
[18.09.2020 08:49:07:559 TRT] 0000016a SystemErr R at org.mozilla.javascript.gen.Komut\_700988\_\_7.\_c\_script\_0(Komut[700988]:8)  
[18.09.2020 08:49:07:559 TRT] 0000016a SystemErr R at org.mozilla.javascript.gen.Komut\_700988\_\_7.call(Komut[700988])  
[18.09.2020 08:49:07:559 TRT] 0000016a SystemErr R at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:394)  
[18.09.2020 08:49:07:559 TRT] 0000016a SystemErr R at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3091)  
[18.09.2020 08:49:07:559 TRT] 0000016a SystemErr R at org.mozilla.javascript.gen.Komut\_700988\_\_7.call(Komut[700988])  
[18.09.2020 08:49:07:559 TRT] 0000016a SystemErr R at org.mozilla.javascript.gen.Komut\_700988\_\_7.exec(Komut[700988])  
...  
[18.09.2020 08:49:07:567 TRT] 0000016a SystemErr R Caused by: java.lang.NullPointerException  
[18.09.2020 08:49:07:568 TRT] 0000016a SystemErr R at com.gunessigorta.pusula.uretim.UretimScriptBean.endeksliDovizCinsiniAyarla(UretimScriptBean.java:6007)  
[18.09.2020 08:49:07:568 TRT] 0000016a SystemErr R at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  
[18.09.2020 08:49:07:568 TRT] 0000016a SystemErr R at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)  
[18.09.2020 08:49:07:568 TRT] 0000016a SystemErr R at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)  
[18.09.2020 08:49:07:568 TRT] 0000016a SystemErr R at java.lang.reflect.Method.invoke(Method.java:620)  
[18.09.2020 08:49:07:568 TRT] 0000016a SystemErr R at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:126)  
[18.09.2020 08:49:07:568 TRT] 0000016a SystemErr R ... 112 more  
[18.09.2020 08:49:07:577 TRT] 0000016a SystemErr R com.filika.sigorta.exceptions.IceAktarimException: Döviz cinsi kontrolü@RISK\_BILGILERI[282456].end.fail, Teklif SIGORTA\_KONULARI kuralları çalıştırılırken hata oluştu!  
\</\>  
How can I combine the same threadID lines as one message if the message does not start with "at"? I could not find how to write this rule in logstash. Pseudo could is like this  
If the log is SystemErr  
if threadID is same  
if message is not starting by at  
starting a new message as msgReal  
else message starts with at  
append the message to msgReal  
else  
set as new msgReal

should parse the SystemErr but not sure how to implement it in logstash. I appreciate your help. Thanks.

---

<div class="post-metadata">

### Author: ![Yigit\_Ozdemir](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yigit_ozdemir/32/77866_2.png) [@Yigit\_Ozdemir](https://discuss.elastic.co/u/Yigit_Ozdemir)
#### Post date: [October 26, 2020, 10:46am UTC](https://discuss.elastic.co/t/how-to-parse-websphere-systemerr-log-in-logstash/252323/2 "2020-10-26T10:46:49Z")

</div>

Hello Utku,

I am working on exactly same issue, do you have any good news to help me?

---

<div class="post-metadata">

### Author: ![eubarkana](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eubarkana/32/77317_2.png) [@eubarkana](https://discuss.elastic.co/u/eubarkana)
#### Post date: [October 28, 2020, 7:24am UTC](https://discuss.elastic.co/t/how-to-parse-websphere-systemerr-log-in-logstash/252323/3 "2020-10-28T07:24:19Z")

</div>

I appreciate if anyone helps me to parse the SystemErr.log of WebSphere with logstash.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [October 28, 2020, 4:19pm UTC](https://discuss.elastic.co/t/how-to-parse-websphere-systemerr-log-in-logstash/252323/4 "2020-10-28T16:19:58Z")

</div>

Please edit your post, select the sections of log file, and click on \</\> in the toolbar above the edit pane. This change the formatting of the log files to be like this

```
[18.09.2020 08:49:07:568 TRT] 0000016a SystemErr R at java.lang.reflect.Method.invoke(Method.java:620)
[18.09.2020 08:49:07:568 TRT] 0000016a SystemErr R at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:126)

```

making it clear where the line breaks are.

---

<div class="post-metadata">

### Author: ![eubarkana](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eubarkana/32/77317_2.png) [@eubarkana](https://discuss.elastic.co/u/eubarkana)
#### Post date: [November 1, 2020, 9:13pm UTC](https://discuss.elastic.co/t/how-to-parse-websphere-systemerr-log-in-logstash/252323/5 "2020-11-01T21:13:09Z")

</div>

> [@eubarkana](#):
>
> IBM WebSphere logs include SystemOut and SystemErr logs for a JVM. Each message in SystemOut is enabled with timestamp that can be parsed with logstash. Exceptions in this log has stack trace but does not include timestamp in front so that they can be combined into one message. Sample of SystemOut.log:  
> // Sample code
> 
> ```
> [02.10.2020 12:00:00:742 TRT] 000000bc SystemOut O Fri Oct 02 12:00:00 TRT 2020 : PusulaQuartzJobListener.END : AsynchronousInvocation(pusulaQueueJob.asyncScan())
> [02.10.2020 12:01:26:224 TRT] 0000005f NGUtil$Server I ASND0002I: Detected server alfa102 started on node valyriaNode01
> [02.10.2020 12:04:04:835 TRT] 000001c0 filter E com.ibm.ws.webcontainer.filter.FilterInstanceWrapper doFilter SRVE8109W: Uncaught exception thrown by filter Seam Filter: java.io.FileNotFoundException: SRVE0190E: File not found: /favicon.ico
> at com.ibm.ws.webcontainer.extension.DefaultExtensionProcessor._processEDR(DefaultExtensionProcessor.java:976)
> at com.ibm.ws.webcontainer.extension.DefaultExtensionProcessor.processEDR(DefaultExtensionProcessor.java:957)
> at com.ibm.ws.webcontainer.extension.DefaultExtensionProcessor.handleRequest(DefaultExtensionProcessor.java:485)
> at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:136)
> at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:97)
> at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:83)
> at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:73)
> at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:73)
> at org.jboss.seam.web.IdentityFilter.doFilter(IdentityFilter.java:40)
> at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
> at org.jboss.seam.web.MultipartFilter.doFilter(MultipartFilter.java:90)
> at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
> at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:73)
> at org.jboss.seam.web.RedirectFilter.doFilter(RedirectFilter.java:45)
> at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
> at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:73)
> at org.ajax4jsf.webapp.BaseXMLFilter.doXmlFilter(BaseXMLFilter.java:206)
> at org.ajax4jsf.webapp.BaseFilter.handleRequest(BaseFilter.java:290)
> at org.ajax4jsf.webapp.BaseFilter.processUploadsAndHandleRequest(BaseFilter.java:388)
> at org.ajax4jsf.webapp.BaseFilter.doFilter(BaseFilter.java:515)
> at org.jboss.seam.web.Ajax4jsfFilter.doFilter(Ajax4jsfFilter.java:56)
> at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
> at org.jboss.seam.web.LoggingFilter.doFilter(LoggingFilter.java:60)
> at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69)
> at org.jboss.seam.servlet.SeamFilter.doFilter(SeamFilter.java:158)
> at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:195)
> at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:91)
> at org.jboss.seam.web.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:42)
> 
> ```
> 
> Up to here parsing is fine. But when it comes to SystemErr.log each message has timestamp in front even if the stack trace has timestamp. Sample for SystemErr.log:  
> Starting of an error message:
> 
> ```
> [18.09.2020 08:49:07:557 TRT] 0000016a SystemErr R org.mozilla.javascript.WrappedException: Wrapped java.lang.NullPointerException (Komut[700988]#5)
> [18.09.2020 08:49:07:558 TRT] 0000016a SystemErr R at org.mozilla.javascript.Context.throwAsScriptRuntimeEx(Context.java:1754)
> [18.09.2020 08:49:07:558 TRT] 0000016a SystemErr R at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:148)
> [18.09.2020 08:49:07:558 TRT] 0000016a SystemErr R at org.mozilla.javascript.NativeJavaMethod.call(NativeJavaMethod.java:225)
> [18.09.2020 08:49:07:559 TRT] 0000016a SystemErr R at org.mozilla.javascript.optimizer.OptRuntime.call1(OptRuntime.java:32)
> [18.09.2020 08:49:07:559 TRT] 0000016a SystemErr R at org.mozilla.javascript.gen.Komut_700988__7._c_f_1(Komut[700988]:5)
> [18.09.2020 08:49:07:559 TRT] 0000016a SystemErr R at org.mozilla.javascript.gen.Komut_700988__7.call(Komut[700988])
> [18.09.2020 08:49:07:559 TRT] 0000016a SystemErr R at org.mozilla.javascript.optimizer.OptRuntime.callName0(OptRuntime.java:74)
> [18.09.2020 08:49:07:559 TRT] 0000016a SystemErr R at org.mozilla.javascript.gen.Komut_700988__7._c_script_0(Komut[700988]:8)
> [18.09.2020 08:49:07:559 TRT] 0000016a SystemErr R at org.mozilla.javascript.gen.Komut_700988__7.call(Komut[700988])
> [18.09.2020 08:49:07:559 TRT] 0000016a SystemErr R at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:394)
> [18.09.2020 08:49:07:559 TRT] 0000016a SystemErr R at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3091)
> [18.09.2020 08:49:07:559 TRT] 0000016a SystemErr R at org.mozilla.javascript.gen.Komut_700988__7.call(Komut[700988])
> [18.09.2020 08:49:07:559 TRT] 0000016a SystemErr R at org.mozilla.javascript.gen.Komut_700988__7.exec(Komut[700988])
> ...
> [18.09.2020 08:49:07:567 TRT] 0000016a SystemErr R Caused by: java.lang.NullPointerException
> [18.09.2020 08:49:07:568 TRT] 0000016a SystemErr R at com.gunessigorta.pusula.uretim.UretimScriptBean.endeksliDovizCinsiniAyarla(UretimScriptBean.java:6007)
> [18.09.2020 08:49:07:568 TRT] 0000016a SystemErr R at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [18.09.2020 08:49:07:568 TRT] 0000016a SystemErr R at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
> [18.09.2020 08:49:07:568 TRT] 0000016a SystemErr R at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
> [18.09.2020 08:49:07:568 TRT] 0000016a SystemErr R at java.lang.reflect.Method.invoke(Method.java:620)
> [18.09.2020 08:49:07:568 TRT] 0000016a SystemErr R at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:126)
> [18.09.2020 08:49:07:568 TRT] 0000016a SystemErr R ... 112 more
> [18.09.2020 08:49:07:577 TRT] 0000016a SystemErr R com.filika.sigorta.exceptions.IceAktarimException: Döviz cinsi kontrolü@RISK_BILGILERI[282456].end.fail, Teklif SIGORTA_KONULARI kuralları çalıştırılırken hata oluştu!
> 
> ```
> 
> How can I combine the same threadID lines as one message if the message does not start with "at"? I could not find how to write this rule in logstash. Pseudo could is like this  
> If the log is SystemErr  
> if threadID is same  
> if message is not starting by at  
> starting a new message as msgReal  
> else message starts with at  
> append the message to msgReal  
> else  
> set as new msgReal
> 
> should parse the SystemErr but not sure how to implement it in logstash. I appreciate your help. Thanks.

I re-formatted the log sections.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [November 1, 2020, 9:51pm UTC](https://discuss.elastic.co/t/how-to-parse-websphere-systemerr-log-in-logstash/252323/6 "2020-11-01T21:51:52Z")

</div>

I dropped both those log sections into a file and ingested it with a file input and this codec:

```
 codec => multiline {
      pattern => '^(\[\d{2}\.\d{2}\.\d{4} \d{2}:\d{2}:\d{2}:\d{3} \w{3}\] [0-9a-f]{8} SystemErr \w )?at' 
      negate => false 
      what => previous 
      auto_flush_interval => 1
 }

```

Basically build a pattern that matches "[18.09.2020 08:49:07:559 TRT] 0000016a SystemErr R " and make it optional by surrounding it with ( )?

If WebSphere interleaves lines from two different stack traces from two different threads you would have to use an aggregate filter.

---

<div class="post-metadata">

### Author: ![eubarkana](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eubarkana/32/77317_2.png) [@eubarkana](https://discuss.elastic.co/u/eubarkana)
#### Post date: [November 2, 2020, 6:16am UTC](https://discuss.elastic.co/t/how-to-parse-websphere-systemerr-log-in-logstash/252323/7 "2020-11-02T06:16:47Z")

</div>

My logstash file has this portion

```
filter {
  if ([fields][log_type] == "was") {
    grok {
          # was_shortname need to be regex, because numbers and $ can be in the word
          match => ["message", "\[%{DATA:wastimestamp} %{WORD:tz}\] %{BASE16NUM:was_threadID} (?<was_shortname>\b[A-Za-z0-9\$]{2,}\b) %{SPACE}%{WORD:was_loglevel}%{SPACE} %{GREEDYDATA:logmessage}"]
          overwrite => ["message"]
          #tag_on_failure => []
    }
    

    if ([was_shortname] == "SystemErr") {
    	aggregate {
        task_id => "%{was_threadID}"
    		code => " map['logmessage'] ||= [] ;
                  map['logmessage'] << event.get('logmessage')"
        push_previous_map_as_event => true
    	}
    }
    else{
      grok {
          # Extract the WebSphere Response Code
          match => ["message", "(?<was_responsecode>[A-Z0-9]{9,10})[:,\s\s]"]
          tag_on_failure => []
      }
      date{
                match => ["timestamp", "dd/MM/YY HH:mm:ss:SSS", "M/d/YY HH:mm:ss:SSS", "MM/d/YY HH:mm:ss:SSS", "M/dd/YY HH:mm:ss:SSS", "MM/dd/YY H:mm:ss:SSS", "M/d/YY H:mm:ss:SSS", "MM/d/YY H:mm:ss:SSS", "M/dd/YY H:mm:ss:SSS"]
      }
    }
  }

```

So should I place your part into

```
   if ([was_shortname] == "SystemErr") {
    	aggregate {
        task_id => "%{was_threadID}"

```

section?

Thanks.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [November 2, 2020, 2:29pm UTC](https://discuss.elastic.co/t/how-to-parse-websphere-systemerr-log-in-logstash/252323/8 "2020-11-02T14:29:19Z")

</div>

No, a codec applies to an input, not a filter.

---

<div class="post-metadata">

### Author: ![eubarkana](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eubarkana/32/77317_2.png) [@eubarkana](https://discuss.elastic.co/u/eubarkana)
#### Post date: [November 2, 2020, 2:34pm UTC](https://discuss.elastic.co/t/how-to-parse-websphere-systemerr-log-in-logstash/252323/9 "2020-11-02T14:34:33Z")

</div>

Our input is:

```
input {
  beats {
    port => 5000
  }
}

```

but we need filter to write our parsing sections. How can we write the same parsing within the filter aggregation?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [November 2, 2020, 2:56pm UTC](https://discuss.elastic.co/t/how-to-parse-websphere-systemerr-log-in-logstash/252323/10 "2020-11-02T14:56:34Z")

</div>

If you are using filebeat to read the logs then you can do equivalent multiline processing in filebeat.

---

<div class="post-metadata">

### Author: ![eubarkana](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eubarkana/32/77317_2.png) [@eubarkana](https://discuss.elastic.co/u/eubarkana)
#### Post date: [November 2, 2020, 3:18pm UTC](https://discuss.elastic.co/t/how-to-parse-websphere-systemerr-log-in-logstash/252323/11 "2020-11-02T15:18:07Z")

</div>

I appreciate if you could show me how I could write it. This syntax is not ringing any bell about how to write it in filebeat.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [November 2, 2020, 3:25pm UTC](https://discuss.elastic.co/t/how-to-parse-websphere-systemerr-log-in-logstash/252323/12 "2020-11-02T15:25:01Z")

</div>

I think the filebeat equivalent of

> [@Badger](#):
>
> ```auto
> codec => multiline {
> pattern => '^(\[\d{2}\.\d{2}\.\d{4} \d{2}:\d{2}:\d{2}:\d{3} \w{3}\] [0-9a-f]{8} SystemErr \w )?at' 
> negate => false 
> what => previous 
> auto_flush_interval => 1
> }
> 
> ```

would be

```auto
multiline.pattern: '^(\[\d{2}\.\d{2}\.\d{4} \d{2}:\d{2}:\d{2}:\d{3} \w{3}\] [0-9a-f]{8} SystemErr \w )?at'
multiline.match: after

```

---

<div class="post-metadata">

### Author: ![eubarkana](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eubarkana/32/77317_2.png) [@eubarkana](https://discuss.elastic.co/u/eubarkana)
#### Post date: [November 3, 2020, 12:20pm UTC](https://discuss.elastic.co/t/how-to-parse-websphere-systemerr-log-in-logstash/252323/13 "2020-11-03T12:20:57Z")

</div>

Thanks Badger, that solved the main parsing problem we had.  
During our testing we saw one thing that I wanted to ask you. Filebeat does parse the log according to "at" which is fine. But considering the rows below:

```
[03.11.2020 11:59:51:603 TRT] 000011db SystemErr R Caused by: java.lang.NullPointerException
[03.11.2020 11:59:51:603 TRT] 000011db SystemErr R at java.util.ArrayList.addAll(ArrayList.java:576)
[03.11.2020 11:59:51:603 TRT] 000011db SystemErr R at com.gunessigorta.pusula.uretim.UretimScriptBean.getRiskGrubuList(UretimScriptBean.java:6803)
[03.11.2020 11:59:51:603 TRT] 000011db SystemErr R at com.gunessigorta.pusula.uretim.UretimScriptBean.musteriYuksekRiskGrubundaMi(UretimScriptBean.java:6688)
[03.11.2020 11:59:51:603 TRT] 000011db SystemErr R at sun.reflect.GeneratedMethodAccessor2110.invoke(Unknown Source)
[03.11.2020 11:59:51:603 TRT] 000011db SystemErr R at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
[03.11.2020 11:59:51:603 TRT] 000011db SystemErr R at java.lang.reflect.Method.invoke(Method.java:620)
[03.11.2020 11:59:51:603 TRT] 000011db SystemErr R at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:126)
[03.11.2020 11:59:51:603 TRT] 000011db SystemErr R ... 120 more
[03.11.2020 12:02:19:850 TRT] 00000a23 SystemErr R RollbackWrapperInterceptor.printStackTrace(): transaction rollback işaretlenecek!!! #null
[03.11.2020 12:02:19:850 TRT] 00000a23 SystemErr R java.lang.IllegalStateException: No conversation context active
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at org.jboss.seam.ScopeType.getContext(ScopeType.java:133)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at org.jboss.seam.Component.getInstance(Component.java:2002)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at org.jboss.seam.Component.getInstance(Component.java:1997)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at com.gunessigorta.pusula.framework.service.base.ServiceUtils.getComponentInstance(ServiceUtils.java:858)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at com.gunessigorta.pusula.framework.service.base.ServiceUtils.getComponentInstance(ServiceUtils.java:854)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at com.gunessigorta.pusula.entegrasyon.business.impl.EntegrasyonKullanici.em(EntegrasyonKullanici.java:99)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at com.gunessigorta.pusula.entegrasyon.business.impl.EntegrasyonKullanici.getKullaniciOkunmamisMesajSayisi(EntegrasyonKullanici.java:593)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at com.gunessigorta.pusula.entegrasyon.business.domain.ActiveUser.getOkunmamisMesajSayisi(ActiveUser.java:326)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at sun.reflect.GeneratedMethodAccessor242.invoke(Unknown Source)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at java.lang.reflect.Method.invoke(Method.java:620)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at org.jboss.seam.util.Reflections.invoke(Reflections.java:22)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at org.jboss.seam.intercept.RootInvocationContext.proceed(RootInvocationContext.java:32)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:56)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at com.gunessigorta.pusula.seam.RollbackWrapperInterceptor.aroundInvoke(RollbackWrapperInterceptor.java:47)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at org.jboss.seam.transaction.RollbackInterceptor.aroundInvoke(RollbackInterceptor.java:28)
[03.11.2020 12:02:19:853 TRT] 00000a23 SystemErr R at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)
[03.11.2020 12:02:19:853 TRT] 00000a23 SystemErr R at org.jboss.seam.core.MethodContextInterceptor.aroundInvoke(MethodContextInterceptor.java:44)
[03.11.2020 12:02:19:853 TRT] 00000a23 SystemErr R at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)
[03.11.2020 12:02:19:853 TRT] 00000a23 SystemErr R at org.jboss.seam.core.SynchronizationInterceptor.aroundInvoke(SynchronizationInterceptor.java:32)
[03.11.2020 12:02:19:853 TRT] 00000a23 SystemErr R at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)
[03.11.2020 12:02:19:853 TRT] 00000a23 SystemErr R at org.jboss.seam.intercept.RootInterceptor.invoke(RootInterceptor.java:107)
[03.11.2020 12:02:19:853 TRT] 00000a23 SystemErr R at org.jboss.seam.intercept.JavaBeanInterceptor.interceptInvocation(JavaBeanInterceptor.java:185)
[03.11.2020 12:02:19:853 TRT] 00000a23 SystemErr R at org.jboss.seam.intercept.JavaBeanInterceptor.invoke(JavaBeanInterceptor.java:103)

```

This is just a portion of the log and I noticed that the parsing of those lines are done as four lines

Section 1:

```
[03.11.2020 11:59:51:603 TRT] 000011db SystemErr R Caused by: java.lang.NullPointerException
[03.11.2020 11:59:51:603 TRT] 000011db SystemErr R at java.util.ArrayList.addAll(ArrayList.java:576)
[03.11.2020 11:59:51:603 TRT] 000011db SystemErr R at com.gunessigorta.pusula.uretim.UretimScriptBean.getRiskGrubuList(UretimScriptBean.java:6803)
[03.11.2020 11:59:51:603 TRT] 000011db SystemErr R at com.gunessigorta.pusula.uretim.UretimScriptBean.musteriYuksekRiskGrubundaMi(UretimScriptBean.java:6688)
[03.11.2020 11:59:51:603 TRT] 000011db SystemErr R at sun.reflect.GeneratedMethodAccessor2110.invoke(Unknown Source)
[03.11.2020 11:59:51:603 TRT] 000011db SystemErr R at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
[03.11.2020 11:59:51:603 TRT] 000011db SystemErr R at java.lang.reflect.Method.invoke(Method.java:620)
[03.11.2020 11:59:51:603 TRT] 000011db SystemErr R at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:126)

```

Section 2:

`[03.11.2020 11:59:51:603 TRT] 000011db SystemErr R ... 120 more`

Section 3:

[03.11.2020 12:02:19:850 TRT] 00000a23 SystemErr R RollbackWrapperInterceptor.printStackTrace(): transaction rollback işaretlenecek!!! #null

Section 4:

```
[03.11.2020 12:02:19:850 TRT] 00000a23 SystemErr R java.lang.IllegalStateException: No conversation context active
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at org.jboss.seam.ScopeType.getContext(ScopeType.java:133)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at org.jboss.seam.Component.getInstance(Component.java:2002)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at org.jboss.seam.Component.getInstance(Component.java:1997)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at com.gunessigorta.pusula.framework.service.base.ServiceUtils.getComponentInstance(ServiceUtils.java:858)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at com.gunessigorta.pusula.framework.service.base.ServiceUtils.getComponentInstance(ServiceUtils.java:854)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at com.gunessigorta.pusula.entegrasyon.business.impl.EntegrasyonKullanici.em(EntegrasyonKullanici.java:99)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at com.gunessigorta.pusula.entegrasyon.business.impl.EntegrasyonKullanici.getKullaniciOkunmamisMesajSayisi(EntegrasyonKullanici.java:593)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at com.gunessigorta.pusula.entegrasyon.business.domain.ActiveUser.getOkunmamisMesajSayisi(ActiveUser.java:326)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at sun.reflect.GeneratedMethodAccessor242.invoke(Unknown Source)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at java.lang.reflect.Method.invoke(Method.java:620)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at org.jboss.seam.util.Reflections.invoke(Reflections.java:22)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at org.jboss.seam.intercept.RootInvocationContext.proceed(RootInvocationContext.java:32)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:56)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at com.gunessigorta.pusula.seam.RollbackWrapperInterceptor.aroundInvoke(RollbackWrapperInterceptor.java:47)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)
[03.11.2020 12:02:19:852 TRT] 00000a23 SystemErr R at org.jboss.seam.transaction.RollbackInterceptor.aroundInvoke(RollbackInterceptor.java:28)
[03.11.2020 12:02:19:853 TRT] 00000a23 SystemErr R at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)
[03.11.2020 12:02:19:853 TRT] 00000a23 SystemErr R at org.jboss.seam.core.MethodContextInterceptor.aroundInvoke(MethodContextInterceptor.java:44)
[03.11.2020 12:02:19:853 TRT] 00000a23 SystemErr R at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)
[03.11.2020 12:02:19:853 TRT] 00000a23 SystemErr R at org.jboss.seam.core.SynchronizationInterceptor.aroundInvoke(SynchronizationInterceptor.java:32)
[03.11.2020 12:02:19:853 TRT] 00000a23 SystemErr R at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68)
[03.11.2020 12:02:19:853 TRT] 00000a23 SystemErr R at org.jboss.seam.intercept.RootInterceptor.invoke(RootInterceptor.java:107)
[03.11.2020 12:02:19:853 TRT] 00000a23 SystemErr R at org.jboss.seam.intercept.JavaBeanInterceptor.interceptInvocation(JavaBeanInterceptor.java:185)
[03.11.2020 12:02:19:853 TRT] 00000a23 SystemErr R at org.jboss.seam.intercept.JavaBeanInterceptor.invoke(JavaBeanInterceptor.java:103)

```

When we take a look at the thread IDs, section 3 is the beginning of section 4, so is there a way we can combine those two sections since the thread IDs

`00000a23`

match?

Thanks alot. I really appreciate you guidance.

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [November 3, 2020, 3:58pm UTC](https://discuss.elastic.co/t/how-to-parse-websphere-systemerr-log-in-logstash/252323/14 "2020-11-03T15:58:21Z")

</div>

Combining section 2 into section 1 would be easy, but using an alternation like

```
 (at|\.\.\.)

```

Combining 3 and 4 is much harder. It is not clear to me that a regexp based approach will work at all.

---

<div class="post-metadata">

### Author: ![eubarkana](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eubarkana/32/77317_2.png) [@eubarkana](https://discuss.elastic.co/u/eubarkana)
#### Post date: [November 4, 2020, 6:18am UTC](https://discuss.elastic.co/t/how-to-parse-websphere-systemerr-log-in-logstash/252323/15 "2020-11-04T06:18:37Z")

</div>

We used this for filebeat

```
- type: log
  paths:
    - /media/cihan/455bd60b-3793-4418-b70f-272901ff06b5/elk/logwas/SystemErr*.log
  fields: {log_type: was}
  multiline.type: pattern
  multiline.pattern: '^(\[\d{2}\.\d{2}\.\d{4} \d{2}:\d{2}:\d{2}:\d{3} \w{3}\]) (\w{8} SystemErr) \w [[:space:]]at'
  multiline.match: after

```

and it worked but as I said we need to combine section 3 and 4 as well if possible.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [December 2, 2020, 6:18am UTC](https://discuss.elastic.co/t/how-to-parse-websphere-systemerr-log-in-logstash/252323/16 "2020-12-02T06:18:42Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.