Extract

[proximator (Amjad)]

Hi

I would like to build a new field from my error messages using some grok patterns, the idea is to extract the class name from the first line and then from each single line that starts with "Caused by:"

In the example below, the value of my new field should be: java.lang.RuntimeException/java.lang.IllegalStateException/java.lang.UnsupportedOperationException

Exception in thread "main" java.lang.RuntimeException: error message
    at Exceptions.main(Exceptions.java:4)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
Caused by: java.lang.IllegalStateException: Some message
    at Exceptions.main(Exceptions.java:3)
    ... 5 more
Caused by: java.lang.UnsupportedOperationException: Some other message 
    at Exceptions.main(Exceptions.java:10)
    ... 5 more

filebeat is sending the full stack trace using multiline already.
Any suggestion how can I achieve this with with minimum code?

Cheers

[Badger]

So if that whole stack comes in as a single event then exactly which pieces of text do you want to extract?

[proximator (Amjad)]

I want to extract the java class name of the first line and all the java class name of the lines that starts with "Caused by: " and concatenate all of them

So in the example above, I should get: java.lang.RuntimeException/java.lang.IllegalStateException/java.lang.UnsupportedOperationException

[Jenni]

There is probably a prettier and better solution, but, if the full error message is in one field, this would create your desired string:

grok {
    match => { "testmessage" => ".*\"\s(?<error>[^\:]+).+?(?=Caused by: )Caused by: (?<cause>.*)" }
  }
  mutate {
    gsub => [
      "cause", "[\n]", "",
      "cause", ":.*Caused\sby:\s", "/",
      "cause", ":.*$", ""
    ]
    add_field => { "full_error" => "%{error}/%{cause}"}
  }

"full_error" => "java.lang.RuntimeException/java.lang.IllegalStateException/java.lang.UnsupportedOperationException"

[Badger]

Just dealing with the multiple 'Caused by' lines... When you use a capture group in String.scan the array you get back contains arrays, not strings, so we need to flatten before doing a join.

  ruby {
    code => "
      s = event.get('message')
      r = s.scan(/Caused by: ([^:]+):/)
      r = r.flatten
      event.set('causedby', r.join('/'))
    "
  }