Extract

[Jenni]

There is probably a prettier and better solution, but, if the full error message is in one field, this would create your desired string:

grok {
    match => { "testmessage" => ".*\"\s(?<error>[^\:]+).+?(?=Caused by: )Caused by: (?<cause>.*)" }
  }
  mutate {
    gsub => [
      "cause", "[\n]", "",
      "cause", ":.*Caused\sby:\s", "/",
      "cause", ":.*$", ""
    ]
    add_field => { "full_error" => "%{error}/%{cause}"}
  }

"full_error" => "java.lang.RuntimeException/java.lang.IllegalStateException/java.lang.UnsupportedOperationException"