Multiline Capture Group

Elastic Stack Logstash
1

(^Mar[\s\S]?Caused) ... or more specifically (^%{MONTH}[\s\S]?Caused) doesn't work to capture a multiline log. Is there a way to capture a multiline log from a start and ending delimiter. Or is there a multiline filter flag that allows for stetting a beginning and end point to a capture group?
Thanks in advance,
Cole

2

It would be easier to help if you give an example of the kind of log you want to parse.

3

Here is my Multiline Filter:

  multiline {
    #Should Work  <-------- This is with Grok Filters
    pattern => "(%{MONTH}\s%{MONTHDAY},\s%{YEAR}\s%{TIME}\s(?:AM|PM)\sorg\.apache\.solr\.common\.SolrException\s[\s\S]*?%{MONTH})"

    #written out in pure Regex
    #pattern => "((\b\d{1,2}\D{0,3})?\b(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|(Nov|Dec)(?:ember)?)\s[0-9],\s20[0-9][0-9]\s[0-9][0-9]:[0-9][0-9]\:[0-9][0-9]\s(?:AM|PM)\sorg\.apache\.solr\.common\.SolrException[\s\S]*?(\b\d{1,2}\D{0,3})?\b(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tember)?|Oct(?:ober)?|(Nov|Dec)(?:ember)?))"

    negate => true
    what => "previous"
    add_tag => []
  }

I am trying to trap SolrExceptions and Multiline in general... here is a sample log that it should be able to capture according to multiline regex rules:

Mar 8, 2016 10:49:30 AM org.apache.solr.common.SolrException log
SEVERE: org.apache.solr.common.SolrException: org.apache.solr.search.SyntaxError: Cannot parse '-(+Topic:)': Encountered " ")" ") "" at line 1, column 9.
Was expecting one of:
    <BAREOPER> ...
    "(" ...
    "*" ...
    <QUOTED> ...
    <TERM> ...
    <PREFIXTERM> ...
    <WILDTERM> ...
    <REGEXPTERM> ...
    "[" ...
    "{" ...
    <LPARAMS> ...
    <NUMBER> ...
    
  at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:147)
  at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:187)
  at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
  at org.apache.solr.core.SolrCore.execute(SolrCore.java:1817)
  at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:639)
  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345)
  at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141)
  at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1307)
  at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:453)
  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
  at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:560)
  at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
  at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1072)
  at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:382)
  at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
  at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1006)
  at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
  at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)
  at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)
  at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
  at org.eclipse.jetty.server.Server.handle(Server.java:365)
  at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:485)
  at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)
  at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:926)
  at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:988)
  at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:635)
  at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235)
  at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)
  at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)
  at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)
  at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)
  at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.solr.search.SyntaxError: Cannot parse '-(+Topic:)': Encountered " ")" ") "" at line 1, column 9.
Was expecting one of:
    <BAREOPER> ...
    "(" ...
    "*" ...
    <QUOTED> ...
    <TERM> ...
    <PREFIXTERM> ...
    <WILDTERM> ...
    <REGEXPTERM> ...
    "[" ...
    "{" ...
    <LPARAMS> ...
    <NUMBER> ...

Mar

Thanks in advance for your help,
Cole

4 
    pattern => "(%{MONTH}\s%{MONTHDAY},\s%{YEAR}\s%{TIME}\s(?:AM|PM)\sorg\.apache\.solr\.common\.SolrException\s[\s\S]*?%{MONTH})"

What's up with this business at the end: [\s\S]*?%{MONTH} That's probably where things go wrong. I also don't understand why you only want the multiline filter to match for org.apache.solr.common.SolrException messages. Why not just use the date as the beginning-of-message marker? This works fine:

%{MONTH}\s%{MONTHDAY},\s%{YEAR}\s%{TIME}\s(?:AM|PM)\s