Logstash configs and grok pattern for java logs

Montego · July 14, 2020, 12:42pm

Hi there!
I have logs, wich looks like:

[INFO ] 28-04-2019 08:49:41.675 PM [main]  client.Application - No active profile set, falling back to default profiles: default
[INFO ] 28-04-2019 08:49:41.735 PM [main]  o.s.b.w.s.c.AnnotationConfigServletWebServerApplicationContext - Refreshing org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext@5cbc508c: startup date 
[ERROR] 28-04-2019 10:43:15.641 PM [https-jsse-nio-8443-exec-7]  o.a.c.c.C.[.[localhost].[/].[dispatcherServlet] - Servlet.service() for servlet [dispatcherServlet] threw exception
javax.servlet.ServletException: Circular view path [error]: would dispatch back to the current handler URL [/error] again. Check your ViewResolver setup! (Hint: This may be the result of an unspecified view, due to default view name generation.)
    at org.springframework.web.servlet.view.InternalResourceView.prepareForRendering(InternalResourceView.java:209)
    at org.springframework.web.servlet.view.InternalResourceView.renderMergedOutputModel(InternalResourceView.java:147)
    at org.springframework.web.servlet.view.AbstractView.render(AbstractView.java:314)
    at org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1325)
    at org.springframework.web.servlet.DispatcherServlet.processDispatchResult(DispatcherServlet.java:1069)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1008)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:925)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:974)
    at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:866)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:635)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:851)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:742)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)
    at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
    at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:119)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:170)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:101)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:101)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
    at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:357)
    at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:270)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:728)
    at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:472)
    at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:395)
    at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:316)
    at org.apache.catalina.core.StandardHostValve.custom(StandardHostValve.java:395)
    at org.apache.catalina.core.StandardHostValve.status(StandardHostValve.java:254)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:177)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.lang.Thread.run(Thread.java:748)

I'm absolutely beginer in it and tried configure logstash.conf:

input {
 beats { port => 5044 }
}
filter {
   grok {
        patterns_dir => ["./patterns"]
        match => { "message" => "%{LOGLEVEL:loglevel} %{CUSTOM_DATE:@timestamp} (\[%{DATA:thread}\] ) %{JAVACLASS:javaclass} - %{GREEDYDATA:loggedString}" }
   }
}
output {
   stdout { codec => rubydebug }
}

where CUSTOME_DATE in ./patterns/custom_pattern:
CUSTOME_DATE %{MONTHDAY}[/-]%{MONTHNUM}[/-]%{YEAR} %{HOUR}:%{MINUTE}:%{SECOND}

Help me with a pattern please(and would be great correct logstash.conf if necessary)

Badger · July 14, 2020, 5:06pm

I would use dissect rather than grok...

dissect { mapping => { "message" => "[%{loglevel}] %{[@metadata][timestamp]} %{+[@metadata][timestamp]} %{+[@metadata][timestamp]} [%{thread}]  %{javaclass} - %{loggedMessage}" } }
mutate { strip => [ "loglevel" ] }
date { match => [ "%{+[@metadata][timestamp]", "dd-MM-YYYY hh:mm:ss.SSS a" ] }

Note I am assuming your hours go from 01 to 12, if they go from 00 to 11 change the hh to KK.

Montego · July 15, 2020, 8:57am

thanks a lot, but something going wrong with it:

        [ERROR][org.logstash.execution.WorkerLoop][main] Exception in pipelineworker, the pipeline stopped processing new events, please check your filter configuration and restart Logstash.
        org.logstash.FieldReference$IllegalSyntaxException: Invalid FieldReference: `%{+[@metadata][timestamp]`

Badger · July 15, 2020, 5:55pm

Sorry about that, remove the %{+

Montego · July 16, 2020, 11:49am

Awersome! Thank you very much! Realy thank you. It is working as i need, except one thing. But i guess it's a problem of filebeat and i'm not sure can i ask here about it or not...
( In any case thank's for helping with pattern)

The problem is - i have an error in logfile like this:

[ERROR] 07-07-2020 12:12:35.564 PM [http-nio-8555-exec-3]  o.a.c.c.C.[.[localhost].[/].[dispatcherServlet] - Servlet.service() for servlet [dispatcherServlet] threw exception
javax.servlet.ServletException: Circular view path [error]: would dispatch back to the current handler URL [/error] again. Check your ViewResolver setup! (Hint: This may be the result of an unspecified view, due to default view name generation.)
	at org.springframework.web.servlet.view.InternalResourceView.prepareForRendering(InternalResourceView.java:210)
	at org.springframework.web.servlet.view.InternalResourceView.renderMergedOutputModel(InternalResourceView.java:148)
	at org.springframework.web.servlet.view.AbstractView.render(AbstractView.java:316)
	at org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1373)
	at org.springframework.web.servlet.DispatcherServlet.processDispatchResult(DispatcherServlet.java:1118)

But logstash show me something like thisin message(all part after word 'exception' in full log:

[WARN ][org.logstash.dissect.Dissector][main][0f7bb7b58054369269f969d0cde4321c19667cd6d536c74e61d99a99d4960e92] Dissector mapping, pattern not found {"field"=>"message", "pattern"=>"[%{loglevel}] %{[@metadata][timestamp]} %{+[@metadata][timestamp]} %{+[@metadata][timestamp]} [%{thread}]  %{javaclass} - %{loggedMessage}", "event"=>{"message"=>"javax.servlet.ServletException: Circular view path [error]: would dispatch back to the current handler URL [/error] again. Check your ViewResolver setup! (Hint: This may be the result of an unspecified view, due to default view name generation.)\n\tat org.springframework.web.servlet.view.InternalResourceView.prepareForRendering(InternalResourceView.java:209)\n\tat org.springframework.web.servlet.view.InternalResourceView.renderMergedOutputModel(InternalResourceView.java:147)\n\tat org.springframework.web.servlet.view.AbstractView.render(AbstractView.java:314)\n\tat org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1325)\n\tat org.springframework.web.servlet.DispatcherServlet.processDispatchResult(DispatcherServlet.java:1069)\n\tat org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1008)

I used in filebeat:

multiline.pattern: '^[[:space:]]'
multiline.negate: false
multiline.match: after

system · August 13, 2020, 11:49am

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.