I finally made good grok pattern with my log and in grok debugger all was fine.
When I implemented it in my stack, I do not have proper fields, my "exception" field is empty or one char.
Log sample:
21:35:13.242 pl.com.agora.squid.core.countingpaywall.ExceptionSuppressingSettingsExtractor {http-nio-8528-exec-183} ERROR : Error while checking paywall type for assetId: 2310333 , sectionId: 75475.
java.lang.RuntimeException: java.util.concurrent.ExecutionException: pl.com.agora.api.client.rest.exception.UriInvocationException: Received error response URI : http://article/v2/id/2310333/section/75475/metadata Received error status code : 404. Received error result object : ArticleApiErrorDto{result=-2, msg='PublicationForGivenSectionNotFoundException(articleId=2310333, sectionId=75475) when invoking: /article/v2/id/2310333/section/75475/metadata; root cause: PublicationForGivenSectionNotFoundException: '}
at pl.com.agora.squid.core.countingpaywall.StandardSettingsExtractor.getPaywallTypeForAssetId(StandardSettingsExtractor.java:208)
at pl.com.agora.squid.core.countingpaywall.ExceptionSuppressingSettingsExtractor.getPaywallTypeForAssetId(ExceptionSuppressingSettingsExtractor.java:156)
at pl.com.agora.squid.paywall.web.PaywallFilter.getPaywallType(PaywallFilter.java:530)
at pl.com.agora.squid.paywall.web.PaywallFilter.doFilterInternal(PaywallFilter.java:132)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at pl.com.agora.mobile.filter.MobileDeviceInfoFilter.doFilter(MobileDeviceInfoFilter.java:58)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at pl.agora.aws.sso.filters.SsoLogonFilter.doFilter(SsoLogonFilter.java:44)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.filters.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:108)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:504)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:677)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1459)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.ExecutionException: pl.com.agora.api.client.rest.exception.UriInvocationException: Received error response URI : http://article-api.gazeta.pl:80/article/v2/id/2310333/section/75475/metadata Received error status code : 404. Received error result object : ArticleApiErrorDto{result=-2, msg='PublicationForGivenSectionNotFoundException(articleId=2310333, sectionId=75475) when invoking: /article/v2/id/2310333/section/75475/metadata; root cause: PublicationForGivenSectionNotFoundException: '}
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
at com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:137)
at pl.com.agora.squid.core.countingpaywall.StandardSettingsExtractor.getPaywallTypeForAssetId(StandardSettingsExtractor.java:195)
... 38 common frames omitted
Caused by: pl.com.agora.api.client.rest.exception.UriInvocationException: Received error response URI : http://article/v2/id/2310333/section/75475/metadata Received error status code : 404. Received error result object : ArticleApiErrorDto{result=-2, msg='PublicationForGivenSectionNotFoundException(articleId=2310333, sectionId=75475) when invoking: /article/v2/id/2310333/section/75475/metadata; root cause: PublicationForGivenSectionNotFoundException: '}
at sun.reflect.GeneratedConstructorAccessor333.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at pl.com.agora.api.client.rest.defaults.ClassInvocationExceptionFactory.onResponse(ClassInvocationExceptionFactory.java:53)
at pl.com.agora.api.client.rest.RestfulClientProxyFactory$1.onResponse(RestfulClientProxyFactory.java:376)
at pl.com.agora.api.client.rest.invocation.UriInvocationSupport.onResponseContent(UriInvocationSupport.java:70)
at pl.com.agora.api.client.rest.jetty.Jetty9UriInvocation$1.onComplete(Jetty9UriInvocation.java:55)
at org.eclipse.jetty.client.HttpRequest$15.onComplete(HttpRequest.java:566)
at org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:193)
at org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:185)
at org.eclipse.jetty.client.HttpReceiver.terminateResponse(HttpReceiver.java:458)
at org.eclipse.jetty.client.HttpReceiver.responseSuccess(HttpReceiver.java:405)
at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.messageComplete(HttpReceiverOverHTTP.java:277)
at org.eclipse.jetty.http.HttpParser.parseContent(HttpParser.java:1617)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:1350)
at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.parse(HttpReceiverOverHTTP.java:159)
at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.process(HttpReceiverOverHTTP.java:120)
at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.receive(HttpReceiverOverHTTP.java:70)
at org.eclipse.jetty.client.http.HttpChannelOverHTTP.receive(HttpChannelOverHTTP.java:90)
at org.eclipse.jetty.client.http.HttpConnectionOverHTTP.onFillable(HttpConnectionOverHTTP.java:115)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
... 1 common frames omitted
Logstash pattern:
filter {
grok {
match => { "message" => "%{TIME:timestamp}\s
(\[)?(%{GREEDYDATA:logger})?
(\])?\s%{GREEDYDATA:thread}\s
%{LOGLEVEL:level}\s:\s
(?<problem>[^:].*)(\n)?
(?<exception>[^:].*)(\n)?
(?<message>(.|\r|\n)*)" }
}
}
Fielbeat yml:
- type: log
enabled: true
paths:
- /logs/localhost-*.log
exclude_files: ['.gz$']
fields:
app_name: tomcat-xx
fields_under_root: true
multiline.pattern: '^[0-2][0-9]:[0-6][0-9]:[0-6][0-9].[0-9][0-9][0-9]'
multiline.negate: true
multiline.match: after
close_timeout: 10m
What went wrong?