Hi i try to execute a grok but return
text Timeout executing grok '\[(?<STUCK>STUCK)\] %{DATA:STUCKMESSAGE}(\n)\[\n(%{DATA:Method} %{DATA:url} %{DATA:Protocol}\n)(?<headers-list>(.|\n)*)\]\", (\n|\r|.)*' against field 'LOGMESSAGE' with value 'Value too large to output (10813 bytes)! First 255 chars are:
i use this grok
grok {
timeout_millis => 300000
match => {"LOGMESSAGE" => ["\[(?<STUCK>STUCK)\] %{DATA:STUCKMESSAGE}(\n)\[\n(%{DATA:Method} %{DATA:url} %{DATA:Protocol}\n)(?<headers-list>(.|\n)*)\]\", (\n|\r|.)*"]}
}
i use timeout_millis but not work
Badger
November 4, 2019, 6:03pm
2
You need to use something more specific that DATA. If I have correctly guessed what that is parsing then NOTSPACE could work. Without seeing multiple examples of what you are parsing I cannot suggest improvements for the rest.
I change my grok to be more specific, like this
\[(%{WORD:STUCK})\] %{DATA:STUCKMESSAGE}(\n)\[\n(%{WORD:method} %{URIPATHPARAM:request} (HTTP/%{NUMBER:httpversion})?(?<headers-list>(.|\n|\r)*)\n)\]\",
but i had the same output
Timeout executing grok '\[(%{WORD:STUCK})\] %{DATA:STUCKMESSAGE}(\n)\[\n(%{WORD:method} %{URIPATHPARAM:request} (HTTP/%{NUMBER:httpversion})?(?<headers-list>(.|\n|\r)*)\n)\]\",' against field 'LOGMESSAGE' with value 'Value too large to output (10808 bytes)! First 255 chars are:
a sample is this
[STUCK] ExecuteThread: '51' for queue: 'xxxxxxxxxx.kernel.Default (self-tuning)' has been busy for "1,298" seconds working on the request "Workmanager: default, Version: 0, Scheduled=true, Started=true, Started time: 1298391 ms
[
POST /xxx/xx/com/xxxxxx/xxxxxx/geoLocationV3.jsf HTTP/1.0
Connection: close
Content-Length: 5173
Accept: application/xml, text/xml, */*; q=0.01
Origin: http://xxxxxx.xxxxxxx.xxxx
X-Requested-With: XMLHttpRequest
Faces-Request: partial/ajax
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.87 Safari/537.36
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Referer: http://xxx.xxxxxxx.xx/xx/xx/xx/xxxxxxxxxxxxxx/geolocation/geoLocationV3.jsf?6.5.8-bdrp
Accept-Encoding: gzip, deflate
Accept-Language: es-ES,es;q=0.9
Cookie: visited=true; _ga=GA1..1564197457; _gid=GA1.2.1403500929.1572268302; JSESSIONID=-WerteM80nfHj!-1441246413; _gat=1
]", which is more than the configured time (StuckThreadMaxTime) of "1,200" seconds. Stack trace:
~120 lines
xxxxxxxxxx.work.ExecuteThread.run(ExecuteThread.java:221)
>
when no use this field all work
(?<headers-list>(.|\n|\r)*)\n)\]\",
Badger
November 4, 2019, 7:50pm
4
Not sure why that would be
grok { match => {"message" => "\[(%{WORD:STUCK})\] %{DATA:STUCKMESSAGE}(\n)\[\n(%{WORD:method} %{URIPATHPARAM:request} (HTTP/%{NUMBER:httpversion})?(?<headers-list>(.|\n|\r)*)\n)\]" } }
matches that data in a fraction of a second for me.
in the grok debugger it doesn't take long but in the pipeline timeout returns
all my .conf file is
input {
file {
path => "/home/l/Cc.out00128"
start_position => "beginning"
codec => multiline {
pattern => "^\<"
negate => true
what => "previous"
max_lines => 120
}
}
}
filter {
grok {
timeout_millis => 300000
match => {"message" => ["^\<%{DATA:DATE}\> \<%{WORD:LEVEL}\> \<%{WORD:SERVICE}\> (\<BEA-%{WORD:BEA}\> )?\<(?<LOGMESSAGE>((?!(\> \n))(.|\r|\n))*)"]}
}
grok {
timeout_millis => 300000
match => {"LOGMESSAGE" => ["\[(%{WORD:STUCK})\] %{DATA:STUCKMESSAGE}(\n)\[\n(%{WORD:method} %{URIPATHPARAM:request} (HTTP/%{NUMBER:httpversion})?(?<headers-list>(.|\n|\r)*)\n)\]\","]}
}
}
output {
elasticsearch {
hosts => "localhost:9200"
manage_template => false
index => "test-%{+yyyy.MM.dd}"
}
}
and all log is
<Nov 1, 2019 4:54:22 PM MST> <Error> <xxxxxxxServer> <BEA-000337> <[STUCK] ExecuteThread: '51' for queue: 'xxxxxxx.kernel.Default (self-tuning)' has been busy for "1,298" seconds working on the request "Workmanager: default, Version: 0, Scheduled=true, Started=true, Started time: 1298391 ms
[
POST /xxxx/xxxx/com/xxxxxxxxxx/geolocation/geoLocationV3.jsf HTTP/1.0
Connection: close
Content-Length: 5173
Accept: application/xml, text/xml, */*; q=0.01
Origin: http://xxxx.xxxxxxxxxx.xxxx
X-Requested-With: XMLHttpRequest
Faces-Request: partial/ajax
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.87 Safari/537.36
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Referer: http://xxxx.xxxxxxxxxx.xxxx/xxxx/xxxx/com/xxxxxxxxxx/geolocation/geoLocationV3.jsf?6.5.8-bdrp
Accept-Encoding: gzip, deflate
Accept-Language: es-ES,es;q=0.9
Cookie: visited=true; _ga=GA1.2.898679231.1564197457; _gid=GA1.2.1403500929.1572268302; JSESSIONID=LDEpJzqpWIZBvoku7PAtBAlM6B8yVKGPw4P1Ejc-WerteM80nfHj!-1441246413; _gat=1
]", which is more than the configured time (StuckThreadMaxTime) of "1,200" seconds. Stack trace:
java.math.MutableBigInteger.divideMagnitude(MutableBigInteger.java:993)
java.math.MutableBigInteger.divide(MutableBigInteger.java:915)
java.math.BigDecimal.divideAndRound(BigDecimal.java:1442)
java.math.BigDecimal.setScale(BigDecimal.java:2406)
com.xxxxxxxxxx.core.web.controller.GeolocationController.getFechaActualByLatDistance(GeolocationController.java:6525)
com.xxxxxxxxxx.core.web.controller.GeolocationController.verificaDistancias(GeolocationController.java:6484)
com.xxxxxxxxxx.core.web.controller.GeolocationController.getStucturedListPositionGM(GeolocationController.java:6397)
com.xxxxxxxxxx.core.web.controller.GeolocationController.getListRecorridoMovilPostgres(GeolocationController.java:6046)
sun.reflect.GeneratedMethodAccessor35302.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:606)
org.jboss.el.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:328)
org.jboss.el.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:273)
org.jboss.el.parser.AstMethodSuffix.getValue(AstMethodSuffix.java:59)
org.jboss.el.parser.AstMethodSuffix.invoke(AstMethodSuffix.java:65)
org.jboss.el.parser.AstValue.invoke(AstValue.java:96)
org.jboss.el.MethodExpressionImpl.invoke(MethodExpressionImpl.java:276)
com.sun.faces.facelets.el.TagMethodExpression.invoke(TagMethodExpression.java:105)
javax.faces.event.MethodExpressionActionListener.processAction(MethodExpressionActionListener.java:148)
javax.faces.event.ActionEvent.processListener(ActionEvent.java:88)
javax.faces.component.UIComponentBase.broadcast(UIComponentBase.java:778)
javax.faces.component.UICommand.broadcast(UICommand.java:300)
javax.faces.component.UIViewRoot.broadcastEvents(UIViewRoot.java:787)
javax.faces.component.UIViewRoot.processApplication(UIViewRoot.java:1252)
com.sun.faces.lifecycle.InvokeApplicationPhase.execute(InvokeApplicationPhase.java:81)
com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
javax.faces.webapp.FacesServlet.service(FacesServlet.java:508)
xxxxxxx.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
xxxxxxx.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
xxxxxxx.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:301)
xxxxxxx.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
xxxxxxx.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:60)
org.primefaces.webapp.filter.FileUploadFilter.doFilter(FileUploadFilter.java:100)
xxxxxxx.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:60)
xxxxxxxx.servlet.gzip.filter.GZIPFilter.doFilter(GZIPFilter.java:70)
xxxxxxx.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:60)
xxxxxxx.servlet.internal.RequestEventsFilter.doFilter(RequestEventsFilter.java:27)
xxxxxxx.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:60)
xxxxxxx.servlet.internal.WebAppServletContext$ServletInvocationAction.wrapRun(WebAppServletContext.java:3748)
xxxxxxx.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3714)
xxxxxxx.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
xxxxxxx.security.service.SecurityManager.runAs(SecurityManager.java:120)
xxxxxxx.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2283)
xxxxxxx.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:2182)
xxxxxxx.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1491)
xxxxxxx.work.ExecuteThread.execute(ExecuteThread.java:256)
xxxxxxx.work.ExecuteThread.run(ExecuteThread.java:221)
>
maybe the error is found when using both filters
system
December 3, 2019, 12:56am
6
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.