# \[Logstash 6.5.2\] Problem parsing long messages

**URL:** <https://discuss.elastic.co/t/logstash-6-5-2-problem-parsing-long-messages/205872>\
**Category:** Logstash\
**Created:** [October 30, 2019, 1:50pm UTC](https://discuss.elastic.co/t/logstash-6-5-2-problem-parsing-long-messages/205872 "2019-10-30T13:50:22Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Fabio-sama](https://avatars.discourse-cdn.com/v4/letter/f/b9e5f3/32.png) [@Fabio-sama](https://discuss.elastic.co/u/Fabio-sama)\
**Post date:** [October 30, 2019, 1:50pm UTC](https://discuss.elastic.co/t/logstash-6-5-2-problem-parsing-long-messages/205872/1 "2019-10-30T13:50:23Z")

</div>

Hi there,

I'm using Logstash to parse a java json log which may have a stacktrace nested field. I wanted to use `json` filter to automatically parse the input (since it is a json with possible nested jsons).  
Now, if I apply a simple pipeline like

```
input {
    stdin{}
}

filter {
    json {
        source => "message"
    }
}

output {
    stdout {
        codec => "rubydebug"
    }
}

```

it beautifully parses simple inputs like  
`{ "field1":"test1", "field2":"test2", "nested_field": { "nested1":["1",2,{ "testobj": "newtest" }], "nested2":"tnest2" } }`, recognizing nested fields and arrays:

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/a/0/a0a48dade062e46cadef776d029f0779d090789b.png)

Though, when I insert a more complicated json (like the nested stacktrace present in the real log)

```
{ "exception":{"stacktrace": "java.lang.IllegalStateException: Impossibile Stabilire nodo primario: ORA-04063: package body LCP1.AGENT_REGISTRYNODES has errors\nORA-06508: PL/SQL: could not find program unit being called: LCP1.AGENT_REGISTRYNODES\nORA-06512: at line 1\n\n\tat it.ltm.mlt.common.scheduler.strategy.MasterNodeStrategy.canExecute(MasterNodeStrategy.java:46)\n\tat it.ltm.mlt.common.scheduler.AbstractScheduler.isPrimary(AbstractScheduler.java:70)\n\tat it.ltm.mlt.common.scheduler.AbstractScheduler.ejbTimeout(AbstractScheduler.java:57)\n\tat sun.reflect.GeneratedMethodAccessor120.invoke(Unknown Source)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)\n\tat java.lang.reflect.Method.invoke(Method.java:597)\n\tat com.bea.core.repackaged.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:310)\n\tat com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)\n\tat com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)\n\tat com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)\n\tat com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)\n\tat com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)\n\tat com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)\n\tat com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)\n\tat com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)\n\tat com.bea.core.repackaged.springframework.jee.spi.MethodInvocationVisitorImpl.visit(MethodInvocationVisitorImpl.java:37)\n\tat weblogic.ejb.container.injection.EnvironmentInterceptorCallbackImpl.callback(EnvironmentInterceptorCallbackImpl.java:54)\n\tat com.bea.core.repackaged.springframework.jee.spi.EnvironmentInterceptor.invoke(EnvironmentInterceptor.java:50)\n\tat com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)\n\tat com.bea.core.repackaged.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)\n\tat com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)\n\tat com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)\n\tat com.bea.core.repackaged.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)\n\tat com.bea.core.repackaged.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)\n\tat com.bea.core.repackaged.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)\n\tat $Proxy79.ejbTimeout(Unknown Source)\n\tat weblogic.ejb.container.timer.TimerImpl.timerExpired(TimerImpl.java:301)\n\tat weblogic.timers.internal.TimerImpl.run(TimerImpl.java:273)\n\tat weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:528)\n\tat weblogic.work.ExecuteThread.execute(ExecuteThread.java:207)\n\tat weblogic.work.ExecuteThread.run(ExecuteThread.java:176)\n\tCaused by: java.sql.SQLException: ORA-04063: package body LCP1.AGENT_REGISTRYNODES has errors\nORA-06508: PL/SQL: could not find program unit being called: LCP1.AGENT_REGISTRYNODES\nORA-06512: at line 1\n\n\tat oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:440)\n\tat oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)\n\tat oracle.jdbc.driver.T4C8Oall_added_to_make_it_longer"} }

```

It immediately returns a `_jsonparsefailure`.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/5/35317cc66f6b36d8b33401eb5adeb57129e8f0cf.png)

This happens because it basically truncates the message (if you see the original json the last part is `_added_to_make_it_longer` but in the image you can only see `_add`) and not finding the closing curly brackets it treats it like an invalid json.

In fact, if I remove the `_added_to_make_it_longer` final part, it works flawlessly

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/7/c798a5ffe5b616f3165977f7e63f7e6f013c48aa.png)

Same thing happens with the grok filter, it returns `_grokparsefailure` if I try to grok a line which exceeds 4095 characters. I thought this limit was increased in LS versions \>5.1. I'm using the 6.5.2 and I still see this error.

Why? And more importantly, which is the workaround?

Thanks

---

<div class="post-metadata">

**Author:** ![Fabio-sama](https://avatars.discourse-cdn.com/v4/letter/f/b9e5f3/32.png) [@Fabio-sama](https://discuss.elastic.co/u/Fabio-sama)\
**Post date:** [October 30, 2019, 1:57pm UTC](https://discuss.elastic.co/t/logstash-6-5-2-problem-parsing-long-messages/205872/2 "2019-10-30T13:57:25Z")

</div>

I have seen this one. [Input STDIN 4096 character input limit?](https://discuss.elastic.co/t/input-stdin-4096-character-input-limit/50630)

Isn't there any way to increase that limit? Also, is it possible it happens with other types of input too (like `redis`?)

Thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 27, 2019, 1:57pm UTC](https://discuss.elastic.co/t/logstash-6-5-2-problem-parsing-long-messages/205872/3 "2019-11-27T13:57:27Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.