Hello,
I've encountered a huge problem which I can't solve myself. I have a file with like 60000 java exceptions and I'm using logstash to send it to elasticsearch. Each exception starts with 30 Jan 2017 and looks like the example below.
30 Jan 2017 09:42:52,060 [[ACTIVE] ExecuteThread: '18' for queue: 'weblogic.kernel.Default (self-tuning)'] ERROR com.cosma.common.data.report.io.Station - Unknown Area: JVLCAG1
com.cosma.common.data.report.io.UnknownStationException: Unknown Area: JVLCAG1
at com.cosma.common.data.report.io.Station.valueOf(Station.java:98)
at com.cosma.common.data.report.io.FilterFactory.getAreaFilter(FilterFactory.java:307)
at com.cosma.common.data.report.io.FilterFactory.newInstance(FilterFactory.java:122)
at com.cosma.distribution.telex.chain.rule.RuleMatcherBuilder.buildFilter(RuleMatcherBuilder.java:368)
at com.cosma.distribution.telex.chain.rule.RuleMatcherBuilder.buildFilter(RuleMatcherBuilder.java:287)
at com.cosma.distribution.telex.chain.rule.RuleMatcherBuilder.build(RuleMatcherBuilder.java:159)
at com.cosma.distribution.telex.ScheduleChangeDAOImpl.buildRuleMatcher(ScheduleChangeDAOImpl.java:432)
at com.cosma.distribution.telex.AbstractScheduleChangeDAO.findRuleMatcher(AbstractScheduleChangeDAO.java:296)
at com.cosma.distribution.telex.chain.RuleScheduleChangeFilter.executeCommand(RuleScheduleChangeFilter.java:67)
at com.cosma.distribution.telex.chain.BaseCommand.execute(BaseCommand.java:77)
at org.apache.commons.chain.impl.ChainBase.execute(ChainBase.java:166)
at com.cosma.distribution.telex.chain.StatefulChainBase.execute(StatefulChainBase.java:27)
at com.cosma.distribution.telex.chain.MasterCommand.executeSlave(MasterCommand.java:30)
at com.cosma.distribution.telex.chain.PatternScheduleChangeLoader.executeCommand(PatternScheduleChangeLoader.java:98)
at com.cosma.distribution.telex.chain.BaseCommand.execute(BaseCommand.java:77)
at org.apache.commons.chain.impl.ChainBase.execute(ChainBase.java:166)
at com.cosma.distribution.telex.chain.StatefulChainBase.execute(StatefulChainBase.java:27)
at com.cosma.distribution.telex.ScheduleMessageBean.processPublicationWorkflow(ScheduleMessageBean.java:842)
at com.cosma.distribution.telex.ScheduleMessageBean.process(ScheduleMessageBean.java:259)
at com.cosma.servercommon.base.jms.AsyncComponent.onMessage(AsyncComponent.java:46)
at com.cosma.distribution.telex.ScheduleMessageBean.onMessage(ScheduleMessageBean.java:149)
at weblogic.ejb.container.internal.MDListener.execute(MDListener.java:451)
at weblogic.ejb.container.internal.MDListener.transactionalOnMessage(MDListener.java:375)
at weblogic.ejb.container.internal.MDListener.onMessage(MDListener.java:310)
I have a configuration that takes the input through tcp and a filter that matches for messages starting with 30 Jan (see below) and output to my local elasticsearch. And as I am on windows system I use the command: TYPE exceptions.log | nc localhost 12345 -w600 (just to make sure everyhing is sent).
input {
tcp {
type => "exception_log"
port => 12345
}
}
filter {
grok {
match => {
"message" => "^(30 Jan)"
}
}
}
output {
if "_grokparsefailure" not in [tags] {
elasticsearch {
hosts => "localhost:9200"
}
}
}
This way everything works fine I get exactly as many exceptions in my kibana as many I have in the file. That's good.
The problem starts when I try to add a multiline codec to the input so I can get the stacktrace to the exceptions as well. So I have the input part modified everything else stays the same.
input {
tcp {
type => "exception_log"
port => 12345
codec => multiline {
pattern => "^(Caused|java|com|\s)"
what => "previous"
}
}
}
This time I get a random nuber of elements in kibana and watching those result I found something strange to the same exception. The message is cut at a totally random position. It's not because the message would be too long because there are much longer elements as well.
message:
30 Jan 2017 09:42:52,060 [[ACTIVE] ExecuteThread: '18' for queue: 'weblogic.kernel.Default (self-tuning)'] ERROR com.cosma.common.data.report.io.Station - Unknown Area: JVLCAG1
com.cosma.common.data.report.io.UnknownStationException: Unknown Area: JVLCAG1
at com.cosma.common.data.report.io.Station.valueOf(Station.java:98)
at com.cosma.common.data.report.io.FilterFactory.getAreaFilter(FilterFactory.java:307)
at com.cosma.common.data.report.io.FilterFactory.newInstance(FilterFactory.java:122)
at com.cosma.distribution.telex.chain.rule.RuleMatcherBuilder.buildFilter(RuleMatcherBuilder.java:368)
at com.cosma.distribution.telex.chain.rule.RuleMatcherBuilder.buildFilter(RuleMatcherBuilder.java:287)
at com.cosma.distribution.telex.chain.rule.RuleMatcherBuilder.build(RuleMatcherBuilder.java:159)
at com.cosma.distribution.telex.ScheduleChangeDAOImpl.buildRuleMatcher(ScheduleChangeDAOImpl.java:432)
at com.cosma.distribution.telex.AbstractScheduleChangeDAO.findRuleMatcher(AbstractScheduleChangeDAO.java:296)
at com.cosma.distribution.telex.chain.RuleScheduleChangeFilter.executeCommand(RuleScheduleChangeFilter.java:67)
at com.cosma.distribution.telex.chain.BaseCommand.execute(BaseCommand.java:77)
at org.apache.commons.chain.impl.ChainBase.execute(ChainBase.java:166)
at com.cosma.distribution.telex.chain.StatefulChainBase.execute(StatefulChainBase.java:27)
at com.cosma.distribution.telex.chain.MasterCommand.executeSlave(MasterCommand.java:30)
at com.cosma.distribution.telex.chain.PatternScheduleChangeLoader.executeCommand(PatternScheduleChangeLoader.java:98)
at c
More to that I get less entries than exceptions.
Can anyone pls help with that?
Thanks in advance,
Best regards