Hi.
I've created a filter that takes in JBoss fuse logs, but it has problems when I get a stack trace or other information that takes more than one line.
My filter:
input {
beats {
port => "5044"
}
}
filter {
dissect {
mapping => {
"message" => "%{timestamp} | %{level} | %{subsystem} | %{route} | %{class} | %{message}"
}
}
date {
match => [ "timestamp", "ISO8601" ]
}
}
output {
elasticsearch {
hosts => ["localhost:9200"]
index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
document_type => "%{[metadata][type]}"
}
}
Example log:
2018-03-20 09:05:20,842 | INFO | #9 - timer://runOnce | ce-reporting-opportunity-route ID-selite-46351-1521533020576-4-19 | com.domain.app.service.run.runfoldizServiceImpl | Insert!!!!!!!!!!!!!
2018-03-20 09:05:20,868 | ERROR | #9 - timer://runOnce | ce-reporting-opportunity-route ID-selite-46351-1521533020576-4-19 | com.domain.app.service.run.runfoldizServiceImpl | Failed to update Opportunities in run. Exception Message: null
2018-03-20 09:05:20,888 | INFO | #9 - timer://runOnce | deadLetterChannel ID-selite-46351-1521533020576-4-19 | com.domain.app.common.errorhandler.route.ErrorRoute | Saving error log for message log id: 375035
2018-03-20 09:05:20,902 | ERROR | #9 - timer://runOnce | ce-reporting-opportunity-route ID-selite-46351-1521533020576-4-19 | com.domain.app.common.errorhandler.routebuilder.ErrorHandlingRouteBuilder | Failed delivery for (MessageId: ID-selite-46351-1521533020576-4-21 on ExchangeId: ID-selite-46351-1521533020576-4-22). Exhausted after delivery attempt: 1 caught: java.lang.NullPointerException. Processed by failure processor: sendTo(Endpoint[direct-vm://deadLetterChannel] InOnly)
Message History
---------------------------------------------------------------------------------------------------------------------------------------
RouteId ProcessorId Processor Elapsed (ms)
[run-foldiz-rep] [run-foldiz-rep] [timer://runOnce?delay=1000&repeatCount=1 ] [ 2792]
[run-foldiz-rep] [process109 ] [Processor@0x7083ef84 ] [ 0]
[run-foldiz-rep] [to30 ] [direct-vm:service-foldiz-opportunity-report ] [ 1321]
[service-foldiz] [log30 ] [log ] [ 1]
[service-foldiz] [process56 ] [Processor@0x1aa1f97a ] [ 1319]
[run-foldiz-rep] [bean28 ] [bean[com.domain.app.integrations.runtofoldiz.mapper.foldizOpportun] [ 8]
[run-foldiz-rep] [bean29 ] [bean[com.domain.app.service.run.runfoldizServiceImpl@62fb90e3] ] [ 403]
[deadLetterChannel ] [setProperty1 ] [setProperty[app-storedInErrorQueue] ] [ 0]
[deadLetterChannel ] [process1 ] [Processor@0xd6bbcf7 ] [ 24]
Exchange
---------------------------------------------------------------------------------------------------------------------------------------
Exchange[
Id ID-se576-4-22
ExchangePattern InOnly
Headers {breadcrumbId=ID-s76-4-19, app-flow=run_foldiz, app-messageKey=Triggered task, app-messageRoute=run_foldiz_REPORTING_OPPORTUNITY, app-messageType=run_TRIGGER, app-originatingMessageKey=Triggered task, app-originatingMessageType=run_TRIGGER, app-resendRouteName=run-foldiz-reporting-opportunity-route-resend, app-routeName=run-foldiz-reporting-opportunity-route, firedTime=Tue Mar 20 09:05:18 CET 2018}
BodyType java.util.ArrayList
Body [com.domain.app.service.run.domain.foldizOpportunity@58d573b3, com.domain.app.service.run.domain.foldizOpportunity@4d22c0c2, com.domain.app.service.run.domain.foldizOpportunity@f528606, com.domain.app.service.run.domain.foldizOpportunity@6b5a1302, com.domain.app.service.run.domain.foldizOpportunity@1861ad31]
]
Stacktrace
---------------------------------------------------------------------------------------------------------------------------------------
java.lang.NullPointerException
at com.domain.app.service.run.dao.foldizDaoImpl$3.setValues(foldizDaoImpl.java:868)
at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:822)
at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:818)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:589)
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:818)
at java.util.TimerThread.mainLoop(Timer.java:555)[:1.8.0_151]
at java.util.TimerThread.run(Timer.java:505)[:1.8.0_151]
2018-03-20 09:05:20,912 | INFO | #9 - timer://runOnce | orti-route-start ID-selite-476-4-19 | com.domain.app.common.errorhandler.routebuilder.MessageLoggingRoutePolicy | Saving messageLog with description: Triggered task, route: run_foldiz_REPORTING_OPPORTUNITY and status: ERROR
2018-03-20 09:10:35,355 | INFO | FelixShutdown | | org.apache.aries.blueprint.container.BlueprintExtender | Destroying BlueprintContainer for bundle com.domain.app.integrations.service-app/20.0.0.SNAPSHOT
2018-03-20 09:10:35,422 | INFO | FelixShutdown | | org.apache.aries.blueprint.container.BlueprintExtender | Destroying BlueprintContainer for bundle io.hawt.hawtio-json-schema-mbean/1.4.0.redhat-621159
2018-03-20 09:10:35,431 | INFO | FelixShutdown | | org.apache.aries.blueprint.container.BlueprintExtender | Destroying BlueprintContainer for bundle org.apache.camel.karaf.camel-karaf-commands/2.15.1.redhat-621159
2018-03-20 09:10:35,569 | INFO | FelixShutdown | | org.apache.aries.blueprint.container.BlueprintExtender | Destroying BlueprintContainer for bundle activemq-karaf/5.11.0.redhat-621159
2018-03-20 09:10:35,610 | INFO | FelixShutdown | | org.apache.aries.blueprint.container.BlueprintExtender | Destroying BlueprintContainer for bundle org.apache.activemq.activemq-osgi/5.11.0.redhat-621159