I have the following setting: Filebeat => Logstash
In order to support java stack trace I added multiline configuration in filebeat.yml
multiline.pattern: '^\[[0-9]{2}-[0-9]{2}-[0-9]{2}'
multiline.negate: true
multiline.match: after
However filebeat handles all log lines as one event and in kibana they displayed as one log line
Log example:
[13-10-20 10:04:42.531+0200] INFO 82 org.quartz.plugins.history.LoggingTriggerHistoryPlugin - Trigger Elastic Search Indexing.-443779357@IDITSystem-2010131004144231799457059968000 fired job Elastic Search Indexing.Elastic Search Incremental Indexing Initiator Job number:-1037783044 at: 10:04:42 10/13/2020
[13-10-20 10:04:42.531+0200] INFO 82 org.quartz.plugins.history.LoggingTriggerHistoryPlugin - Trigger Elastic Search Indexing.-443779357@IDITSystem-2010131004144231799457059968000 fired job Elastic Search Indexing.Elastic Search Incremental Indexing Initiator Job number:-1037783044 at: 10:04:42 10/13/2020
[13-10-20 10:04:42.533+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.session.BasicEnvironment - fetchUserContext: Creating a 'NOT LOGGED IN' user context.
[13-10-20 10:04:42.533+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.utils.TaskLogger - [SYS_TASK INFO : Elastic Search Incremental Indexing Initiator Job]: Job priority : 1
[13-10-20 10:04:42.533+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.session.BasicEnvironment - fetchUserContext: Creating a 'NOT LOGGED IN' user context.
[13-10-20 10:04:42.533+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.utils.TaskLogger - [SYS_TASK INFO : Elastic Search Incremental Indexing Initiator Job]: Job name : Elastic Search Incremental Indexing Initiator Job number:-1037783044
[13-10-20 10:04:42.533+0200] INFO 82 -603403668@IDITSystem com.idit.reference.server.manager.IDITGeneralLoggerManager - updateBatchLogStatus(batchLog=BatchLogVO { status=CURRENT, versionNr=2, id=171066, groupName="Elastic Search Indexing", fileName="null", }, newStatus=8, startTime=Tue Oct 13 10:04:42 CEST 2020, groupName=Elastic Search Indexing, totalRecords=null, jobType=3) with old status = 8, old records = null
[13-10-20 10:04:42.534+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.session.BasicEnvironment - fetchUserContext: Creating a 'NOT LOGGED IN' user context.
[13-10-20 10:04:42.537+0200] INFO 82 -603403668@IDITSystem com.idit.framework.common.context.EnvironmentAccessManager - setEnvironment( NULL ) will remove SuperEnvironment@2cc972c2 {user=NOT_LOGGED_IN, subSessionId=null, userContext=null }
[13-10-20 10:04:42.544+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.utils.TaskLogger - [SYS_TASK INFO : Elastic Search Incremental Indexing Initiator Job]: Job log status was updated to Unscheduled
[13-10-20 10:04:42.544+0200] INFO 82 -603403668@IDITSystem com.idit.reference.server.manager.IDITGeneralLoggerManager - updateBatchLogStatus(batchLog=BatchLogVO { status=CHANGED, versionNr=3, id=171066, groupName="Elastic Search Indexing", fileName="null", }, newStatus=10) with old status = 8
[13-10-20 10:04:42.547+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.utils.TaskLogger - [SYS_TASK INFO : Elastic Search Incremental Indexing Initiator Job]: The job Elastic Search Incremental Indexing Initiator Job is already in process
[13-10-20 10:04:42.550+0200] INFO 82 -603403668@IDITSystem com.idit.framework.common.jaas.common.impl.IDITSecurityManager - Disconnect username: -603403668@IDITSystem
[13-10-20 10:04:42.550+0200] ERROR 82 -603403668@IDITSystem com.idit.framework.backend.interceptor.helper.TxContextUtil - Calling removeEntitiesUnderTx without TX entry in Context! container.hashCode322808389 TxContextStack{txStack=[]}method name : logout transactionPhase.name() BEFORE_COMPLETION
[13-10-20 10:04:42.550+0200] ERROR 82 -603403668@IDITSystem com.idit.framework.backend.interceptor.helper.TxContextUtil - Calling removeEntitiesUnderTx without TX entry in Context! container.hashCode 322808389 TxContextStack{txStack=[]} method name : logout transactionPhase.name() AFTER_FAILURE
[13-10-20 10:04:42.550+0200] INFO 82 -603403668@IDITSystem com.idit.framework.common.context.EnvironmentAccessManager - setEnvironment( NULL ) will remove SuperEnvironment@16398e89 {user=-603403668@IDITSystem, subSessionId=1472225099, userContext=com.idit.framework.security.server.impl.UserContext {isArtificial=true, subSession ids=[], subSession map={} } }
[13-10-20 10:04:42.550+0200] INFO 82 org.quartz.plugins.history.LoggingTriggerHistoryPlugin - Trigger Elastic Search Indexing.-443779357@IDITSystem-2010131004144231799457059968000 completed firing job Elastic Search Indexing.Elastic Search Incremental Indexing Initiator Job number:-1037783044 at 10:04:42 10/13/2020 with resulting trigger instruction code: DELETE TRIGGER
filebeat log: (all log lines sentd by one event)
2020-10-14T08:59:08.438+0300 DEBUG [multiline] multiline/multiline.go:175 Multiline event flushed because timeout reached.
2020-10-14T08:59:11.861+0300 DEBUG [input] log/input.go:191 Start next scan
2020-10-14T08:59:11.861+0300 DEBUG [processors] processing/processors.go:186 Publish event: {
"@timestamp": "2020-10-14T05:59:03.437Z",
"@metadata": {
"beat": "filebeat",
"type": "_doc",
"version": "7.5.2"
},
"message": "[13-10-20 10:04:42.531+0200] INFO 82 org.quartz.plugins.history.LoggingTriggerHistoryPlugin - Trigger Elastic Search Indexing.-443779357@IDITSystem-2010131004144231799457059968000 fired job Elastic Search Indexing.Elastic Search Incremental Indexing Initiator Job number:-1037783044 at: 10:04:42 10/13/2020\n[13-10-20 10:04:42.531+0200] INFO 82 org.quartz.plugins.history.LoggingTriggerHistoryPlugin - Trigger Elastic Search Indexing.-443779357@IDITSystem-2010131004144231799457059968000 fired job Elastic Search Indexing.Elastic Search Incremental Indexing Initiator Job number:-1037783044 at: 10:04:42 10/13/2020\n[13-10-20 10:04:42.533+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.session.BasicEnvironment - fetchUserContext: Creating a 'NOT LOGGED IN' user context.\n[13-10-20 10:04:42.533+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.utils.TaskLogger - [SYS_TASK INFO : Elastic Search Incremental Indexing Initiator Job]: Job priority : 1\n[13-10-20 10:04:42.533+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.session.BasicEnvironment - fetchUserContext: Creating a 'NOT LOGGED IN' user context.\n[13-10-20 10:04:42.533+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.utils.TaskLogger - [SYS_TASK INFO : Elastic Search Incremental Indexing Initiator Job]: Job name : Elastic Search Incremental Indexing Initiator Job number:-1037783044\n[13-10-20 10:04:42.533+0200] INFO 82 -603403668@IDITSystem com.idit.reference.server.manager.IDITGeneralLoggerManager - updateBatchLogStatus(batchLog=BatchLogVO { status=CURRENT, versionNr=2, id=171066, groupName="Elastic Search Indexing", fileName="null", }, newStatus=8, startTime=Tue Oct 13 10:04:42 CEST 2020, groupName=Elastic Search Indexing, totalRecords=null, jobType=3) with old status = 8, old records = null\n[13-10-20 10:04:42.534+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.session.BasicEnvironment - fetchUserContext: Creating a 'NOT LOGGED IN' user context.\n[13-10-20 10:04:42.537+0200] INFO 82 -603403668@IDITSystem com.idit.framework.common.context.EnvironmentAccessManager - setEnvironment( NULL ) will remove SuperEnvironment@2cc972c2 {user=NOT_LOGGED_IN, subSessionId=null, userContext=null }\n[13-10-20 10:04:42.544+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.utils.TaskLogger - [SYS_TASK INFO : Elastic Search Incremental Indexing Initiator Job]: Job log status was updated to Unscheduled\n[13-10-20 10:04:42.544+0200] INFO 82 -603403668@IDITSystem com.idit.reference.server.manager.IDITGeneralLoggerManager - updateBatchLogStatus(batchLog=BatchLogVO { status=CHANGED, versionNr=3, id=171066, groupName="Elastic Search Indexing", fileName="null", }, newStatus=10) with old status = 8\n[13-10-20 10:04:42.547+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.utils.TaskLogger - [SYS_TASK INFO : Elastic Search Incremental Indexing Initiator Job]: The job Elastic Search Incremental Indexing Initiator Job is already in process\n[13-10-20 10:04:42.550+0200] INFO 82 -603403668@IDITSystem com.idit.framework.common.jaas.common.impl.IDITSecurityManager - Disconnect username: -603403668@IDITSystem\n[13-10-20 10:04:42.550+0200] ERROR 82 -603403668@IDITSystem com.idit.framework.backend.interceptor.helper.TxContextUtil - Calling removeEntitiesUnderTx without TX entry in Context! container.hashCode322808389 TxContextStack{txStack=}method name : logout transactionPhase.name() BEFORE_COMPLETION\n[13-10-20 10:04:42.550+0200] ERROR 82 -603403668@IDITSystem com.idit.framework.backend.interceptor.helper.TxContextUtil - Calling removeEntitiesUnderTx without TX entry in Context! container.hashCode 322808389 TxContextStack{txStack=} method name : logout transactionPhase.name() AFTER_FAILURE\n[13-10-20 10:04:42.550+0200] INFO 82 -603403668@IDITSystem com.idit.framework.common.context.EnvironmentAccessManager - setEnvironment( NULL ) will remove SuperEnvironment@16398e89 {user=-603403668@IDITSystem, subSessionId=1472225099, userContext=com.idit.framework.security.server.impl.UserContext {isArtificial=true, subSession ids=, subSession map={} } }\n[13-10-20 10:04:42.550+0200] INFO 82 org.quartz.plugins.history.LoggingTriggerHistoryPlugin - Trigger Elastic Search Indexing.-443779357@IDITSystem-2010131004144231799457059968000 completed firing job Elastic Search Indexing.Elastic Search Incremental Indexing Initiator Job number:-1037783044 at 10:04:42 10/13/2020 with resulting trigger instruction code: DELETE TRIGGER",
"log": {
"offset": 103397,
"file": {
"path": "c:\WLS1221Domains\wls-7003\IDITLog\IDITLog.log"
},
"flags": [
"multiline"
]
},
"input": {
"type": "log"
},
"agent": {
"ephemeral_id": "b46b9d8b-3ca7-4f77-91e3-0637e43c907b",
"hostname": "core-qa13",
"id": "862e6741-2e75-44ab-af34-6259c096ebe1",
"version": "7.5.2",
"type": "filebeat"
},
"ecs": {
"version": "1.1.0"
},
"host": {
"name": "core-qa13",
"hostname": "core-qa13",
"architecture": "x86_64",
"os": {
"name": "Windows Server 2016 Standard",
"kernel": "10.0.14393.693 (rs1_release.161220-1747)",
"build": "14393.693",
"platform": "windows",
"version": "10.0",
"family": "windows"
},
"id": "fa6e8934-542c-482a-90a9-314577929bd6"
}
}
Any idea? thanks