# Filebeat multiline configuration consolidate all logs line to one event
**URL:** https://discuss.elastic.co/t/filebeat-multiline-configuration-consolidate-all-logs-line-to-one-event/251993
**Category:** Beats
**Tags:** filebeat
**Created:** [October 14, 2020, 6:07am UTC](https://discuss.elastic.co/t/filebeat-multiline-configuration-consolidate-all-logs-line-to-one-event/251993 "2020-10-14T06:07:05Z")
**Posts on this page:** 2
**Page:** 1
### Author:  [@HadarPeeran](https://discuss.elastic.co/u/HadarPeeran)
#### Post date: [October 14, 2020, 6:07am UTC](https://discuss.elastic.co/t/filebeat-multiline-configuration-consolidate-all-logs-line-to-one-event/251993/1 "2020-10-14T06:07:05Z")
I have the following setting: Filebeat =\> Logstash
In order to support java stack trace I added [multiline configuration](https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html#multiline-examples) in filebeat.yml
```
multiline.pattern: '^\[[0-9]{2}-[0-9]{2}-[0-9]{2}'
multiline.negate: true
multiline.match: after
```
However filebeat handles all log lines as one event and in kibana they displayed as one log line
Log example:
```
[13-10-20 10:04:42.531+0200] INFO 82 org.quartz.plugins.history.LoggingTriggerHistoryPlugin - Trigger Elastic Search Indexing.-443779357@IDITSystem-2010131004144231799457059968000 fired job Elastic Search Indexing.Elastic Search Incremental Indexing Initiator Job number:-1037783044 at: 10:04:42 10/13/2020
[13-10-20 10:04:42.531+0200] INFO 82 org.quartz.plugins.history.LoggingTriggerHistoryPlugin - Trigger Elastic Search Indexing.-443779357@IDITSystem-2010131004144231799457059968000 fired job Elastic Search Indexing.Elastic Search Incremental Indexing Initiator Job number:-1037783044 at: 10:04:42 10/13/2020
[13-10-20 10:04:42.533+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.session.BasicEnvironment - fetchUserContext: Creating a 'NOT LOGGED IN' user context.
[13-10-20 10:04:42.533+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.utils.TaskLogger - [SYS_TASK INFO : Elastic Search Incremental Indexing Initiator Job]: Job priority : 1
[13-10-20 10:04:42.533+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.session.BasicEnvironment - fetchUserContext: Creating a 'NOT LOGGED IN' user context.
[13-10-20 10:04:42.533+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.utils.TaskLogger - [SYS_TASK INFO : Elastic Search Incremental Indexing Initiator Job]: Job name : Elastic Search Incremental Indexing Initiator Job number:-1037783044
[13-10-20 10:04:42.533+0200] INFO 82 -603403668@IDITSystem com.idit.reference.server.manager.IDITGeneralLoggerManager - updateBatchLogStatus(batchLog=BatchLogVO { status=CURRENT, versionNr=2, id=171066, groupName="Elastic Search Indexing", fileName="null", }, newStatus=8, startTime=Tue Oct 13 10:04:42 CEST 2020, groupName=Elastic Search Indexing, totalRecords=null, jobType=3) with old status = 8, old records = null
[13-10-20 10:04:42.534+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.session.BasicEnvironment - fetchUserContext: Creating a 'NOT LOGGED IN' user context.
[13-10-20 10:04:42.537+0200] INFO 82 -603403668@IDITSystem com.idit.framework.common.context.EnvironmentAccessManager - setEnvironment( NULL ) will remove SuperEnvironment@2cc972c2 {user=NOT_LOGGED_IN, subSessionId=null, userContext=null }
[13-10-20 10:04:42.544+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.utils.TaskLogger - [SYS_TASK INFO : Elastic Search Incremental Indexing Initiator Job]: Job log status was updated to Unscheduled
[13-10-20 10:04:42.544+0200] INFO 82 -603403668@IDITSystem com.idit.reference.server.manager.IDITGeneralLoggerManager - updateBatchLogStatus(batchLog=BatchLogVO { status=CHANGED, versionNr=3, id=171066, groupName="Elastic Search Indexing", fileName="null", }, newStatus=10) with old status = 8
[13-10-20 10:04:42.547+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.utils.TaskLogger - [SYS_TASK INFO : Elastic Search Incremental Indexing Initiator Job]: The job Elastic Search Incremental Indexing Initiator Job is already in process
[13-10-20 10:04:42.550+0200] INFO 82 -603403668@IDITSystem com.idit.framework.common.jaas.common.impl.IDITSecurityManager - Disconnect username: -603403668@IDITSystem
[13-10-20 10:04:42.550+0200] ERROR 82 -603403668@IDITSystem com.idit.framework.backend.interceptor.helper.TxContextUtil - Calling removeEntitiesUnderTx without TX entry in Context! container.hashCode322808389 TxContextStack{txStack=[]}method name : logout transactionPhase.name() BEFORE_COMPLETION
[13-10-20 10:04:42.550+0200] ERROR 82 -603403668@IDITSystem com.idit.framework.backend.interceptor.helper.TxContextUtil - Calling removeEntitiesUnderTx without TX entry in Context! container.hashCode 322808389 TxContextStack{txStack=[]} method name : logout transactionPhase.name() AFTER_FAILURE
[13-10-20 10:04:42.550+0200] INFO 82 -603403668@IDITSystem com.idit.framework.common.context.EnvironmentAccessManager - setEnvironment( NULL ) will remove SuperEnvironment@16398e89 {user=-603403668@IDITSystem, subSessionId=1472225099, userContext=com.idit.framework.security.server.impl.UserContext {isArtificial=true, subSession ids=[], subSession map={} } }
[13-10-20 10:04:42.550+0200] INFO 82 org.quartz.plugins.history.LoggingTriggerHistoryPlugin - Trigger Elastic Search Indexing.-443779357@IDITSystem-2010131004144231799457059968000 completed firing job Elastic Search Indexing.Elastic Search Incremental Indexing Initiator Job number:-1037783044 at 10:04:42 10/13/2020 with resulting trigger instruction code: DELETE TRIGGER
```
filebeat log: (all log lines sentd by one event)
> 2020-10-14T08:59:08.438+0300 DEBUG [multiline] multiline/multiline.go:175 Multiline event flushed because timeout reached.
> 2020-10-14T08:59:11.861+0300 DEBUG [input] log/input.go:191 Start next scan
> 2020-10-14T08:59:11.861+0300 DEBUG [processors] processing/processors.go:186 Publish event: {
> "@timestamp": "2020-10-14T05:59:03.437Z",
> "@metadata": {
> "beat": "filebeat",
> "type": "\_doc",
> "version": "7.5.2"
> },
> "message": "[13-10-20 10:04:42.531+0200] INFO 82 org.quartz.plugins.history.LoggingTriggerHistoryPlugin - Trigger Elastic Search Indexing.-443779357@IDITSystem-2010131004144231799457059968000 fired job Elastic Search Indexing.Elastic Search Incremental Indexing Initiator Job number:-1037783044 at: 10:04:42 10/13/2020\n[13-10-20 10:04:42.531+0200] INFO 82 org.quartz.plugins.history.LoggingTriggerHistoryPlugin - Trigger Elastic Search Indexing.-443779357@IDITSystem-2010131004144231799457059968000 fired job Elastic Search Indexing.Elastic Search Incremental Indexing Initiator Job number:-1037783044 at: 10:04:42 10/13/2020\n[13-10-20 10:04:42.533+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.session.BasicEnvironment - fetchUserContext: Creating a 'NOT LOGGED IN' user context.\n[13-10-20 10:04:42.533+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.utils.TaskLogger - [SYS\_TASK INFO : Elastic Search Incremental Indexing Initiator Job]: Job priority : 1\n[13-10-20 10:04:42.533+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.session.BasicEnvironment - fetchUserContext: Creating a 'NOT LOGGED IN' user context.\n[13-10-20 10:04:42.533+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.utils.TaskLogger - [SYS\_TASK INFO : Elastic Search Incremental Indexing Initiator Job]: Job name : Elastic Search Incremental Indexing Initiator Job number:-1037783044\n[13-10-20 10:04:42.533+0200] INFO 82 -603403668@IDITSystem com.idit.reference.server.manager.IDITGeneralLoggerManager - updateBatchLogStatus(batchLog=BatchLogVO { status=CURRENT, versionNr=2, id=171066, groupName="Elastic Search Indexing", fileName="null", }, newStatus=8, startTime=Tue Oct 13 10:04:42 CEST 2020, groupName=Elastic Search Indexing, totalRecords=null, jobType=3) with old status = 8, old records = null\n[13-10-20 10:04:42.534+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.session.BasicEnvironment - fetchUserContext: Creating a 'NOT LOGGED IN' user context.\n[13-10-20 10:04:42.537+0200] INFO 82 -603403668@IDITSystem com.idit.framework.common.context.EnvironmentAccessManager - setEnvironment( NULL ) will remove SuperEnvironment@2cc972c2 {user=NOT\_LOGGED\_IN, subSessionId=null, userContext=null }\n[13-10-20 10:04:42.544+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.utils.TaskLogger - [SYS\_TASK INFO : Elastic Search Incremental Indexing Initiator Job]: Job log status was updated to Unscheduled\n[13-10-20 10:04:42.544+0200] INFO 82 -603403668@IDITSystem com.idit.reference.server.manager.IDITGeneralLoggerManager - updateBatchLogStatus(batchLog=BatchLogVO { status=CHANGED, versionNr=3, id=171066, groupName="Elastic Search Indexing", fileName="null", }, newStatus=10) with old status = 8\n[13-10-20 10:04:42.547+0200] INFO 82 -603403668@IDITSystem com.idit.framework.backend.utils.TaskLogger - [SYS\_TASK INFO : Elastic Search Incremental Indexing Initiator Job]: The job Elastic Search Incremental Indexing Initiator Job is already in process\n[13-10-20 10:04:42.550+0200] INFO 82 -603403668@IDITSystem com.idit.framework.common.jaas.common.impl.IDITSecurityManager - Disconnect username: -603403668@IDITSystem\n[13-10-20 10:04:42.550+0200] ERROR 82 -603403668@IDITSystem com.idit.framework.backend.interceptor.helper.TxContextUtil - Calling removeEntitiesUnderTx without TX entry in Context! container.hashCode322808389 TxContextStack{txStack=}method name : logout transactionPhase.name() BEFORE\_COMPLETION\n[13-10-20 10:04:42.550+0200] ERROR 82 -603403668@IDITSystem com.idit.framework.backend.interceptor.helper.TxContextUtil - Calling removeEntitiesUnderTx without TX entry in Context! container.hashCode 322808389 TxContextStack{txStack=} method name : logout transactionPhase.name() AFTER\_FAILURE\n[13-10-20 10:04:42.550+0200] INFO 82 -603403668@IDITSystem com.idit.framework.common.context.EnvironmentAccessManager - setEnvironment( NULL ) will remove SuperEnvironment@16398e89 {user=-603403668@IDITSystem, subSessionId=1472225099, userContext=com.idit.framework.security.server.impl.UserContext {isArtificial=true, subSession ids=, subSession map={} } }\n[13-10-20 10:04:42.550+0200] INFO 82 org.quartz.plugins.history.LoggingTriggerHistoryPlugin - Trigger Elastic Search Indexing.-443779357@IDITSystem-2010131004144231799457059968000 completed firing job Elastic Search Indexing.Elastic Search Incremental Indexing Initiator Job number:-1037783044 at 10:04:42 10/13/2020 with resulting trigger instruction code: DELETE TRIGGER",
> "log": {
> "offset": 103397,
> "file": {
> "path": "c:\WLS1221Domains\wls-7003\IDITLog\IDITLog.log"
> },
> "flags": [
> "multiline"
> ]
> },
> "input": {
> "type": "log"
> },
> "agent": {
> "ephemeral\_id": "b46b9d8b-3ca7-4f77-91e3-0637e43c907b",
> "hostname": "core-qa13",
> "id": "862e6741-2e75-44ab-af34-6259c096ebe1",
> "version": "7.5.2",
> "type": "filebeat"
> },
> "ecs": {
> "version": "1.1.0"
> },
> "host": {
> "name": "core-qa13",
> "hostname": "core-qa13",
> "architecture": "x86\_64",
> "os": {
> "name": "Windows Server 2016 Standard",
> "kernel": "10.0.14393.693 (rs1\_release.161220-1747)",
> "build": "14393.693",
> "platform": "windows",
> "version": "10.0",
> "family": "windows"
> },
> "id": "fa6e8934-542c-482a-90a9-314577929bd6"
> }
> }
Any idea? thanks
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [November 11, 2020, 8:07am UTC](https://discuss.elastic.co/t/filebeat-multiline-configuration-consolidate-all-logs-line-to-one-event/251993/2 "2020-11-11T08:07:08Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.