# Multiple logs are concatenated to single event by Logstash

**URL:** <https://discuss.elastic.co/t/multiple-logs-are-concatenated-to-single-event-by-logstash/248947>\
**Category:** Logstash\
**Created:** [September 17, 2020, 9:16am UTC](https://discuss.elastic.co/t/multiple-logs-are-concatenated-to-single-event-by-logstash/248947 "2020-09-17T09:16:39Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Khushboo\_Kumari](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/khushboo_kumari/32/75758_2.png) [@Khushboo\_Kumari](https://discuss.elastic.co/u/Khushboo_Kumari)\
**Post date:** [September 17, 2020, 9:16am UTC](https://discuss.elastic.co/t/multiple-logs-are-concatenated-to-single-event-by-logstash/248947/1 "2020-09-17T09:16:39Z")

</div>

```auto
Hi all,
I am using Filebeat to send Multiline logs to Logstash ,
Filebeat input is a file ( logs.txt), In case multiple events are published to (Logs.txt ) in a millisecond, they all are added as 1 log event to ES. Please find below the configuration:

```

```auto
**==================Filebeat Configuration=============================**
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - C:\store\Logs\Logs.txt
  multiline.type: pattern
  multiline.pattern: '^((\-{40}.*\s)Message: (?P<Message>.*\s*)ID: (?P<ID>.*\s*)Severity: (?P<Severity>.*\s*)Timestamp: (?P<Timestamp>.*\s*)ExtendedProperties: (?P<ExtendedProperties>[\s\S]*?)(\-{40}))'
  multiline.negate: true 
  multiline.match: before
  tags: ["multilinelogs"]
**===================================================================**

```

```auto
**==================Logstash Configuration=============================**
input {

    beats {
		port => 5000
	}
}
filter {
        grok {
			match => { "message" => "^((\-{40}.*\s)Message: (?<Message>.*\s*)ID: (?<ID>.*\s*)Severity: (?<Severity>.*\s*)Timestamp: (?<Timestamp>.*\s*)ExtendedProperties: (?<ExtendedProperties>[\s\S]*?)(\-{40}))"
			
		}
	}
	
}

output {
	elasticsearch {
		hosts => ["http://localhost:9200"]
		index => "multilinelogs"
	}	
}

**===================================================================**

```

```auto
**==================Logs.txt=============================**
----------------------------------------
Message: successfully created
ID: 1
Severity: Information
Timestamp: 2020-09-13T01:31:18.344+05:32
Extended Properties: MsgCreateTime - 2020-09-13T01:18:09.262+05:30
ForceLog - True
----------------------------------------
----------------------------------------
Message: successfully created
ID: 2
Severity: Information
Timestamp: 2020-09-13T01:31:18.344+05:32
Extended Properties: MsgCreateTime - 2020-09-13T01:18:09.262+05:30
ForceLog - True
----------------------------------------
----------------------------------------
Message: successfully created
ID: 3
Severity: Information
Timestamp: 2020-09-13T01:31:18.344+05:32
Extended Properties: MsgCreateTime - 2020-09-13T01:18:09.262+05:30
ForceLog - True
----------------------------------------

**===================================================================**

```

`Issue1:`  
`In kibana -> it puts all logs together in one event. `

```auto
**FileBeat Image**![FB|690x77](upload://swJZdm595qHmsd58MDRlL3pqPQJ.png)

**Logstash image**![Logstash|690x142](upload://9KhXMrCx1KoFvwXLbLx9oSYaRsY.png) 

**Kibana image**![kibana|690x424](upload://ebO27zcPCtundaLnMbohMEXDFLt.png) 

```

```auto
**Expected Output**
All the three logs should be read as 3 separate events .
How do we acheive that in Logstash.. where is the problem? in Logstash or Filebeat? Is there issue with read rate of logstash? sometime it aggregates 2 events or 5 events together.

```

---

<div class="post-metadata">

**Author:** ![Khushboo\_Kumari](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/khushboo_kumari/32/75758_2.png) [@Khushboo\_Kumari](https://discuss.elastic.co/u/Khushboo_Kumari)\
**Post date:** [September 17, 2020, 12:01pm UTC](https://discuss.elastic.co/t/multiple-logs-are-concatenated-to-single-event-by-logstash/248947/2 "2020-09-17T12:01:18Z")

</div>

This Issue happens with Multi-line logs specifically.

---

<div class="post-metadata">

**Author:** ![Khushboo\_Kumari](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/khushboo_kumari/32/75758_2.png) [@Khushboo\_Kumari](https://discuss.elastic.co/u/Khushboo_Kumari)\
**Post date:** [September 21, 2020, 3:44pm UTC](https://discuss.elastic.co/t/multiple-logs-are-concatenated-to-single-event-by-logstash/248947/3 "2020-09-21T15:44:20Z")

</div>

`  
Resolved the issue ... I was supposed to use multiline.flush\_pattern: '^(-{40})$'  
This line arks the endpoint of my log .  
So now Filebeat is able to flush each log as separate event. 😅

`

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 19, 2020, 3:44pm UTC](https://discuss.elastic.co/t/multiple-logs-are-concatenated-to-single-event-by-logstash/248947/4 "2020-10-19T15:44:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.