# Merge lines

**URL:** https://discuss.elastic.co/t/merge-lines/168890
**Category:** Beats
**Tags:** filebeat
**Created:** [February 18, 2019, 7:23pm UTC](https://discuss.elastic.co/t/merge-lines/168890 "2019-02-18T19:23:14Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![Adrian\_Martinez\_Doca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adrian_martinez_doca/32/61141_2.png) [@Adrian\_Martinez\_Doca](https://discuss.elastic.co/u/Adrian_Martinez_Doca)
#### Post date: [February 18, 2019, 7:23pm UTC](https://discuss.elastic.co/t/merge-lines/168890/1 "2019-02-18T19:23:14Z")

</div>

Hi, i am new in elastic and i am working to send logs from filebeat to logstash, but i have a issue and i not found solution. the log have multiple related lines ID, information and status, i would like join them in single line to process more easy in elastic.

Please can you help me with my questions:

Is possible to do that i want?  
what will be the procedure?

Regards

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [February 18, 2019, 7:27pm UTC](https://discuss.elastic.co/t/merge-lines/168890/2 "2019-02-18T19:27:50Z")

</div>

Are these lines 'intermixed' or always consecutive?  
Any kind of start/stop pattern?

---

<div class="post-metadata">

### Author: ![Adrian\_Martinez\_Doca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adrian_martinez_doca/32/61141_2.png) [@Adrian\_Martinez\_Doca](https://discuss.elastic.co/u/Adrian_Martinez_Doca)
#### Post date: [February 18, 2019, 8:01pm UTC](https://discuss.elastic.co/t/merge-lines/168890/3 "2019-02-18T20:01:19Z")

</div>

Hi, steffens, these lines are intermixed but always have the same pattern, for example

ID:  
Address:  
Enconding:  
Content-Type:  
Status:

i could exclude the lines that not are relevant for me.  
exclude\_lines: ['^Address','^Encoding','^Content-Type']

i would like have a single line with ID + status  
Regards

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [February 19, 2019, 9:19am UTC](https://discuss.elastic.co/t/merge-lines/168890/4 "2019-02-19T09:19:08Z")

</div>

Due to the lines being intermixed, but having an ID this is no plain multiline problem, but a correlation problem. You can try to solve this via aggregate filter in Logstash.

---

<div class="post-metadata">

### Author: ![Adrian\_Martinez\_Doca](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/adrian_martinez_doca/32/61141_2.png) [@Adrian\_Martinez\_Doca](https://discuss.elastic.co/u/Adrian_Martinez_Doca)
#### Post date: [February 19, 2019, 1:18pm UTC](https://discuss.elastic.co/t/merge-lines/168890/5 "2019-02-19T13:18:32Z")

</div>

Hi, but how i can related the lines with the specific id, i have other line that say me if the record is inbound or outbound, a example is this

ID:  
Address:  
Enconding:  
Content-Type:  
Status:  
Inbound:

ID:  
Address:  
Enconding:  
Content-Type:  
Status:  
Outbound:

Regards

---

<div class="post-metadata">

### Author: ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)
#### Post date: [February 20, 2019, 3:18pm UTC](https://discuss.elastic.co/t/merge-lines/168890/6 "2019-02-20T15:18:47Z")

</div>

Assuming 3 events and lines being intermixed your log might look like this at worst:

```auto
id_1
address_1
id_2
address_2
encoding_2
encoding_1
id_3
content-type_1
address_3
encoding_3
content-type_3
status_1
content-type_2
inbound_1
status_2
inbound_2
status_3
inbound_3

```

The problem with this is: you can't even rely on order, as event\_3 might surpass same lines from event\_1 or event\_2. In order to untangle these you need a correlation ID.

Your example suggest events are not intermixed, but written atomically. Like always:

```auto
id_1:
address_1:
encoding_1:
content-type_1:
status_1:
inbound_1:
id_2:
address_2:
encoding_2:
content-type_2:
status_2:
inbound_2:
id_3:
address_3:
encoding_3:
content-type_3:
status_3:
inbound_3:

```

If events are indeed written atomically, you can actually use multiline. Create a pattern looking for `^ID:` (assuming ID is always the first entry) and use `negate: true`, `match: after`. See the table in the [multiline docs](https://www.elastic.co/guide/en/beats/filebeat/current/multiline-examples.html#multiline).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 20, 2019, 3:18pm UTC](https://discuss.elastic.co/t/merge-lines/168890/7 "2019-03-20T15:18:52Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
