# Multiline message with repeated header

**URL:** <https://discuss.elastic.co/t/multiline-message-with-repeated-header/215324>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [January 16, 2020, 2:07pm UTC](https://discuss.elastic.co/t/multiline-message-with-repeated-header/215324 "2020-01-16T14:07:42Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![widhalmt](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/widhalmt/32/8237_2.png) [@widhalmt](https://discuss.elastic.co/u/widhalmt)\
**Post date:** [January 16, 2020, 2:07pm UTC](https://discuss.elastic.co/t/multiline-message-with-repeated-header/215324/1 "2020-01-16T14:07:42Z")

</div>

Hi,

I have a rather tricky situation. Multiline messages are sent through some transportation service which adds its own header. Due to policy (and technical) issues it's not possible to install Filebeat on the originating system.

What Filebeat reads looks like this:

```auto
12:34, service1, 124, This is a single line message, extradata123
12:36, service1, 138, This is a, extradata145
12:36, service1, 138, multline message, extradata145

```

So the multiline message is packed between other columns so I can't use the usual multiline solutions in Filebeat. Please mind that the other columns are identical in these multline messages. There is some sort of id or hash (represented by `124` and `138` in my example).

Before I discovered that there's this "special" form of multiline message I just used the `csv` Filter in Logstash and all was fine and cozy. Now I found these multiline messages and honestly I don't know that to do.

The data is written to a file which I collect with a bash script which pipes the data into filebeat. (I know, there are other ways but there are other reasons why I have to use the script). I'm just giving this information because I wouldn't mind solving this in a bash/python/ruby/assembler tool before piping it into filebeat.

Maybe you could give me a hint how to solve it? (Hint: Beating the developers of the originating tool until they send proper multiline messages might be fun but is not an option)

Cheers,  
Thomas

---

<div class="post-metadata">

**Author:** ![shaunak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shaunak/32/6643_2.png) [@shaunak](https://discuss.elastic.co/u/shaunak)\
**Post date:** [January 16, 2020, 8:21pm UTC](https://discuss.elastic.co/t/multiline-message-with-repeated-header/215324/2 "2020-01-16T20:21:25Z")

</div>

I wonder if you could solve this in Logstash using the `aggregate` filter: [https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html](https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [February 13, 2020, 8:22pm UTC](https://discuss.elastic.co/t/multiline-message-with-repeated-header/215324/3 "2020-02-13T20:22:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.