# Merge the varnish logs into a single line

**URL:** <https://discuss.elastic.co/t/merge-the-varnish-logs-into-a-single-line/186610>\
**Category:** Logstash\
**Created:** [June 20, 2019, 7:08am UTC](https://discuss.elastic.co/t/merge-the-varnish-logs-into-a-single-line/186610 "2019-06-20T07:08:19Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Yuvraj\_S](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuvraj_s/32/48410_2.png) [@Yuvraj\_S](https://discuss.elastic.co/u/Yuvraj_S)\
**Post date:** [June 20, 2019, 7:08am UTC](https://discuss.elastic.co/t/merge-the-varnish-logs-into-a-single-line/186610/1 "2019-06-20T07:08:19Z")

</div>

hi team,

i am using varnish application which gives a lots of logs in the messages. The need is to merge these lines into a single line in the message. Can you please suggest any options of merging those lines.

actual logs:

- \<\< BeReq \>\>

- Begin
- Timestamp
- BereqURL
- BereqProtocol
- BereqHeader
- BereqHeader
- BereqHeader
- BereqHeader
- BereqHeader
- BereqHeader  
END

expected logs in the ELK,

- \<\< BeReq \>\> - Begin - Timestamp - BereqURL - BereqProtocol - BereqHeader - BereqHeader - BereqHeader - BereqHeader - BereqHeader - BereqHeader END

need to use a string matching in the grok filter which will start merging the logs when it find the string "BeReq" and merge till it finds the string "END"

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 20, 2019, 11:18am UTC](https://discuss.elastic.co/t/merge-the-varnish-logs-into-a-single-line/186610/2 "2019-06-20T11:18:44Z")

</div>

You could do this using a [multiline](https://www.elastic.co/guide/en/logstash/current/plugins-codecs-multiline.html) codec on your input.

---

<div class="post-metadata">

**Author:** ![Yuvraj\_S](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuvraj_s/32/48410_2.png) [@Yuvraj\_S](https://discuss.elastic.co/u/Yuvraj_S)\
**Post date:** [June 21, 2019, 5:53am UTC](https://discuss.elastic.co/t/merge-the-varnish-logs-into-a-single-line/186610/3 "2019-06-21T05:53:39Z")

</div>

Hi Thanks for the reply. But I saw it in multiline, it just allows me to club 2 lines at a time, which is not what i want.

here is how my log looks like,  
#cat test.log  
rob  
adam  
ram  
shyam  
amanda  
bob  
silva  
adam  
d  
e  
f  
paks

Output which is coming currently:

adam  
ram  
#############  
the next line comes  
....  
...  
then,  
adam  
d  
################  
the next line starts again.

actual output which i require is,

adam  
ram  
shyam  
amanda  
bob  
silva

merge the lines when it meets adam till it finds next adam,

here is my config how it looks,

multiline.pattern: ^adam  
multiline.negate: false  
multiline.match: before

adding the info on the architecture am using,

using filbeat in client and then ship the logs to logstash,

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [June 21, 2019, 1:45pm UTC](https://discuss.elastic.co/t/merge-the-varnish-logs-into-a-single-line/186610/4 "2019-06-21T13:45:03Z")

</div>

```
codec => multiline { pattern => "^adam" negate => true what => previous auto_flush_interval => 1 } }

```

will get you

```
   "message" => "rob",
   "message" => "adam\nram\nshyam\namanda\nbob\nsilva",
   "message" => "adam\nd\ne\nf\npaks",
```

---

<div class="post-metadata">

**Author:** ![Yuvraj\_S](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuvraj_s/32/48410_2.png) [@Yuvraj\_S](https://discuss.elastic.co/u/Yuvraj_S)\
**Post date:** [July 9, 2019, 5:08am UTC](https://discuss.elastic.co/t/merge-the-varnish-logs-into-a-single-line/186610/5 "2019-07-09T05:08:43Z")

</div>

Thanks Badger, tried the same and it was working

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 6, 2019, 5:08am UTC](https://discuss.elastic.co/t/merge-the-varnish-logs-into-a-single-line/186610/6 "2019-08-06T05:08:49Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.