# Filebeat for shipping the entire logfile as it is

**URL:** <https://discuss.elastic.co/t/filebeat-for-shipping-the-entire-logfile-as-it-is/178565>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [April 26, 2019, 6:36am UTC](https://discuss.elastic.co/t/filebeat-for-shipping-the-entire-logfile-as-it-is/178565 "2019-04-26T06:36:52Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![shub8050](https://avatars.discourse-cdn.com/v4/letter/s/ccd318/32.png) [@shub8050](https://discuss.elastic.co/u/shub8050)\
**Post date:** [April 26, 2019, 6:36am UTC](https://discuss.elastic.co/t/filebeat-for-shipping-the-entire-logfile-as-it-is/178565/1 "2019-04-26T06:36:52Z")

</div>

I need to get the entire log file as it is in the elk stack.  
Consider I have this log: `<line1> <line2> <line3>|`  
If my pointer is placed on the line 3 i will get the entire log file except `<line3>`  
. If the cursor is in new line then i will be able to get the file as expected.

This is my filebeat: `multiline.pattern: '.*' multiline.negate: false multiline.match: after`

Kindly help me out with the settings. ![attach1](https://us1.discourse-cdn.com/elastic/original/3X/9/d/9dcd904123f62e7a93d73715386fffd0b0270421.png)

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [April 26, 2019, 3:23pm UTC](https://discuss.elastic.co/t/filebeat-for-shipping-the-entire-logfile-as-it-is/178565/2 "2019-04-26T15:23:21Z")

</div>

What is written in line 3? One needs a regular expression to detect multiline patterns. The pattern `.*` just captures everything.

---

<div class="post-metadata">

**Author:** ![shub8050](https://avatars.discourse-cdn.com/v4/letter/s/ccd318/32.png) [@shub8050](https://discuss.elastic.co/u/shub8050)\
**Post date:** [April 26, 2019, 3:27pm UTC](https://discuss.elastic.co/t/filebeat-for-shipping-the-entire-logfile-as-it-is/178565/3 "2019-04-26T15:27:54Z")

</div>

> [@steffens](#):
>
> ression to detect multiline patterns.

I actually want the entire data from the log file. There are different type of logs which iam trying to import so in my case there is no fixed pattern as such.

---

<div class="post-metadata">

**Author:** ![shub8050](https://avatars.discourse-cdn.com/v4/letter/s/ccd318/32.png) [@shub8050](https://discuss.elastic.co/u/shub8050)\
**Post date:** [April 29, 2019, 11:37am UTC](https://discuss.elastic.co/t/filebeat-for-shipping-the-entire-logfile-as-it-is/178565/4 "2019-04-29T11:37:39Z")

</div>

Iam waiting for it. Please respond.

---

<div class="post-metadata">

**Author:** ![Abhilash\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abhilash_b/32/40270_2.png) [@Abhilash\_B](https://discuss.elastic.co/u/Abhilash_B)\
**Post date:** [April 29, 2019, 11:42am UTC](https://discuss.elastic.co/t/filebeat-for-shipping-the-entire-logfile-as-it-is/178565/5 "2019-04-29T11:42:12Z")

</div>

Try using `multiline.pattern: '.'` to capture the entire log file as a single event.

---

<div class="post-metadata">

**Author:** ![shub8050](https://avatars.discourse-cdn.com/v4/letter/s/ccd318/32.png) [@shub8050](https://discuss.elastic.co/u/shub8050)\
**Post date:** [April 29, 2019, 12:04pm UTC](https://discuss.elastic.co/t/filebeat-for-shipping-the-entire-logfile-as-it-is/178565/6 "2019-04-29T12:04:31Z")

</div>

This pattern sends each line as a separate event.

---

<div class="post-metadata">

**Author:** ![Abhilash\_B](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/abhilash_b/32/40270_2.png) [@Abhilash\_B](https://discuss.elastic.co/u/Abhilash_B)\
**Post date:** [April 29, 2019, 12:41pm UTC](https://discuss.elastic.co/t/filebeat-for-shipping-the-entire-logfile-as-it-is/178565/7 "2019-04-29T12:41:12Z")

</div>

Can you attach a sample log file with a few lines?

---

<div class="post-metadata">

**Author:** ![shub8050](https://avatars.discourse-cdn.com/v4/letter/s/ccd318/32.png) [@shub8050](https://discuss.elastic.co/u/shub8050)\
**Post date:** [April 29, 2019, 12:59pm UTC](https://discuss.elastic.co/t/filebeat-for-shipping-the-entire-logfile-as-it-is/178565/8 "2019-04-29T12:59:41Z")

</div>

![line1](https://us1.discourse-cdn.com/elastic/original/3X/f/1/f189a8ec6cdaf3e7ee97b8f8a23197e5595596e1.png)

Iam able to get the entire file when the pointer is pointing in new line as shown above but if the pointer is placed at the end of `<EndLine>`| , then I will get all the data till `<SecondLine>ABC` skipping the end line.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [April 29, 2019, 3:14pm UTC](https://discuss.elastic.co/t/filebeat-for-shipping-the-entire-logfile-as-it-is/178565/9 "2019-04-29T15:14:50Z")

</div>

Filebeat first splits the file into lines. The multiline processing combines them again. One can use `multiline.timeout` to trigger a flush of the multiline buffer, but `EndLine` might not be included in this case.

---

<div class="post-metadata">

**Author:** ![shub8050](https://avatars.discourse-cdn.com/v4/letter/s/ccd318/32.png) [@shub8050](https://discuss.elastic.co/u/shub8050)\
**Post date:** [April 29, 2019, 3:20pm UTC](https://discuss.elastic.co/t/filebeat-for-shipping-the-entire-logfile-as-it-is/178565/10 "2019-04-29T15:20:04Z")

</div>

That's exactly my issue, kindly tell me if it is possible to get the full file including the `<EndLine>`. I'm stuck here for a long time, I have also attached my filebeat config file above in my question description for further details.

---

<div class="post-metadata">

**Author:** ![steffens](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/steffens/32/79630_2.png) [@steffens](https://discuss.elastic.co/u/steffens)\
**Post date:** [April 30, 2019, 10:58am UTC](https://discuss.elastic.co/t/filebeat-for-shipping-the-entire-logfile-as-it-is/178565/11 "2019-04-30T10:58:59Z")

</div>

Without the final new line control character it's currently not possible.  
There is an open enhancement request for sending complete files: [https://github.com/elastic/beats/issues/4982](https://github.com/elastic/beats/issues/4982)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 28, 2019, 10:59am UTC](https://discuss.elastic.co/t/filebeat-for-shipping-the-entire-logfile-as-it-is/178565/12 "2019-05-28T10:59:36Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
