# Log insertion with Filebeat

**URL:** <https://discuss.elastic.co/t/log-insertion-with-filebeat/232653>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [May 14, 2020, 1:41pm UTC](https://discuss.elastic.co/t/log-insertion-with-filebeat/232653 "2020-05-14T13:41:58Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![JulienN](https://avatars.discourse-cdn.com/v4/letter/j/5daacb/32.png) [@JulienN](https://discuss.elastic.co/u/JulienN)\
**Post date:** [May 14, 2020, 1:41pm UTC](https://discuss.elastic.co/t/log-insertion-with-filebeat/232653/1 "2020-05-14T13:41:58Z")

</div>

Hi ,

I use Filebeat to recover my scheduler's log .

It's functional but the elasticsearch insertion is not optimal ... it inserts one hit by line on my log.

Ideally , i would like to insert the entire log i one hit..

Do you have any idea to permit this?

Thanks

Regards

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [May 14, 2020, 6:51pm UTC](https://discuss.elastic.co/t/log-insertion-with-filebeat/232653/2 "2020-05-14T18:51:58Z")

</div>

Hey @JulienN,

in principle you can configure [multiline](https://www.elastic.co/guide/en/beats/filebeat/7.7/multiline-examples.html) rules to match the whole file as a single event.

You may face some problems depending on how your scheduler writes its logs. For example filebeat needs that all lines are terminated by a new line, this is important to get the last line of a log file. Also, the file needs to be properly rotated, filebeat is not going to start reading from the beginning a file that it has already read.

Could you give more details on how these log files are written?

---

<div class="post-metadata">

**Author:** ![JulienN](https://avatars.discourse-cdn.com/v4/letter/j/5daacb/32.png) [@JulienN](https://discuss.elastic.co/u/JulienN)\
**Post date:** [May 15, 2020, 3:17pm UTC](https://discuss.elastic.co/t/log-insertion-with-filebeat/232653/3 "2020-05-15T15:17:20Z")

</div>

Hi Jaime ,

Thanks for your reply !!

Yes the multiline could be the solution , but my logs are a bit different by machine.

For example , a log can begin by timestamp YYYY-MM-DD , or directly by functional details of the execution (a character) .. so it's difficult to determine a single pattern.. have you any idea ??

Thanks

Regards

Julien

---

<div class="post-metadata">

**Author:** ![JulienN](https://avatars.discourse-cdn.com/v4/letter/j/5daacb/32.png) [@JulienN](https://discuss.elastic.co/u/JulienN)\
**Post date:** [May 22, 2020, 1:21pm UTC](https://discuss.elastic.co/t/log-insertion-with-filebeat/232653/4 "2020-05-22T13:21:42Z")

</div>

I found the good pattern with multiple tests on [https://play.golang.org/](https://play.golang.org/)

Now with this pattern , it would be good :

multiline.pattern: '^=[A-Z]+|^$'  
multiline.negate: false

But i've a problem , it's exactly the same thing , one hit on Elastic by line on my logs...

Any ideas ??

Regards

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [May 25, 2020, 9:22am UTC](https://discuss.elastic.co/t/log-insertion-with-filebeat/232653/5 "2020-05-25T09:22:11Z")

</div>

Do you mean that with this multiline configuration, the result is the same? Could you share the whole configuration you are using now?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [June 22, 2020, 9:22am UTC](https://discuss.elastic.co/t/log-insertion-with-filebeat/232653/6 "2020-06-22T09:22:17Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.