# Filebeat failed to insert too much log

**URL:** <https://discuss.elastic.co/t/filebeat-failed-to-insert-too-much-log/254607>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [November 7, 2020, 12:00pm UTC](https://discuss.elastic.co/t/filebeat-failed-to-insert-too-much-log/254607 "2020-11-07T12:00:44Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mohammad\_Mousavi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mohammad_mousavi/32/56508_2.png) [@Mohammad\_Mousavi](https://discuss.elastic.co/u/Mohammad_Mousavi)\
**Post date:** [November 7, 2020, 12:00pm UTC](https://discuss.elastic.co/t/filebeat-failed-to-insert-too-much-log/254607/1 "2020-11-07T12:00:44Z")

</div>

Hi, It's look like my filebeats agents are failed to insert some logs into elasticsearch.  
I have multiple filebeats and 3 logstash and 3 clustered elasticsearch (1 shard , 2 replica).  
I see some data exists in nginx logs but misses in kibana, when I restart my filebeats and logstash this happen:

 ![1](https://us1.discourse-cdn.com/elastic/original/3X/4/e/4e08ddde822a5310026d44eb9740d565333a63e0.png)

It's start to insert logs that was missed, based on timestamps it's inserting logs for some hours ago.  
Here an example, the number of nginx logs for yesterday is 1,340,918 but number of docs in elastcsearch for that agent is 214,000. it's look like I'm missing some data everyday.  
I didn't config any queue or anything else but logpath and logstash endpoin in filebeat config file.  
I'm confused, Can someone help me with this ? thanx a lot  
(3 elasticsearch node 8 GB JVM, 16 GB Memory on VM )  
(3 logstash with 1500 MB JVM)

---

<div class="post-metadata">

**Author:** ![mtojek](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mtojek/32/63863_2.png) [@mtojek](https://discuss.elastic.co/u/mtojek)\
**Post date:** [November 9, 2020, 8:28am UTC](https://discuss.elastic.co/t/filebeat-failed-to-insert-too-much-log/254607/2 "2020-11-09T08:28:04Z")

</div>

Did you check your filebeat instances for error logs?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 7, 2020, 10:28am UTC](https://discuss.elastic.co/t/filebeat-failed-to-insert-too-much-log/254607/3 "2020-12-07T10:28:06Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.