# Filebeat push event in very slow rate **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [December 6, 2021, 7:25am UTC](https://discuss.elastic.co/t/filebeat-push-event-in-very-slow-rate/291013 "2021-12-06T07:25:06Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![lusynda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusynda/32/53557_2.png) [@lusynda](https://discuss.elastic.co/u/lusynda)\ **Post date:** [December 6, 2021, 7:25am UTC](https://discuss.elastic.co/t/filebeat-push-event-in-very-slow-rate/291013/1 "2021-12-06T07:25:07Z")
Hi all I have a problems with one of my filebeat installed server. That server is config to get f5 log. then i install filebeat to get log from that server to Elasticsearch. But back before that setup working fine and the log rate are pushed at a normal rate but now with the new server the rate of pushing is like weirdly push on elastic, about every 10-15 minute it push a chunk of log with few thousands event to elastic. I have tried everything i can things of to fix the issue but nothing works, i have increased the ram of the filebeat server, increased the worker of logstash, increased internal queue of filebeat. Please help! Thanks for your time. ---
**Author:** ![Mario\_Castro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mario_castro/32/35107_2.png) [@Mario\_Castro](https://discuss.elastic.co/u/Mario_Castro)\ **Post date:** [December 6, 2021, 10:58am UTC](https://discuss.elastic.co/t/filebeat-push-event-in-very-slow-rate/291013/2 "2021-12-06T10:58:00Z")
Hi @lusynda 🙂 I'm gonna start with the obvious: there's something different on your new server so I'll start looking there. From your description, I can only assume that your new server is sending too few logs and filebeat doesn't send the messages until the queue is totally filled. Can you describe the rate and load of your events? How many events per second? How big each event is in KB? Check also that you have activated the expected log level in your new server. It might be the case that you were logging DEBUG logs before and now your are using INFO 😅 ---
**Author:** ![lusynda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusynda/32/53557_2.png) [@lusynda](https://discuss.elastic.co/u/lusynda)\ **Post date:** [December 6, 2021, 11:48am UTC](https://discuss.elastic.co/t/filebeat-push-event-in-very-slow-rate/291013/3 "2021-12-06T11:48:18Z")
Thanks for your replies. > [@Mario\_Castro](#): > > From your description, I can only assume that your new server is sending too few logs and filebeat doesn't send the messages until the queue is totally filled. The log on the new server cannot send too few logs since i view the log file directly and there is a lots of new log data there. > [@Mario\_Castro](#): > > Can you describe the rate and load of your events? How many events per second? How big each event is in KB? Currently, about 10-15 minute the log got ingest for about 4000 event each time. I'm not sure about the size of the event but back on the old server it got ingest without any problems. > [@Mario\_Castro](#): > > Check also that you have activated the expected log level in your new server. It might be the case that you were logging DEBUG logs before and now your are using INFO 😅 The log i've been getting are log from f5 waf that we got from rsyslog server. ---
**Author:** ![Mario\_Castro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mario_castro/32/35107_2.png) [@Mario\_Castro](https://discuss.elastic.co/u/Mario_Castro)\ **Post date:** [December 7, 2021, 4:40pm UTC](https://discuss.elastic.co/t/filebeat-push-event-in-very-slow-rate/291013/4 "2021-12-07T16:40:50Z")
Try removing the folder with the data files to see if that helps [Directory layout | Filebeat Reference [7.16] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/current/directory-layout.html) But if you are using the same filebeat, the same Elasticsearch and the **only** thing that changed was your sever, I strongly suggest to look into the new server or your new network topology, firewall or permissions there for potential issues. ---
**Author:** ![lusynda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusynda/32/53557_2.png) [@lusynda](https://discuss.elastic.co/u/lusynda)\ **Post date:** [December 8, 2021, 3:42am UTC](https://discuss.elastic.co/t/filebeat-push-event-in-very-slow-rate/291013/5 "2021-12-08T03:42:22Z")
Is there a possibility that due to the size of each event is too large for filebeat to process it. i have just check the log, each event is about 4kb each. Is that the problems because i thinks that is a bit too large and there are about 1000000 event each day. ---
**Author:** ![lusynda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lusynda/32/53557_2.png) [@lusynda](https://discuss.elastic.co/u/lusynda)\ **Post date:** [December 9, 2021, 7:01am UTC](https://discuss.elastic.co/t/filebeat-push-event-in-very-slow-rate/291013/6 "2021-12-09T07:01:51Z")
Somehow i fix the problems, I change the queue type on the filebeat from mem to disk and then everything works fine. i don't really know how and why it works but at least it works. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 6, 2022, 9:02am UTC](https://discuss.elastic.co/t/filebeat-push-event-in-very-slow-rate/291013/7 "2022-01-06T09:02:12Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.