# Logstash/Filebeat lag issue - Logs delayed by hours **URL:** \ **Category:** Beats\ **Tags:** filebeat\ **Created:** [February 18, 2026, 5:00pm UTC](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101 "2026-02-18T17:00:34Z")\ **Posts on this page:** 20\ **Page:** 2
**Author:** ![RainTown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raintown/32/140206_2.png) [@RainTown](https://discuss.elastic.co/u/RainTown)\ **Post date:** [February 25, 2026, 12:02am UTC](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101/21 "2026-02-25T00:02:51Z")
Just to check I’m understanding , the Kibana screenshot shows 2 Elasticsearch documents with the same @tinestamp value. They have slightly (45 seconds) later ls\_timestamp values, which were generated as they passed through logstash. And a message field that looks to be constructed from some date/time value. So where _precisely_ is the @timestamp coming from? Is filebeat adding it? And where _precisely_ is the date/time value in message coming from ? I once had an issue where I was confused with sequencing of timestamps. And it turned out the timestamp in the application log file was the time the random mobile device thought it was, and consumer devices are just time and dare screwed in a myriad of ways!! That meant that “timestamp” had to be (broadly) ignored. ---
**Author:** ![Cesar\_Mejia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cesar_mejia/32/141794_2.png) [@Cesar\_Mejia](https://discuss.elastic.co/u/Cesar_Mejia)\ **Post date:** [February 25, 2026, 2:21pm UTC](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101/22 "2026-02-25T14:21:30Z")
You are spot on. The @timestamp is coming from Filebeat, it records the exact moment Filebeat reads and harvests that specific log line from the file. The date/time in the message field is coming directly from the application itself (our Tomcat and SMPP apps) at the exact millisecond the event occurred and was written to the local log file, the delay you see between the message time, the @timestamp (Filebeat read time), and the ls\_timestamp (Logstash processing time) is simply the visual evidence of the processing queue backlog and transit time we were experiencing ---
**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\ **Post date:** [February 25, 2026, 2:46pm UTC](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101/23 "2026-02-25T14:46:22Z")
> [@Cesar\_Mejia](#): > > the @timestamp (Filebeat read time), and the ls\_timestamp (Logstash processing time) is simply the visual evidence of the processing queue backlog and transit time we were experiencing The difference between the `@timestamp` coming from Filebeat and the `ls_timestamp` generated are small, just some seconds. In the screenshot you shared you have 2 events, in both the `@timestamp` and `ls_timestamp` are pretty close, but the time in the `message` field in one of them the timestamp is something like `2026-02-24 10:45:45`, which is also pretty close, but in the other you have `2026-02-24 06:04:08` which has a 4 hour difference from the other times. I think this suggests that the backlog is on Filebeat side, not on Logstash, as you mentioned you logs rotate pretty fast, this can cause this backlog if I'm not wrong. Can you share a print with the same 2 events again, but include the filebeat offset and path fields? I would make a change in your Filebeat output, since you have only one Logstash, remove the other ports, use just one port and increase the number of workers, use something like 8 or 12. ---
**Author:** ![RainTown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raintown/32/140206_2.png) [@RainTown](https://discuss.elastic.co/u/RainTown)\ **Post date:** [February 25, 2026, 6:26pm UTC](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101/24 "2026-02-25T18:26:18Z")
Well, as well as all that, and I also though a delay of 45 seconds wasn’t particularly remarkable, where I was going is that you don’t have evidence of precisely when a specific log entry is written to the log file. And the 2 _application_ timestamps for those 2 entries was just a bit weird. One a few seconds before ls\_timestamp and one ages before. ---
**Author:** ![amitkumar.gupta](https://avatars.discourse-cdn.com/v4/letter/a/ee7513/32.png) [@amitkumar.gupta](https://discuss.elastic.co/u/amitkumar.gupta)\ **Post date:** [February 26, 2026, 11:30am UTC](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101/25 "2026-02-26T11:30:22Z")
> [@Cesar\_Mejia](#): > > Logs are queuing up at the Filebeat level. When I check Kibana, I see "recent" arrival times (@timestamptimestamp is current), but the actual log content (message) shows timestamps from 2+ hours ago. It seems like the pipeline "clogs" and then releases, causing variable delays is there any solution identified for this issue? We are facing the same issue and not sure where to check. ---
**Author:** ![amitkumar.gupta](https://avatars.discourse-cdn.com/v4/letter/a/ee7513/32.png) [@amitkumar.gupta](https://discuss.elastic.co/u/amitkumar.gupta)\ **Post date:** [February 26, 2026, 11:36am UTC](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101/26 "2026-02-26T11:36:25Z")
Hi @Cesar_Mejia - Could you please confirm if your Filebeat/ logstash are on same data center or they are on different and far data cenetr? where logstash is saving the logs? do you think this may be back pressure issue because of network latency? ---
**Author:** ![Cesar\_Mejia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cesar_mejia/32/141794_2.png) [@Cesar\_Mejia](https://discuss.elastic.co/u/Cesar_Mejia)\ **Post date:** [February 27, 2026, 3:34pm UTC](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101/27 "2026-02-27T15:34:25Z")
I disabled pipelining and increased the pool to 8 workers. I've noticed the bottleneck consistently occurs with larger files. While most files are only a few KB or MB, others reach up to 100MB, which seems to be causing the slowdown. ---
**Author:** ![Cesar\_Mejia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cesar_mejia/32/141794_2.png) [@Cesar\_Mejia](https://discuss.elastic.co/u/Cesar_Mejia)\ **Post date:** [February 27, 2026, 3:46pm UTC](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101/28 "2026-02-27T15:46:48Z")
They’re in the same data center, and Logstash is saving to a volume group. and for the network latency I've noticed a pattern where logs aren't missing, but they are arriving with significant latency because Filebeat is struggling to connect to Logstash, even with a timeout of 10000 ![image](https://us1.discourse-cdn.com/elastic/optimized/3X/6/f/6fbe6400e1679984453d6c9ce7a278842be51191_2_690x39.png) ---
**Author:** ![Cesar\_Mejia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cesar_mejia/32/141794_2.png) [@Cesar\_Mejia](https://discuss.elastic.co/u/Cesar_Mejia)\ **Post date:** [February 27, 2026, 3:57pm UTC](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101/29 "2026-02-27T15:57:40Z")
Sorry it took me a bit. Here’s a capture of another similar case ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/2/3241cbe887841f0ff81c71968775466da6acac7a.png) ```bash 22M /var/log/smpp89_20260227.log 326M /var/log/smpp72_20260227.log ``` ---
**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\ **Post date:** [February 27, 2026, 5:39pm UTC](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101/30 "2026-02-27T17:39:46Z")
Something is not OK, maybe LS cannot process and respond fast enough. Have you checked LS CPU utilization? Have you try to do output FB to a local file or directly to ES, just for a test? Actually your batch is quite big size: `pipeline.batch.size: 2048`, set it to default value. `pipeline.workers: 12` - no need to set, _This defaults to the number of the host's CPU cores_. Add comments rubydebug to increase performances. ```auto #stdout { # codec => rubydebug { # metadata => true # } # } ``` ---
**Author:** ![Cesar\_Mejia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cesar_mejia/32/141794_2.png) [@Cesar\_Mejia](https://discuss.elastic.co/u/Cesar_Mejia)\ **Post date:** [February 27, 2026, 5:54pm UTC](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101/31 "2026-02-27T17:54:23Z")
Hi Rios!! Just a quick note: Logstash and Elasticsearch share the same server. I limit the workers manually so Logstash doesn't grab all the cores and starve ES. Also, the server CPU is only at 20-30%, so Logstash isn't using its full capacity. I noticed the backlog issue is specifically happening with the massive files (\>200MB). And yes, I already commented out stdout and the rubydebug ---
**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\ **Post date:** [February 27, 2026, 8:43pm UTC](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101/32 "2026-02-27T20:43:18Z")
Few more things: - You have too many GREEDYDATA in the middle message, replace with DATA, should be faster - Add `id => "grok"` in grok and `id => "elasticsearch"` in the output. You can also add in beats. Check LS statistics with url [http://localhost:9600/\_node/stats/pipelines?pretty](http://localhost:9600/_node/stats/pipelines?pretty) to see which plugin consume the most time - Use dissect instead grok, where is possible, probably with few more IFs - Check LS and ES logs, maybe you have delay or issue somewhere - Try to test FB-\>LS connectivity without anything in the filter section and large files, just to see is there any network or settings issues. You can use LS output in a file. ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [March 1, 2026, 3:58pm UTC](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101/33 "2026-03-01T15:58:34Z")
> [@Cesar\_Mejia](#): > > Just a quick note: Logstash and Elasticsearch share the same server. This is not ideal... And though you may think they are not competing for resources they can be. ---
**Author:** ![RainTown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raintown/32/140206_2.png) [@RainTown](https://discuss.elastic.co/u/RainTown)\ **Post date:** [March 2, 2026, 3:56am UTC](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101/34 "2026-03-02T03:56:47Z")
> [@Cesar\_Mejia](#): > > They’re in the same data center, and Logstash is saving to a volume group. and for the network latency I've noticed a pattern where logs aren't missing, but they are arriving with significant latency because Filebeat is struggling to connect to Logstash, even with a timeout of 10000 > > ![image](https://us1.discourse-cdn.com/elastic/optimized/3X/6/f/6fbe6400e1679984453d6c9ce7a278842be51191_2_690x39.png) Maybe obvious to others, but not to me, what is this image showing. What is the y-axis ? Even, assuming the x-axis is a timestamp of some sort, since the thread involves a few, which one is it? > [@Cesar\_Mejia](#): > > Filebeat is struggling to connect to Logstash does not seem to sit with: > [@Cesar\_Mejia](#): > > Just a quick note: Logstash and Elasticsearch share the same server. I limit the workers manually so Logstash doesn't grab all the cores and starve ES. Also, the server CPU is only at 20-30%, so Logstash isn't using its full capacity. whats the specific evidence that "Filebeat is struggling to connect to Logstash" ? > [@Cesar\_Mejia](#): > > I disabled pipelining and increased the pool to 8 workers. I've noticed the bottleneck consistently occurs with larger files. While most files are only a few KB or MB, others reach up to 100MB, which seems to be causing the slowdown. What about rotating the log files so that they don't get so large? Though did you not say earlier in the thread the log files were rotating rapidly? Is that rotating rapidly (specifically, how rapidly?) AND quickly reaching sizes of 100MB? ---
**Author:** ![Cesar\_Mejia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cesar_mejia/32/141794_2.png) [@Cesar\_Mejia](https://discuss.elastic.co/u/Cesar_Mejia)\ **Post date:** [March 2, 2026, 3:53pm UTC](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101/35 "2026-03-02T15:53:22Z")
> [@RainTown](#): > > Maybe obvious to others, but not to me, what is this image showing. What is the y-axis ? Even, assuming the x-axis is a timestamp of some sort, since the thread involves a few, which one is it? is the logs amount > [@RainTown](#): > > whats the specific evidence that "Filebeat is struggling to connect to Logstash" ? filebeat log shows this, ping and telnet works ```bash 2026-03-02T09:49:45.335-0600 ERROR [logstash] logstash/async.go:280 Failed to publish events caused by: write tcp IP:PORT->LOGSTASHIP:PORT: write: connection reset by peer 2026-03-02T09:49:45.335-0600 INFO [publisher] pipeline/retry.go:219 retryer: send unwait signal to consumer 2026-03-02T09:49:45.335-0600 INFO [publisher] pipeline/retry.go:223 done 2026-03-02T09:49:45.392-0600 ERROR [logstash] logstash/async.go:280 Failed to publish events caused by: client is not connected 2026-03-02T09:49:45.392-0600 INFO [publisher] pipeline/retry.go:219 retryer: send unwait signal to consumer 2026-03-02T09:49:45.392-0600 INFO [publisher] pipeline/retry.go:223 done 2026-03-02T09:49:45.449-0600 ERROR [logstash] logstash/async.go:280 Failed to publish events caused by: client is not connected 2026-03-02T09:49:45.449-0600 INFO [publisher] pipeline/retry.go:219 retryer: send unwait signal to consumer 2026-03-02T09:49:45.449-0600 INFO [publisher] pipeline/retry.go:223 done 2026-03-02T09:49:46.505-0600 ERROR [publisher_pipeline_output] pipeline/output.go:180 failed to publish events: client is not connected 2026-03-02T09:49:46.646-0600 ERROR [publisher_pipeline_output] pipeline/output.go:180 failed to publish events: write tcp IP:PORT->LOGSTASHIP:PORT: write: connection reset by peer 2026-03-02T09:49:46.896-0600 ERROR [publisher_pipeline_output] pipeline/output.go:180 failed to publish events: client is not connected ``` > [@RainTown](#): > > What about rotating the log files so that they don't get so large? Though did you not say earlier in the thread the log files were rotating rapidly? Is that rotating rapidly (specifically, how rapidly?) AND quickly reaching sizes of 100MB? it rotates every 300MB ---
**Author:** ![RainTown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raintown/32/140206_2.png) [@RainTown](https://discuss.elastic.co/u/RainTown)\ **Post date:** [March 2, 2026, 6:30pm UTC](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101/36 "2026-03-02T18:30:49Z")
> [@Cesar\_Mejia](#): > > ![image](https://us1.discourse-cdn.com/elastic/optimized/3X/6/f/6fbe6400e1679984453d6c9ce7a278842be51191_2_690x39.png) > [@Cesar\_Mejia](#): > > is the logs amount But what does it tell you/us? if the log volume (from applications) varies over time, so will a view like that in kibana. That's normal, that bare graph shows nothing really. Remember we have close to no idea on your use case, we know only what you tell us. To see an actual issue, a discrepancy, you should try to share graphs that highlight/illustrate the issue. e.g. 2 plots of log volume over time, one using say the application timestamp, and the other using say `ls_timestamp`. Generally, when things are working fine, these will be pretty much identical shapes. In your case they might be quite different. > [@Cesar\_Mejia](#): > > filebeat log shows this, ping and telnet works telnet is a bit of a blast from the past 🙂 > [@Cesar\_Mejia](#): > > `write tcp IP:PORT->LOGSTASHIP:PORT: write: connection reset by peer` Did you try capturing/analysing the traffic with wireshark/tpcdump/similar? Last time I looked at something like this logstash was sending a RSET, but I would be a bit surprised if logstash would close a busy, active, non-idle TCP connection without logging somewhere why it did so. Are logstash and elasticsearch sharing the same network interface(s)? > [@Cesar\_Mejia](#): > > Just a quick note: Logstash and Elasticsearch share the same server. Did you try logstash running elsewhere? ---
**Author:** ![RainTown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raintown/32/140206_2.png) [@RainTown](https://discuss.elastic.co/u/RainTown)\ **Post date:** [March 4, 2026, 1:26pm UTC](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101/37 "2026-03-04T13:26:25Z")
@Cesar_Mejia this was quite an interesting puzzle/thread - any progress/update ? ---
**Author:** ![Cesar\_Mejia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cesar_mejia/32/141794_2.png) [@Cesar\_Mejia](https://discuss.elastic.co/u/Cesar_Mejia)\ **Post date:** [March 4, 2026, 4:52pm UTC](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101/38 "2026-03-04T16:52:55Z")
> [@RainTown](#): > > @Cesar_Mejia this was quite an interesting puzzle/thread - any progress/update ? Yep, thanks for the help, i don't have a graphic to show you, but every day from 9 AM to 12 PM, logs start to disappear. Talking to some coworkers, they say this time window has a lot of traffic and is the period of peak demand during the day. I believe this happens because Elastic is stalling or lacks sufficient resources to handle the load; I am concluding this based on the next log: ```auto [2026-03-04T10:37:18,981][INFO][logstash.outputs.elasticsearch][main]Retrying failed action {:status=>429, :action=>["index", {:_id=>nil, :_index=>"index_reports", :routing=>nil}, {"host"=>{"name"=>"machine01"}, "event"=>{"original"=>"20260303082500: SOME INFORMATION"}, error=>{"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of primary operation [coordinating_and_primary_bytes=50280130, replica_bytes=862626885, all_bytes=912907015, primary_operation_bytes=180245, max_primary_bytes=825019596]"}} ``` ---
**Author:** ![RainTown](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/raintown/32/140206_2.png) [@RainTown](https://discuss.elastic.co/u/RainTown)\ **Post date:** [March 4, 2026, 5:33pm UTC](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101/39 "2026-03-04T17:33:40Z")
Yes, concerning. I note your error shows a timestamp of `2026-03-04T10:37:18` but has `"event"=>{"original"=>"20260303082500` which looks a bit like 2026-03-03T08:25:00, i.e. almost a full day before? Assuming you have enabled monitoring within elasticsearch, you should be able to see clear indications of higher load, and how load varies over time generally, under Stack Monitoring in kibana. Can you also detail the storage you have attached to the server, i.e. specifically what type is it? Answers like "we have a RAID5 volume composed of 8 x 2TB locally attached SSD disks", Or non-RAID NVMe storage. Or older style spinning/HDD disk(s). and an idea of per/second (or per-anything) data volume. ---
**Author:** ![Cesar\_Mejia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/cesar_mejia/32/141794_2.png) [@Cesar\_Mejia](https://discuss.elastic.co/u/Cesar_Mejia)\ **Post date:** [March 4, 2026, 6:10pm UTC](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101/40 "2026-03-04T18:10:59Z")
i have three elastic instances to send data from logstash with loadbalancing activated: ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/8/7898c4b4ccd562d02b046ef62b5c09592645c7a2.png) ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/9/89aadbe401b9f8e076417d89d4719f16828d69fa.png) ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/0/c0a4935041843933fbaf9ea82e00d13aea4309ed.png) We have a locally attached hardware RAID volume (exact RAID level and physical disk count abstracted from the OS by a Dell PERC H730P Mini controller) presenting a 1.8TB virtual disk composed of traditional HDDs: ```bash lsblk -d -o name,rota NAME ROTA sda 1 cat /sys/block/sda/device/model PERC H730P Mini cat /sys/block/sda/device/vendor DELL ``` [Previous page](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101.md?page=1) [Next page](https://discuss.elastic.co/t/logstash-filebeat-lag-issue-logs-delayed-by-hours/385101.md?page=3)