# Logstash has a lag in pushing events to Elastic

**URL:** <https://discuss.elastic.co/t/logstash-has-a-lag-in-pushing-events-to-elastic/374798>\
**Category:** Logstash\
**Created:** [February 20, 2025, 8:54am UTC](https://discuss.elastic.co/t/logstash-has-a-lag-in-pushing-events-to-elastic/374798 "2025-02-20T08:54:44Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![devops\_training](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/devops_training/32/131845_2.png) [@devops\_training](https://discuss.elastic.co/u/devops_training)\
**Post date:** [February 20, 2025, 8:54am UTC](https://discuss.elastic.co/t/logstash-has-a-lag-in-pushing-events-to-elastic/374798/1 "2025-02-20T08:54:44Z")

</div>

logstash config

```auto
input {
  file {
    path => "C:/Program Files (x86)/db/Server/logs/oracle.log"
    type => "localhost_access_log"
    start_position => "beginning"
	ignore_older => 86400 # ignore files older than 24 hours
    close_older => 86400 # free the resources
	sincedb_path => "C:\setup\tools\logstash\logstash-8.17.0\logs\oracle_log.sincedb"
  }
  file {
    path => "C:/Program Files (x86)/db/Server/logs/system.log"
    type => "system"
    codec => multiline {
      pattern => "^%{TIMESTAMP_ISO8601}"
      negate => true
      what => "previous"
      charset => "ISO-8859-1"
    }
    start_position => "beginning"
	ignore_older => 86400 # ignore files older than 24 hours
    close_older => 86400 # free the resources
	sincedb_path => "C:\setup\tools\logstash\logstash-8.17.0\logs\system.sincedb"
  }
}

filter {
   if [type] == "localhost_access_log" {
    grok {
      match => {"message" => "\[%{HTTPDATE:logtimeStamp}\] %{IP:hostip} %{URIPROTO:method} %{URIPATH:post-data} (?:%{NOTSPACE:queryparam}|-) %{NUMBER:useragent} %{NUMBER:responsestatus} \[%{GREEDYDATA:message}\] - %{NUMBER:time-taken:int}"}
      overwrite => ["message"]
    }
    mutate {
      remove_field => ["logtimeStamp"]
    }
  }
  if [type] == "system" {
    mutate {
      gsub => [
        "message", "\[\] ", " ",
        "message", "\- ", " ",
        "message", "\s+", " "
      ]
    }
    mutate {
      strip => ["message"]
    }
    grok {
      match => {"message" => ["%{TIMESTAMP_ISO8601:logtimeStamp} %{WORD:loglevel} \[%{USERNAME:httpcall}] %{USERNAME:dbName} %{USERNAME:tenantGuid} %{INT:tenantId} %{INT:userId} %{USERNAME:sessionID} %{GREEDYDATA:message}",
                              "%{TIMESTAMP_ISO8601:logtimeStamp} %{WORD:loglevel} %{GREEDYDATA:message}" ]}
      overwrite => ["message"]
    }
    mutate {
      remove_field => ["logtimeStamp"]
    }
  }
 
}
output {
    elastic {
      ecs_compatibility => disabled
      hosts => ["https://${ ********** }:443"]
      ssl => true
      index => "${NODE_ROLE}-%{+YYYY.MM.dd}"
	  document_id => "%{fingerprint}"
    }
  }
}

```

While running logstash I have identified the issue with lag.

log message: Why it is sending the log 5 hours late?  
Issue is happening with windows instance. It has hava heap memory set to 4G.

```auto
[2025-02-19T10:22:58,795][DEBUG][logstash.filters.grok][main][40e2ddf7bf9bfcdd3a98f5f146527cc76da83c1955901bbe12a47cbd6a078fa9] Event now: {:event=>{"loglevel"=>"DEBUG", "host"=>{"name"=>"web-server"}, "@timestamp"=>2025-02-19T10:22:58.743645800Z, "nodeRole"=>"web-server-1", "event"=>{"original"=>"2025-02-19T05:13:17,823 DEBUG [scheduler-TaskQueueEngine-thread-231] jdbc.sqlonly - select TenantId, Name, Uid, DefaultLocale, PartnerUID, Guid from [OracleTenant] where [OracleTenant].[TenantId]=(int)1\r"}, "message"=>"[scheduler-TaskQueueEngine-thread-231] jdbc.sqlonly select TenantId, Name, Uid, DefaultLocale, PartnerUID, Guid from [OracleTenant] where [OracleTenant].[TenantId]=(int)1", "log"=>{"file"=>{"path"=>"C:/Program Files (x86)/db/Server/logs/system.log"}}

```

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [February 20, 2025, 2:35pm UTC](https://discuss.elastic.co/t/logstash-has-a-lag-in-pushing-events-to-elastic/374798/2 "2025-02-20T14:35:19Z")

</div>

Maybe your LS is in different zone or you are not converting the logtimeStamp field to date and also you have deleted it.

---

<div class="post-metadata">

**Author:** ![devops\_training](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/devops_training/32/131845_2.png) [@devops\_training](https://discuss.elastic.co/u/devops_training)\
**Post date:** [February 20, 2025, 2:40pm UTC](https://discuss.elastic.co/t/logstash-has-a-lag-in-pushing-events-to-elastic/374798/3 "2025-02-20T14:40:52Z")

</div>

I see this in sincedb. This is file which is been sent again.

```auto
C:\setup\tools\logstash\logstash-8.17.0\logs\system.sincedb

3336151636-287522-262144 0 0 369559 1739945755.935 C:/Program Files (x86)/db/Server/logs/system.log
3336151636-375609-2162688 0 0 858170 1740061908.87 C:/Program Files (x86)/db/Server/logs/system.log

```

There was a file system.log that got renamed to system\_2025\_19\_2.log

how to fix this issue with sincedb?

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [February 20, 2025, 2:45pm UTC](https://discuss.elastic.co/t/logstash-has-a-lag-in-pushing-events-to-elastic/374798/4 "2025-02-20T14:45:03Z")

</div>

> [@devops\_training](#):
>
> log message: Why it is sending the log 5 hours late?

How and where did you identify this delay?

---

<div class="post-metadata">

**Author:** ![devops\_training](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/devops_training/32/131845_2.png) [@devops\_training](https://discuss.elastic.co/u/devops_training)\
**Post date:** [February 20, 2025, 2:47pm UTC](https://discuss.elastic.co/t/logstash-has-a-lag-in-pushing-events-to-elastic/374798/5 "2025-02-20T14:47:12Z")

</div>

```auto
[2025-02-19T10:22:58,795][DEBUG][logstash.filters.grok][main][40e2ddf7bf9bfcdd3a98f5f146527cc76da83c1955901bbe12a47cbd6a078fa9] Event now: {:event=>{"loglevel"=>"DEBUG", "host"=>{"name"=>"web-server"}, "@timestamp"=>2025-02-19T10:22:58.743645800Z, "nodeRole"=>"web-server-1", "event"=>{"original"=>"2025-02-19T05:13:17,823 DEBUG [scheduler-TaskQueueEngine-thread-231] jdbc.sqlonly - select TenantId, Name, Uid, DefaultLocale, PartnerUID, Guid from [OracleTenant] where [OracleTenant].[TenantId]=(int)1\r"}, "message"=>"[scheduler-TaskQueueEngine-thread-231] jdbc.sqlonly select TenantId, Name, Uid, DefaultLocale, PartnerUID, Guid from [OracleTenant] where [OracleTenant].[TenantId]=(int)1", "log"=>{"file"=>{"path"=>"C:/Program Files (x86)/db/Server/logs/system.log"}}

```

the above is the debug message from logstash

logstash process time 2025-02-19T10:22:58  
event which it picked from the above log file 2025-02-19T05:13:17,823

```auto
2025-02-19T05:13:17,823 DEBUG [scheduler-TaskQueueEngine-thread-231] jdbc.sqlonly - select TenantId, Name, Uid, DefaultLocale, PartnerUID, Guid from [OracleTenant] where [OracleTenant].[TenantId]=(int)1\r"}, "message"=>"[scheduler-TaskQueueEngine-thread-231] jdbc.sqlonly select TenantId, Name, Uid, DefaultLocale, PartnerUID, Guid from [OracleTenant] where [OracleTenant].[TenantId]=(int)1

```

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [February 20, 2025, 2:49pm UTC](https://discuss.elastic.co/t/logstash-has-a-lag-in-pushing-events-to-elastic/374798/6 "2025-02-20T14:49:47Z")

</div>

Yeah, but what is the timezone of the date in the file?

There is no information in the timestamp, so it is not clear that this time is in UTC.

You need to confirm what is the timezone that the application that is writing those logs is using.

---

<div class="post-metadata">

**Author:** ![devops\_training](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/devops_training/32/131845_2.png) [@devops\_training](https://discuss.elastic.co/u/devops_training)\
**Post date:** [February 20, 2025, 2:50pm UTC](https://discuss.elastic.co/t/logstash-has-a-lag-in-pushing-events-to-elastic/374798/7 "2025-02-20T14:50:32Z")

</div>

@leandrojmp All are in UTC.

please advise.

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [February 20, 2025, 3:04pm UTC](https://discuss.elastic.co/t/logstash-has-a-lag-in-pushing-events-to-elastic/374798/8 "2025-02-20T15:04:14Z")

</div>

Can you provide a line from the oracle.log or system.log?

---

<div class="post-metadata">

**Author:** ![devops\_training](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/devops_training/32/131845_2.png) [@devops\_training](https://discuss.elastic.co/u/devops_training)\
**Post date:** [February 20, 2025, 3:09pm UTC](https://discuss.elastic.co/t/logstash-has-a-lag-in-pushing-events-to-elastic/374798/9 "2025-02-20T15:09:41Z")

</div>

@Rios @leandrojmp please find the logs from the original log file.

```auto
system.log
[20/Feb/2025:00:01:47 +0000] 10.3.185.541 GET /Core/ready-status HTTP/1.1 18 200 [https-jsse-nio-443-exec-16] [-] - 0ms

oracle.log

2025-02-20T15:07:49,058 INFO [https-jsse-nio-443-exec-4] servlet.ReadyStatusServlet - doGet call for ReadyStatusServlet. Calling doPost.
2025-02-20T15:07:49,480 INFO [https-jsse-nio-443-exec-5] servlet.ReadyStatusServlet - doGet call for ReadyStatusServlet. Calling doPost.
2025-02-20T15:07:50,101 INFO [https-jsse-nio-443-exec-2] servlet.ReadyStatusServlet - doGet call for ReadyStatusServlet. Calling doPost.

```

---

<div class="post-metadata">

**Author:** ![devops\_training](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/devops_training/32/131845_2.png) [@devops\_training](https://discuss.elastic.co/u/devops_training)\
**Post date:** [February 20, 2025, 4:30pm UTC](https://discuss.elastic.co/t/logstash-has-a-lag-in-pushing-events-to-elastic/374798/10 "2025-02-20T16:30:07Z")

</div>

Any thoughts why is this happening?

---

<div class="post-metadata">

**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\
**Post date:** [February 20, 2025, 7:20pm UTC](https://discuss.elastic.co/t/logstash-has-a-lag-in-pushing-events-to-elastic/374798/11 "2025-02-20T19:20:24Z")

</div>

are you sitting in Eastern time zone? which has five hour different from UTC

---

<div class="post-metadata">

**Author:** ![devops\_training](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/devops_training/32/131845_2.png) [@devops\_training](https://discuss.elastic.co/u/devops_training)\
**Post date:** [February 21, 2025, 3:58am UTC](https://discuss.elastic.co/t/logstash-has-a-lag-in-pushing-events-to-elastic/374798/12 "2025-02-21T03:58:29Z")

</div>

yes we are using EST.

If you see logstash is reading older file. That is the issue. Sharing the screenshot from kibana.

![image](https://us1.discourse-cdn.com/elastic/optimized/3X/5/2/52a508a0b1295d73b3c6ea5d3d1e14a867b163f8_2_690x30.png)

---

<div class="post-metadata">

**Author:** ![elasticforme](https://avatars.discourse-cdn.com/v4/letter/e/f05b48/32.png) [@elasticforme](https://discuss.elastic.co/u/elasticforme)\
**Post date:** [February 21, 2025, 3:08pm UTC](https://discuss.elastic.co/t/logstash-has-a-lag-in-pushing-events-to-elastic/374798/13 "2025-02-21T15:08:14Z")

</div>

this is I suspect. when you read a time from logfile using logstash it does not have timezone. and hence elastic is converting ( moving 5 hour) and putting it as ust. now when you see that on kibana it display wrong time.

try this

```auto
date { match => ["time_field_name", "dd-MM-yy HH:mm:ss", "dd-MMM-yy HH:mm:ss", "ISO8601"] --> make sure this format matches your date format
        timezone => "EST" --> make sure this is correct 
        target => "time_field_name"
}

```

if this does not work try  
timezone = "Etc/UTC"

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [February 21, 2025, 8:33pm UTC](https://discuss.elastic.co/t/logstash-has-a-lag-in-pushing-events-to-elastic/374798/14 "2025-02-21T20:33:24Z")

</div>

Your grok pattern should be changed.

$ systemlog

```auto
input {
  generator { 
    message => ['[20/Feb/2025:00:01:47 +0000] 10.3.185.541 GET /Core/ready-status HTTP/1.1 18 200 [https-jsse-nio-443-exec-16] [-] - 0ms' ]
    count => 1 } 
}
output {
 stdout {codec => rubydebug}
}
filter {
  grok { match => { "message" => "\[%{HAPROXYDATE:[@metadata][timestamp]}\] %{IPORHOST:[source][address]} %{WORD:[http][request][method]} %{URIPATH:[url][uripath]} HTTP/%{NUMBER:[http][version]} %{INT:[http][response][body][bytes]:int} %{INT:[http][response][status_code]:int} \[%{SYSLOG5424PRINTASCII:[process][name]}\] \[%{NOTSPACE:something1}\] %{NOTSPACE:something2} %{INT:total_time:int}(?<unit>[A-Za-z]+)" } }

  date { match => ["[@metadata][timestamp]", "dd/MMM/yyyy:HH:mm:ss Z"] 
     #timezone => "Asia/Dubai" 
  }
}

```

Output:

```auto
{
    "something2" => "-",
      "@version" => "1",
           "url" => {
        "uripath" => "/Core/ready-status"
    },
          "http" => {
        "response" => {
                   "body" => {
                "bytes" => 18
            },
            "status_code" => 200
        },
         "version" => "1.1",
         "request" => {
            "method" => "GET"
        }
    },
    "@timestamp" => 2025-02-20T00:01:47.000Z,
    "total_time" => 0,
        "source" => {
        "address" => "10.3.185.541"
    },
          "unit" => "ms",
       "message" => "[20/Feb/2025:00:01:47 +0000] 10.3.185.541 GET /Core/ready-status HTTP/1.1 18 200 [https-jsse-nio-443-exec-16] [-] - 0ms",
    "something1" => "-",
       "process" => {
        "name" => "https-jsse-nio-443-exec-16"
    }
}

```

Note: Rename fields something1 and something2 or any other field

$ oracle.log

```auto
input {
  generator { 
    message => "2025-02-20T15:07:49,058 INFO [https-jsse-nio-443-exec-4] servlet.ReadyStatusServlet - doGet call for ReadyStatusServlet. Calling doPost."
    count => 1 } 
}
output {
 stdout {codec => rubydebug{ metadata => true}}
}
filter {
  grok { match => { "message" => "%{TIMESTAMP_ISO8601:[@metadata][timestamp]} %{LOGLEVEL:[log][level]}%{SPACE}\[%{NOTSPACE:[process][name]}\] %{NOTSPACE:method}%{SPACE}-%{SPACE}%{GREEDYDATA:msg}" } }

  date { match => ["[@metadata][timestamp]", "ISO8601"] 
     timezone => "Asia/Dubai" 
  }

  mutate { remove_field => ["event", "host"] }

}

```

Output:

```auto
{
           "msg" => "doGet call for ReadyStatusServlet. Calling doPost.",
       "process" => {
        "name" => "https-jsse-nio-443-exec-4"
    },
    "@timestamp" => 2025-02-20T14:07:49.058Z,
        "method" => "servlet.ReadyStatusServlet",
           "log" => {
        "level" => "INFO"
    },
     "@metadata" => {
        "timestamp" => "2025-02-20T15:07:49,058"
    },
      "@version" => "1",
       "message" => "2025-02-20T15:07:49,058 INFO [https-jsse-nio-443-exec-4] servlet.ReadyStatusServlet - doGet call for ReadyStatusServlet. Calling doPost."
}

```

---

<div class="post-metadata">

**Author:** ![devops\_training](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/devops_training/32/131845_2.png) [@devops\_training](https://discuss.elastic.co/u/devops_training)\
**Post date:** [February 24, 2025, 5:14am UTC](https://discuss.elastic.co/t/logstash-has-a-lag-in-pushing-events-to-elastic/374798/15 "2025-02-24T05:14:52Z")

</div>

@Rios @elasticforme @leandrojmp Thank you all for your time.

The above solution worked.

---

<div class="post-metadata">

**Author:** ![devops\_training](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/devops_training/32/131845_2.png) [@devops\_training](https://discuss.elastic.co/u/devops_training)\
**Post date:** [February 26, 2025, 2:57am UTC](https://discuss.elastic.co/t/logstash-has-a-lag-in-pushing-events-to-elastic/374798/16 "2025-02-26T02:57:03Z")

</div>

@Rios @leandrojmp @elasticforme

Issue still exists. Sharing the screen shot. The problem is why did it pick the event which is older than one day.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/f/1/f11e7e0ea06c69a9d780e744dc97f5bda10c5f3e.png)  
timestamp in the below Image is extracted from message.

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/1/6177ff527953d9d50f5283383f5a208cb8af1a6c.png)

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [February 26, 2025, 2:11pm UTC](https://discuss.elastic.co/t/logstash-has-a-lag-in-pushing-events-to-elastic/374798/17 "2025-02-26T14:11:56Z")

</div>

There is no the lag. You have 2 datetime values, which cause that the first overwrites the second value.  
There are options:

1. have two timestamp fields:

- @timestamp e.g. `type= localhost_access_log`
- @timestamp2 e.g. `type= system` you can set in the target. You should add in date:  
`target => "@timestamp2"` # default value is `@timestamp`

1. have two pipelines, which means two separated .conf files and of course 2 @timestamp values

---

<div class="post-metadata">

**Author:** ![devops\_training](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/devops_training/32/131845_2.png) [@devops\_training](https://discuss.elastic.co/u/devops_training)\
**Post date:** [March 5, 2025, 5:51am UTC](https://discuss.elastic.co/t/logstash-has-a-lag-in-pushing-events-to-elastic/374798/18 "2025-03-05T05:51:29Z")

</div>

@Rios Once again thank you for your time in helping me understand.

---

<div class="post-metadata">

**Author:** ![Rios](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rios/32/95745_2.png) [@Rios](https://discuss.elastic.co/u/Rios)\
**Post date:** [March 5, 2025, 7:28am UTC](https://discuss.elastic.co/t/logstash-has-a-lag-in-pushing-events-to-elastic/374798/19 "2025-03-05T07:28:14Z")

</div>

You are welcome.

Long live the King and the Elastic team.