# Lag in logs

**URL:** https://discuss.elastic.co/t/lag-in-logs/342233
**Category:** Logstash
**Created:** [September 4, 2023, 9:53am UTC](https://discuss.elastic.co/t/lag-in-logs/342233 "2023-09-04T09:53:45Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![kriti\_dabas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kriti_dabas/32/118506_2.png) [@kriti\_dabas](https://discuss.elastic.co/u/kriti_dabas)
#### Post date: [September 4, 2023, 9:53am UTC](https://discuss.elastic.co/t/lag-in-logs/342233/1 "2023-09-04T09:53:45Z")

</div>

```auto
filter {
  json {
    source => "message"
  }
  
  grok {
    match => {
      "message" => [
         "%{DATA:description} default %{DATA:connection_details} : SPCBId %{DATA:spcbId} - ClientIP %{DATA:clientIP} - ClientPort %{DATA:clientPort} - VserverServiceIP %{DATA:vserverServiceIP} - VserverServicePort %{DATA:vserverServicePort} - ClientVersion %{DATA:clientVersion} - CipherSuite %{DATA:cipherSuite} - Session %{DATA:SessionStatus} - HandshakeTime %{DATA:handshakeTime} ms",
         "%{DATA:description} default %{DATA:connection_details} : SPCBId %{DATA:spcbId} - ClientIP %{DATA:clientIP} - ClientPort %{DATA:clientPort} - VserverServiceIP %{DATA:vserverServiceIP} - VserverServicePort %{DATA:vserverServicePort} - ClientVersion %{DATA:clientVersion} - CipherSuite \"%{DATA:cipherSuite}\"Session %{DATA:SessionStatus} - Reason \"%{DATA:reason}\"",
         "%{DATA:description} default %{DATA:connection_details} Source %{IP:SourceIP}:%{NUMBER:SourcePort} - Destination %{IP:DestinationIP}:%{NUMBER:DestinationPort} - Start Time %{DATA:Starttime} - End Time %{DATA:Endtime} %{GREEDYDATA:Total_bytes_recv_snd}",
         "%{DATA:description} default %{DATA:connection_details} Source %{IP:SourceIP}:%{NUMBER:SourcePort} - Vserver %{IP:VserverIP}:%{NUMBER:VserverPort} - NatIP %{DATA:NatIP}:%{NUMBER:NatIPPort} - Destination %{IP:DestinationIP}:%{NUMBER:DestinationPort} - Delink Time %{DATA:DelinkTime} %{DATA:Total_bytes_recv_snd}",
         "%{DATA:description} default %{DATA:connection_details} Source %{IP:SourceIP}:%{NUMBER:SourcePort} - Destination %{IP:Destination1IP}:%{NUMBER:Destination1Port} - NatIP %{DATA:NatIP}:%{NUMBER:NatIPPort} - Destination %{IP:Destination2IP}:%{NUMBER:Destination2Port} - Start Time %{DATA:Starttime} - Delink Time %{DATA:DelinkTime} %{GREEDYDATA:Total_bytes_recv_snd}(?= - Closure Reason) - %{GREEDYDATA:Reason}",
         "%{DATA:description} : default %{GREEDYDATA:connection_details} - %{DATA:Profile} %{DATA:Action} %{URI:url} %{GREEDYDATA:Status}"
    ]
    }
  }
}

```

because of the use of this filter my cpu usage on logstash is increasing due to which there is a lag in my logs so how to simplify my filter ?

---

<div class="post-metadata">

### Author: ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [September 4, 2023, 11:33am UTC](https://discuss.elastic.co/t/lag-in-logs/342233/2 "2023-09-04T11:33:10Z")

</div>

Grok can be CPU intensive, and your grok filter is not optimized and probably can be improved.

I recommend reading this [blog post](https://www.elastic.co/blog/do-you-grok-grok) about grok.

Also, can you share sample messages of each one of your grok patterns so people can try to replicate your filter and see how to optimize it?

To have some suggestions on how to optimize it you would need to share some sample messages.

---

<div class="post-metadata">

### Author: ![miltonhultgren](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/miltonhultgren/32/100401_2.png) [@miltonhultgren](https://discuss.elastic.co/u/miltonhultgren)
#### Post date: [September 4, 2023, 11:35am UTC](https://discuss.elastic.co/t/lag-in-logs/342233/3 "2023-09-04T11:35:01Z")

</div>

Before you try to optimize your Grok patterns, look if you can get by with other cheaper processor like [dissect](https://www.elastic.co/guide/en/logstash/current/plugins-filters-dissect.html).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [October 2, 2023, 11:35am UTC](https://discuss.elastic.co/t/lag-in-logs/342233/4 "2023-10-02T11:35:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
