# How grok filter use more cpu?

**URL:** <https://discuss.elastic.co/t/how-grok-filter-use-more-cpu/130194>\
**Category:** Logstash\
**Created:** [May 2, 2018, 6:51am UTC](https://discuss.elastic.co/t/how-grok-filter-use-more-cpu/130194 "2018-05-02T06:51:40Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![RameshNagargoje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rameshnagargoje/32/30212_2.png) [@RameshNagargoje](https://discuss.elastic.co/u/RameshNagargoje)\
**Post date:** [May 2, 2018, 6:51am UTC](https://discuss.elastic.co/t/how-grok-filter-use-more-cpu/130194/1 "2018-05-02T06:51:40Z")

</div>

Hi, I have used grok filter to match the first line after the second line, When config file is loaded then it used more CPU I am not getting why it is happening

here is my grok pattern

(?\<Matched\_Lines\>hd\_bus\_rxctl: rxcnt\_timeout(.\*\n)+(?i).\*dhd\_check\_hang: Event HANG send up due to)

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 2, 2018, 7:00am UTC](https://discuss.elastic.co/t/how-grok-filter-use-more-cpu/130194/2 "2018-05-02T07:00:15Z")

</div>

> I have used grok filter to match the first line after the second line

What do you mean? Please show your full configuration and example input.

---

<div class="post-metadata">

**Author:** ![RameshNagargoje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rameshnagargoje/32/30212_2.png) [@RameshNagargoje](https://discuss.elastic.co/u/RameshNagargoje)\
**Post date:** [May 2, 2018, 9:23am UTC](https://discuss.elastic.co/t/how-grok-filter-use-more-cpu/130194/3 "2018-05-02T09:23:13Z")

</div>

here is input:

Expressions can be long and complex. Expressions can contain other expressions, you can negate expressions with !, and you can group them with parentheses (...).

```
For example, the following conditional uses the mutate filter to remove the field secret if the field action has a value of login:

rxctl: rxcnt_timeout=5, rxlen=0
15119.753889] [<c0064364>] (kthread+0xe0/0xe4) from [<c000f1a0>] (ret_from_fork+0x14/0x20)
15119.753889] dhd_bus_rxctl: rxcnt_timeout=5, rxlen=0
15119.753889] dhd_bus_rxctl: rxcnt_timeout=5, rxlen=0
15119.753889] dhd_bus_rxctl: rxcnt_timeout=5, rxlen=0
15119.753889] dhd_bus_rxctl: rxcnt_timeout=5, rxlen=0

[15119.753902] dhd_check_hang: Event HANG send up due to re=5 te=0 e=-110 s=2
[15119.753902] Dhd_check_hang: Event HANG send up due to re=5 te=0 e=-110 s=2
[15119.753902] Dhd_check_hang: Event HANG send up due to re=5 te=0 e=-110 s=2
[15119.753902] Dhd_check_hang: Event HANG send up due to re=5 te=0 e=-110 s=2
[15119.753917] Dhd_check_hang: Event HANG send up due to re=5 te=0 e=-110 s=2

[15119.753937] Dhd_prot_ioctl : bus is down. we have nothing to do
[15119.791431] [<c0ab1378>] (schedule_timeout+0x158/0x25c) from [<ea1e0000>] (0xea1e0000)
[15119.799331] kworker/3:2 R running 0 29597 2 0x00000000
[15119.805699] [<c0ab2fd4>] (__schedule+0x3d0/0x8a4) from [<c005d71c>] (worker_thread+0x1fc/0x3dc)
[15119.814384] [<c005d71c>] (worker_thread+0x1fc/0x3dc) from [<c0064364>] (kthread+0xe0/0xe4)
[15119.822637] [<c0064364>] (kthread+0xe0/0xe4) from [<c000f1a0>] (ret_from_fork+0x14/0x20)
[15119.830710] kworker/u8:1 S c0ab2fd4 0 29738 2 0x00000000
[15119.837078] [<c0ab2fd4>] (__schedule+0x3d0/0x8a4) from [<c005d71c>] (worker_thread+0x1fc/0x3dc)
[15119.845763] [<c005d71c>] (worker_thread+0x1fc/0x3dc) from [<c0064364>] (kthread+0xe0/0xe4)
[15119.854015] [<c0064364>] (kthread+0xe0/0xe4) from [<c000f1a0>] (ret_from_fork+0x14/0x20)
[15119.862088] kworker/u8:4 S c0ab2fd4 0 29739 2 0x00000000
[15119.868455] [<c0ab2fd4>] (__schedule+0x3d0/0x8a4) from [<c005d71c>] (worker_thread+0x1fc/0x3dc)
[15119.877140] [<c005d71c>] (worker_thread+0x1fc/0x3dc) from [<c0064364>] (kthread+0xe0/0xe4)
[15119.885391] [<c0064364>] (kthread+0xe0/0xe4) from [<c000f1a0>] (ret_from_fork+0x14/0x20)
[15119.893468] Sched Debug Version: v0.10, 3.10.96+ #1
[15119.898337] ktime

```

here is config file

if[type] == 'dmesg'  
{

```
  grok
  {
    match => { logs => "(?<Matched_Lines>.*hd_bus_rxctl: rxcnt_timeout.*(.*\n)+.*(?i)dhd_check_hang: Event HANG send up due to.*)"}
    add_tag => ["regexMatched"]      
  }

  if "regexMatched" in [tags]
  { 

          ruby
          {
            code => 'event.set("logSnippet", event.get("logs").scan(/.*(?i)dhd_check_hang: Event HANG send up due to.*/))'
          }
          mutate
          {
            add_field => {"tagName"=>"GENERIC_EMERGENCY"}
            add_field => {"module" => "null"}
            add_tag => ["SUCCESS"]
            remove_tag =>"regexMatched"
          }
       
  }

```

}

when config is loaded then cpu sounds starts increasing

here is error is displayed.

```
For example, the following conditional uses the mutate filter to remove the field se'!

```

[2018-05-02T02:17:26,666][WARN][logstash.filters.grok] Timeout executing grok '(?\<Matched\_Lines\>._hd\_bus\_rxctl: rxcnt\_timeout._(._\n)+._(?i)dhd\_check\_hang: Event HANG send up due to.\*)' against field 'logs' with value 'Value too large to output (2470 bytes)! First 255 chars are:  
Expressions can be long and complex. Expressions can contain other expressions, you can negate expressions with !, and you can group them with parentheses (...).

---

<div class="post-metadata">

**Author:** ![RameshNagargoje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rameshnagargoje/32/30212_2.png) [@RameshNagargoje](https://discuss.elastic.co/u/RameshNagargoje)\
**Post date:** [May 2, 2018, 9:30am UTC](https://discuss.elastic.co/t/how-grok-filter-use-more-cpu/130194/4 "2018-05-02T09:30:39Z")

</div>

@magnusbaeck, please have a look

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [May 2, 2018, 12:18pm UTC](https://discuss.elastic.co/t/how-grok-filter-use-more-cpu/130194/5 "2018-05-02T12:18:36Z")

</div>

It looks like you want to capture all lines between the lines matching two regular expressions. Doing this with a regexp isn't very efficient. Doing it in a ruby filter that loops over the lines in the field should be much faster.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 30, 2018, 12:18pm UTC](https://discuss.elastic.co/t/how-grok-filter-use-more-cpu/130194/6 "2018-05-30T12:18:44Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.