# Using regex in gsub raises CPU usage significantly

**URL:** <https://discuss.elastic.co/t/using-regex-in-gsub-raises-cpu-usage-significantly/202408>\
**Category:** Logstash\
**Created:** [October 5, 2019, 2:25am UTC](https://discuss.elastic.co/t/using-regex-in-gsub-raises-cpu-usage-significantly/202408 "2019-10-05T02:25:53Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ged](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ged/32/65761_2.png) [@Ged](https://discuss.elastic.co/u/Ged)\
**Post date:** [October 5, 2019, 2:25am UTC](https://discuss.elastic.co/t/using-regex-in-gsub-raises-cpu-usage-significantly/202408/1 "2019-10-05T02:25:53Z")

</div>

Hi guys,

I want to share my observations about using gsub with regex.

I use gsub for extracting application instance name from log file path.  
Example path: **/logs/pppas/logs\_tcc/app619\_pppas-prod1/payload.log** where  
**app619\_pppas-prod1** is instance name.

Originally i used static values in gsub:

```
mutate {
        gsub => [path, "/logs/pppas/logs_tcc/", ""]
        gsub => [path, "/payload.log", ""]
      }

```

and that works fine.  
But I've changed to below using regex:

```
mutate {
        gsub => [path, ".*\/([^\/]+)+\/+[^\/]+", "\1"]
}

```

and then my CPU usage doubled from 50% to 100% on 2x6 cores machine !

So be careful with using that.

Could anyone look at that and explain the reason ?

Thanks  
Ged

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [October 5, 2019, 5:02am UTC](https://discuss.elastic.co/t/using-regex-in-gsub-raises-cpu-usage-significantly/202408/2 "2019-10-05T05:02:40Z")

</div>

I would say that is expected as regex processing depending on pattern can be a lot more CPU intensive than matching static strings.

---

<div class="post-metadata">

**Author:** ![Ged](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ged/32/65761_2.png) [@Ged](https://discuss.elastic.co/u/Ged)\
**Post date:** [October 5, 2019, 11:33am UTC](https://discuss.elastic.co/t/using-regex-in-gsub-raises-cpu-usage-significantly/202408/3 "2019-10-05T11:33:06Z")

</div>

Correct, and when i have huge amount of log lines for processing impact on CPU usage is significant. Just posted this to Logstash users to be aware of that.

Thanks !

Ged

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [October 5, 2019, 1:09pm UTC](https://discuss.elastic.co/t/using-regex-in-gsub-raises-cpu-usage-significantly/202408/4 "2019-10-05T13:09:53Z")

</div>

> [@Ged](#):
>
> mutate { gsub =\> [path, ".\*/([^/]+)+/+[^/]+", "\1"] }

That could backtrack a lot. It might be faster if you anchor it to the end of line using $.

```
mutate { gsub => [path, ".*\/([^\/]+)+\/+[^\/]+$", "\1"] }

```

---

<div class="post-metadata">

**Author:** ![Ged](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ged/32/65761_2.png) [@Ged](https://discuss.elastic.co/u/Ged)\
**Post date:** [October 19, 2019, 12:12pm UTC](https://discuss.elastic.co/t/using-regex-in-gsub-raises-cpu-usage-significantly/202408/5 "2019-10-19T12:12:00Z")

</div>

Thanks a lot !  
I'll try it and let know of results.  
Hoping Logstash parses regex only on configuration load and keeps it in memory not at every event.

Ged

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [October 19, 2019, 12:55pm UTC](https://discuss.elastic.co/t/using-regex-in-gsub-raises-cpu-usage-significantly/202408/6 "2019-10-19T12:55:39Z")

</div>

> [@Ged](#):
>
> Hoping Logstash parses regex only on configuration load and keeps it in memory not at every event.

The regular expresions are compiled during [initialization](https://github.com/logstash-plugins/logstash-filter-grok/blob/155eb3ec0bd6b5026655304e1e9a5e541dcd2699/lib/logstash/filters/grok.rb#L246) and re-used.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 16, 2019, 12:55pm UTC](https://discuss.elastic.co/t/using-regex-in-gsub-raises-cpu-usage-significantly/202408/7 "2019-11-16T12:55:39Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.