# Date and mutate filter performance

**URL:** <https://discuss.elastic.co/t/date-and-mutate-filter-performance/40638>\
**Category:** Logstash\
**Created:** [February 1, 2016, 2:18pm UTC](https://discuss.elastic.co/t/date-and-mutate-filter-performance/40638 "2016-02-01T14:18:38Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![matejzero](https://avatars.discourse-cdn.com/v4/letter/m/e9bcb4/32.png) [@matejzero](https://discuss.elastic.co/u/matejzero)\
**Post date:** [February 1, 2016, 2:18pm UTC](https://discuss.elastic.co/t/date-and-mutate-filter-performance/40638/1 "2016-02-01T14:18:39Z")

</div>

Hello,

I'm doing a little bit of performance testing and optimizations with my current setup and I'm seeing weird performance drops with mutate and date filter.

My setup:

- Logstash node with 6 CPUs (VM) and I run logstash with -w 4 parameter
- Logstash 2.1.1
- file input where I feed in a combined one day log file from various services that I send to logstash (cca 2,9GB)
- null output plugin
- metrics plugin for measuring performance
- different filters after metric plugin

Performance I see:

- input, metric filter, output: cca 42000 events/s
- input, metric filter, grok filter, output: cca 41000 events/s
- input, metric filter, grok filter, mutate filter, output: cca 20000 events/s
- input, metric filter, grok filter, date filter, output: cca 15000 events/s

Mutate filter is renaming one field and removing one, and date filter is only updating timestamp field.

Is such a drop in performance normal? I would expect grok to be the plugin where I would lost the most performance, not mutate or date filters.

When testing, all 4 filter workers use 100% CPU.

Config file:

**00-input:**

> input {  
> file {  
> path =\> "/root/logstash-performance/logstash-test-log.1"  
> start\_position =\> "beginning"  
> sincedb\_path =\> "/dev/null"  
> add\_field =\> { "syslog\_format" =\> "true" }  
> }  
> }

**05-metrics** :

> filter {  
> metrics {  
> meter =\> "events"  
> add\_tag =\> "metric"  
> add\_field =\> { "metric\_type" =\> "Input" }  
> flush\_interval =\> 10  
> }  
> }

**10-filter** :

> filter {  
> if [syslog\_format] == "true" {  
> grok {  
> patterns\_dir =\> ['/etc/logstash/patterns']  
> match =\> ['message', '%{SYSLOGLINE}']  
> overwrite =\> ['message']  
> add\_tag =\> ['\_grok\_syslog\_prefilter\_success']  
> tag\_on\_failure =\>   
> }  
> mutate {  
> rename =\> ["logsource", "host"]  
> remove\_field =\> ["syslog\_format"]  
> }  
> date {  
> match =\> ["timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"]  
> }  
> }  
> }

**99-output** :

> output {  
> if "metric" in [tags] {  
> stdout {  
> codec =\> line {  
> format =\> "%{[metric\_type]}: 1m rate: %{[events][rate\_1m]} "  
> }  
> }  
> }  
> null {}  
> }

Thanks, Matej

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 5:13am UTC](https://discuss.elastic.co/t/date-and-mutate-filter-performance/40638/2 "2017-07-06T05:13:33Z")

</div>