# Kv filter killing logstash performance (II) \[Solved\] **URL:** \ **Category:** Logstash\ **Tags:** elastic-stack-monitoring\ **Created:** [November 28, 2018, 12:50am UTC](https://discuss.elastic.co/t/kv-filter-killing-logstash-performance-ii-solved/158435 "2018-11-28T00:50:14Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![nacesprin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nacesprin/32/30024_2.png) [@nacesprin](https://discuss.elastic.co/u/nacesprin)\ **Post date:** [November 28, 2018, 12:50am UTC](https://discuss.elastic.co/t/kv-filter-killing-logstash-performance-ii-solved/158435/1 "2018-11-28T00:50:14Z")
Hello. I have exactly the same problem like [KV filter killing logstash performance](https://discuss.elastic.co/t/kv-filter-killing-logstash-performance/47817) but I continue on this post due to I cannot reply on it. I mean, by using KV filter, the CPU go after some hours at 100%. If I disable KV filter, the CPU usage down to normal usage. So, only disabling KV filter, all works fine. This is my KV filter: ``` kv { include_keys => ["HTTP_REFERER", "REQUEST_URI"] #the keys above can be: "HTTP_REFERER=the-referer.html" OR "HTTP_REFERER:the-referer.html" value_split_pattern => "=|:" } } ``` In the same message can exists up to 3 times repeated with differents patterns as follow: ``` REQUEST_URI=/the/url.html Request_Uri: /the/url.html [REQUEST_URI] => /the/url.html ``` ---
**Author:** ![nacesprin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nacesprin/32/30024_2.png) [@nacesprin](https://discuss.elastic.co/u/nacesprin)\ **Post date:** [November 28, 2018, 11:11am UTC](https://discuss.elastic.co/t/kv-filter-killing-logstash-performance-ii-solved/158435/2 "2018-11-28T11:11:04Z")
Is there any way to replace KV filter by a GROK filter? ---
**Author:** ![Eniqmatic](https://avatars.discourse-cdn.com/v4/letter/e/ea5d25/32.png) [@Eniqmatic](https://discuss.elastic.co/u/Eniqmatic)\ **Post date:** [November 28, 2018, 12:07pm UTC](https://discuss.elastic.co/t/kv-filter-killing-logstash-performance-ii-solved/158435/3 "2018-11-28T12:07:53Z")
There is but is not as nice. Whats the specs of the machine? ---
**Author:** ![nacesprin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nacesprin/32/30024_2.png) [@nacesprin](https://discuss.elastic.co/u/nacesprin)\ **Post date:** [November 28, 2018, 12:26pm UTC](https://discuss.elastic.co/t/kv-filter-killing-logstash-performance-ii-solved/158435/4 "2018-11-28T12:26:10Z")
I'm trying this grok ``` grok { match => { "message" => [ "REQUEST_URI=%{URIPATHPARAM:request_uri}" ] } } ``` but it doesn't work neither for ``` Stack trace: #0 /var/www/framework/yii-1.1.19/yiilite.php(1695): CWebApplication->runController('articulos/cienc...') #1 /var/www/framework/yii-1.1.19/yiilite.php(1212): CWebApplication->processRequest() #2 /var/www/protected/components/WebApplicationEndBehavior.php(27): CApplication->run() #3 /var/www/framework/yii-1.1.19/yiilite.php(709): WebApplicationEndBehavior->runEnd('front') #4 /var/www/index.php(34): CComponent->__call('runEnd', Array) #5 {main} REQUEST_URI=/articulos/ciencia-y-tecnologia/t1326/test? --- IP: 2a01:7e01:0:0:f03c:91ff:fefb:bsde ``` ---
**Author:** ![Eniqmatic](https://avatars.discourse-cdn.com/v4/letter/e/ea5d25/32.png) [@Eniqmatic](https://discuss.elastic.co/u/Eniqmatic)\ **Post date:** [November 28, 2018, 12:46pm UTC](https://discuss.elastic.co/t/kv-filter-killing-logstash-performance-ii-solved/158435/5 "2018-11-28T12:46:04Z")
You would need to post your full log lines ---
**Author:** ![nacesprin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nacesprin/32/30024_2.png) [@nacesprin](https://discuss.elastic.co/u/nacesprin)\ **Post date:** [November 28, 2018, 3:28pm UTC](https://discuss.elastic.co/t/kv-filter-killing-logstash-performance-ii-solved/158435/6 "2018-11-28T15:28:15Z")
Thanks @Eniqmatic for your help, but yet is solved. I write it for future references: patterns\_dir: ``` #Fechas: "[2018-12-31 12:12:12]", "2018-12-31 12:12:12", "[2018/12/-31 12:12:12]", "2018/12/-31 12:12:12", "---- + FECHA" TIMESTAMP_1 ^(?:----)?(?:\[)?\d{4}[\/-](?:\d{2}|\w{3})[\/-]\d{2}\s\d{2}:\d{2}:\d{2} #Fecha [24-Apr-2018 12:12:12] TIMESTAMP_2 ^(?:----)?\[\d{2}-\w{3}-\d{4}\s\d{2}:\d{2}:\d{2}\] #Merge TIMESTAMP_1 and 2 VARIOS_TIMESTAMPS %{TIMESTAMP_1}|%{TIMESTAMP_2} ``` filter: ``` #GROKs matches must be splitted in multiples entries. These Groks replaces KV filter grok { patterns_dir => ["/etc/logstash/conf.d/patrones-grok"] match => { "message" => "%{VARIOS_TIMESTAMPS:tempLogTimestamp}" } } #These GROKs replaces KV filter which is downing CPU performance if [message] =~ /(?i)\bREQUEST_URI\b/ { grok { match => { "message" => "REQUEST_URI=%{URIPATHPARAM:request_uri}" } } } if [message] =~ /(?i)\bIP\b/ { grok { match => { "message" => "IP:\s?%{IP:ip}" } } } ``` ---
**Author:** ![nacesprin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nacesprin/32/30024_2.png) [@nacesprin](https://discuss.elastic.co/u/nacesprin)\ **Post date:** [November 30, 2018, 10:48am UTC](https://discuss.elastic.co/t/kv-filter-killing-logstash-performance-ii-solved/158435/7 "2018-11-30T10:48:19Z")
By nesting KV filter in an IF sentence, the CPU performance worked nice: ``` if [message] =~ /(?i)\bREQUEST_URI\b/ { grok { match => { "message" => "REQUEST_URI=%{URIPATHPARAM:request_uri}" } } } ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 28, 2018, 10:53am UTC](https://discuss.elastic.co/t/kv-filter-killing-logstash-performance-ii-solved/158435/8 "2018-12-28T10:53:53Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.