# Getting error in Logstash grok filter for rds slow query

**URL:** <https://discuss.elastic.co/t/getting-error-in-logstash-grok-filter-for-rds-slow-query/176878>\
**Category:** Logstash\
**Created:** [April 15, 2019, 10:19am UTC](https://discuss.elastic.co/t/getting-error-in-logstash-grok-filter-for-rds-slow-query/176878 "2019-04-15T10:19:47Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![Rajat\_Ravi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajat_ravi/32/43909_2.png) [@Rajat\_Ravi](https://discuss.elastic.co/u/Rajat_Ravi)\
**Post date:** [April 15, 2019, 10:19am UTC](https://discuss.elastic.co/t/getting-error-in-logstash-grok-filter-for-rds-slow-query/176878/1 "2019-04-15T10:19:47Z")

</div>

Hi Team Below are the error that I am getting

Error registering plugin {:pipeline\_id=\>"main", :plugin=\>"#\<LogStash::FilterDelegator:0x622f012e @metric\_events\_out=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: out value:0, @metric\_events\_in=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: in value:0, @metric\_events\_time=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: duration\_in\_millis value:0, @id="950af9e45367a48d0d80d2016dabb2c00b6443c6da22ed5b72a3a2d31dcef7a6", @klass=LogStash::Filters::Grok, @metric\_events=#LogStash::Instrument::NamespacedMetric:0x240caa68, @filter=\<LogStash::Filters::Grok match=\>{"message"=\>"^# User@Host: %{USER:mysql\_user}(?:\\[[^\\]]+\\])?\\s+@\\s+%{HOST:mysql\_client\_host}?\\s+\\[%{IP:mysql\_client\_ip}?\\]"}, id=\>"950af9e45367a48d0d80d2016dabb2c00b6443c6da22ed5b72a3a2d31dcef7a6", enable\_metric=\>true, periodic\_flush=\>false, patterns\_files\_glob=\>"\*", break\_on\_match=\>true, named\_captures\_only=\>true, keep\_empty\_captures=\>false, tag\_on\_failure=\>["\_grokparsefailure"], timeout\_millis=\>30000, tag\_on\_timeout=\>"\_groktimeout"\>\>", :error=\>"pattern %{HOST:mysql\_client\_host} not defined", :thread=\>"#\<Thread:0x3f9ab6d@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:156 run\>"}

2019-04-15T08:49:06,459][ERROR][logstash.pipeline] Pipeline aborted due to error {:pipeline\_id=\>"main", :exception=\>#\<Grok::PatternError: pattern %{HOST:mysql\_client\_host} not defined\>, :backtrace=\>["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/jls-grok-0.11.5/lib/grok-pure.rb:123:in `block in compile'", "org/jruby/RubyKernel.java:1292:in`loop'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/jls-grok-0.11.5/lib/grok-pure.rb:93:in `compile'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-grok-4.0.3/lib/logstash/filters/grok.rb:281:in`block in register'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-grok-4.0.3/lib/logstash/filters/grok.rb:275:in`block in register'", "org/jruby/RubyHash.java:1343:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-grok-4.0.3/lib/logstash/filters/grok.rb:270:in`register'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:241:in `register_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:252:in`block in register\_plugins'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:252:in`register\_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:594:in `maybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:262:in`start\_workers'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:199:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:159:in`block in start'"], :thread=\>"#\<Thread:0x3f9ab6d@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:156 run\>"}  
[2019-04-15T08:49:06,468][ERROR][logstash.agent] Failed to execute action {:id=\>:main, :action\_type=\>LogStash::ConvergeResult::FailedAction, :message=\>"

And this is my conf file.

filter{  
if [type] == "rds" {  
grok {  
match =\> ["message", "^# User@Host: %{USER:mysql\_user}(?:[[^]]+])?\s+@\s+%{HOST:mysql\_client\_host}?\s+[%{IP:mysql\_client\_ip}?]"]  
}  
grok {  
match =\> ["message", "^# Query\_time: %{NUMBER:query\_duration\_s:float}\s+Lock\_time: %{NUMBER:lock\_wait\_s:float} Rows\_sent: %{NUMBER:query\_sent\_rows:int} \s_Rows\_examined: %{NUMBER:query\_examined\_rows:int}"]  
}  
grok {  
match =\> ["message", "^SET timestamp=%{NUMBER:timestamp};"]  
}  
grok {  
match =\> {  
"message" =\> [  
"^FROM %{NOTSPACE:query\_table}",  
"^UPDATE %{NOTSPACE:query\_table}._",  
"^INSERT INTO %{NOTSPACE:query\_table}._",  
"^DELETE FROM %{NOTSPACE:query\_table}._"  
]  
}  
}  
mutate {  
add\_field =\> {  
"[@metadata][indexname]" =\> "rds-logs"  
}  
}  
}  
}

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 15, 2019, 11:29am UTC](https://discuss.elastic.co/t/getting-error-in-logstash-grok-filter-for-rds-slow-query/176878/2 "2019-04-15T11:29:56Z")

</div>

> [@Rajat\_Ravi](#):
>
> :error=\>"pattern %{HOST:mysql\_client\_host} not defined"

Do you mean %{HOSTNAME:mysql\_client\_host} ?

---

<div class="post-metadata">

**Author:** ![Rajat\_Ravi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajat_ravi/32/43909_2.png) [@Rajat\_Ravi](https://discuss.elastic.co/u/Rajat_Ravi)\
**Post date:** [April 15, 2019, 11:30am UTC](https://discuss.elastic.co/t/getting-error-in-logstash-grok-filter-for-rds-slow-query/176878/3 "2019-04-15T11:30:45Z")

</div>

yes same

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 15, 2019, 11:42am UTC](https://discuss.elastic.co/t/getting-error-in-logstash-grok-filter-for-rds-slow-query/176878/4 "2019-04-15T11:42:10Z")

</div>

No, it is not the same. HOSTNAME exists in the default set of pattern definitions, HOST does not.

---

<div class="post-metadata">

**Author:** ![Rajat\_Ravi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajat_ravi/32/43909_2.png) [@Rajat\_Ravi](https://discuss.elastic.co/u/Rajat_Ravi)\
**Post date:** [April 15, 2019, 11:47am UTC](https://discuss.elastic.co/t/getting-error-in-logstash-grok-filter-for-rds-slow-query/176878/5 "2019-04-15T11:47:09Z")

</div>

Let me check.  
Thanks

---

<div class="post-metadata">

**Author:** ![Rajat\_Ravi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajat_ravi/32/43909_2.png) [@Rajat\_Ravi](https://discuss.elastic.co/u/Rajat_Ravi)\
**Post date:** [April 15, 2019, 12:06pm UTC](https://discuss.elastic.co/t/getting-error-in-logstash-grok-filter-for-rds-slow-query/176878/6 "2019-04-15T12:06:16Z")

</div>

Thanks Its working fine.  
I need one more help as I am getting index field TYPE in string but need in Number for some purpose can you help me.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 15, 2019, 12:24pm UTC](https://discuss.elastic.co/t/getting-error-in-logstash-grok-filter-for-rds-slow-query/176878/7 "2019-04-15T12:24:24Z")

</div>

If you have a field on an event that is a string and you want it to be a number then you can use the [mutate+convert](https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html#plugins-filters-mutate-convert) function.

---

<div class="post-metadata">

**Author:** ![Rajat\_Ravi](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rajat_ravi/32/43909_2.png) [@Rajat\_Ravi](https://discuss.elastic.co/u/Rajat_Ravi)\
**Post date:** [April 16, 2019, 9:14am UTC](https://discuss.elastic.co/t/getting-error-in-logstash-grok-filter-for-rds-slow-query/176878/8 "2019-04-16T09:14:54Z")

</div>

Using convert function can I do the same in existing index or do I need to create new index.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [April 16, 2019, 11:47am UTC](https://discuss.elastic.co/t/getting-error-in-logstash-grok-filter-for-rds-slow-query/176878/9 "2019-04-16T11:47:40Z")

</div>

In logstash you can make the change on-the-fly, but in elasticsearch you will need a new index to change something from string to number or vice versa.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 14, 2019, 11:47am UTC](https://discuss.elastic.co/t/getting-error-in-logstash-grok-filter-for-rds-slow-query/176878/10 "2019-05-14T11:47:57Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.