Hi,
I am getting this error while starting logstash.
[2018-06-24T09:55:59,748][ERROR][logstash.filters.grok ] Unknown setting 'mutate' for grok
[2018-06-24T09:55:59,752][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Something is wrong with your configuration.", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/config/mixin.rb:89:in `config_init'", "/usr/share/logstash/logstash-core/lib/logstash/filters/base.rb:128:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/filter_delegator.rb:22:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/plugins/plugin_factory.rb:87:in `plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:112:in `plugin'", "(eval):445:in `<eval>'", "org/jruby/RubyKernel.java:994:in `eval'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:84:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:169:in `initialize'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:40:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:315:in `block in converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:312:in `block in converge_state'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:299:in `converge_state'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:166:in `block in converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:141:in `with_pipelines'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:164:in `converge_state_and_update'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:90:in `execute'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:348:in `block in execute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:24:in `block in initialize'"]}
There's nothing a normal user could understand with this descriptive error message. I'd like to have an idea what's wrong with the config.
This is the relevant area where I think logstash is failing:
input {
beats {
client_inactivity_timeout => 600
port => 5044
}
}
# The filter part of this file is commented out to indicate that it is
# optional.
filter {
if "onm-server-subdata-recs-stat" in [tags] {
if "invalid entries" in [message] {
grok {
match => { "message" => "%{NUMBER:invalid_entries}" }
}
}
if [message] =~ "Removed \d+ old devices" {
grok {
match => { "message" => "%{NUMBER:remove_old_devices}" }
}
}
if "customer details" in [message] {
grok {
match => { "message" => "%{NUMBER:updated_customer_details}" }
}
}
if "new devices" in [message] {
grok {
match => { "message" => "%{NUMBER:new_devices_added}" }
}
}
if "Marked effective end on" in [message] {
grok {
match => { "message" => "%{NUMBER:effective_end_old_devices} old devices" }
tag_on_failure => ["not-old-devices"]
}
if "not-old-devices" in [tags] {
grok {
match => { "message" => "%{NUMBER:effective_end_devices} devices" }
remove_tag => ["not-old-devices"]
}
}
}
if "devices already processed" in [message] {
grok {
match => { "message" => "%{NUMBER:purged_devices_processed}" }
}
}
if "Added new start for" in [message] {
grok {
match => { "message" => "%{NUMBER:added_new_start_for_devices}" }
}
}
mutate {
convert => { "invalid_entries" => "integer" }
convert => { "added_new_start_for_devices" => "integer" }
convert => { "purged_devices_processed" => "integer" }
convert => { "effective_end_devices" => "integer" }
convert => { "effective_end_old_devices" => "integer" }
convert => { "updated_customer_details" => "integer" }
convert => { "new_devices_added" => "integer" }
convert => { "remove_old_devices" => "integer" }
}
}
}
On a side note, even after specifying theese variables as NUMBER and adding integer after them doesn't make them numeric. So I was forced to use mutate filter for conversion.