Mutate Filter - Multiple Datatype Conversion


(VISHNU) #1

Hi,
I am using the following logstash input configuration. Previously I was using only the dissect filter. I added the mutate filter to change the datatypes for ease of use in KIBANA.

filter {
  dissect {
    mapping => { "exec[stdout]" => "%{host} %{up} %{users} %{load} %{IP} %{Memoryfree} %{MemoryTotal} %{SwapTotal} %{SwapFree} %{Psql_active} %{psql_established} %{Java_Thread} {}" }
  }

mutate {
 convert =>     {"up" => "float"}
 convert =>     {"users" => "float"}
 convert =>     {"load" => "double"}
 convert =>     {"IP" => "string"}
 convert =>     {"Memoryfree" => "long"}
 convert =>     {"MemoryTotal" => "long"}
 convert =>     {"SwapTotal" => "long"}
 convert =>     {"SwapFree" => "long"}
 convert =>     {"Psql_active" => "float"}
 convert =>     {"psql_established" => "float"}
 convert =>     {"Java_Thread" => "float"}

  }

}

But when I add the muate filter its prompting error in the logstash.

[2018-10-25T13:36:22,120][ERROR][logstash.pipeline ] Error registering plugin {:pipeline_id=>"main", :plugin=>"#<LogStash::FilterDelegator:0x57e87ed8 @metric_events_out=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: out value:0, @metric_events_in=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: in value:0, @metric_events_time=org.jruby.proxy.org.logstash.instrument.metrics.counter.LongCounter$Proxy2 - name: duration_in_millis value:0, @id="9dc51586cb3b5586447b55682d9409ff4aec394cb69e9d6b72a2488650b7b5e2", @klass=LogStash::Filters::Mutate, @metric_events=#LogStash::Instrument::NamespacedMetric:0x31174fa7, @filter=<LogStash::Filters::Mutate convert=>{"up"=>"float", "users"=>"float", "load"=>"double", "IP"=>"string", "Memoryfree"=>"long", "MemoryTotal"=>"long", "SwapTotal"=>"long", "SwapFree"=>"long", "Psql_active"=>"float", "psql_established"=>"float", "Java_Thread"=>"float"}, id=>"9dc51586cb3b5586447b55682d9409ff4aec394cb69e9d6b72a2488650b7b5e2", enable_metric=>true, periodic_flush=>false>>", :error=>"translation missing: en.logstash.agent.configuration.invalid_plugin_register", :thread=>"#<Thread:0x35783db4 run>"}
[2018-10-25T13:36:22,136][ERROR][logstash.pipeline ] Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<LogStash::ConfigurationError: translation missing: en.logstash.agent.configuration.invalid_plugin_register>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-mutate-3.3.3/lib/logstash/filters/mutate.rb:219:in block in register'", "org/jruby/RubyHash.java:1343:ineach'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-filter-mutate-3.3.3/lib/logstash/filters/mutate.rb:217:in register'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:242:inregister_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:253:in block in register_plugins'", "org/jruby/RubyArray.java:1734:ineach'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:253:in register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:595:inmaybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:263:in start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:200:inrun'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:160:in `block in start'"], :thread=>"#<Thread:0x35783db4 run>"}
[2018-10-25T13:36:22,159][ERROR][logstash.agent ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create, action_result: false", :backtrace=>nil}

Request to advise if I am doing a wrong method of converting!

Thanks in advance!


(VISHNU) #2

Cleared the issue. The problem was with the wrong datatype conversion.


(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.