Want to replace hostname as identhostname in logstash log output

Currently Our config file 000-grok.conf file has below content:

filter {

mutate {
# squash multiple whitespace
gsub => [ "message", "{2,}"," " ]
# ditch the dashes
gsub => [ "message", "{5}.*{50,}\n","" ]
# remove newlines
gsub => [ "message", "\n", " "]
}

grok {
match => [ "message", "%{DATE:date} %{TIME:time} %{DATA:halfday} -" ]
}

if [halfday] =~ /[AP]M/ {
mutate { add_field => [ "ts" , "%{date} %{time} %{halfday}" ] }
} else {
mutate { add_field => [ "ts" , "%{date} %{time}" ] }
}

grok {
match => [ "message", "%{DATE:date} %{TIME:time} %{DATA:halfday }- Process(%{DATA:process}) User(%{WORD:usrName}) Program(%{WORD:program})\s+Host(%{DATA:hostname}) Installation(%{WORD:installation})\s+VRMF(%{DATA:vrmf})( QMgr(%{WORD:qmName}))?\s+%{DATA:errCode}: %{DATA:errDesc}\s+EXPLANATION: %{DATA:explanation}\s+ACTION: %{DATA:action}" ]
}

if [ts] =~ /date/ {
drop { }
}

date {
match => [ "ts", "MM/dd/yyyy hh:mm:ss aa", "MM/dd/yy hh:mm:ss" ]
target => "@timestamp"
}

}

#############################

We are getting the log out put like below :slight_smile:
LEEF:1.0|IBM|MQ|8.0.0.7|POSZ0R|usrName=mqm hostname=nl000917 qmName=BLNPQ errDesc=Channel 'BLNPQ.IQ' to host 'nl00001 (98.9024.56)' ended abnormally.

We want the out put to show 'identHostName' instead of hostaname.

So in which files we need to do changes and what changes we can do for the out put to get identHostName' instead of hostaname ?

We have 99-output.conf file has below content:
output {

if "_qradar" in [tags] {

stdout {
  codec => leef {
    vendor => "IBM"
    product => "MQ"
    version => "%{vrmf}"
    syslogheader => false
    eventid => "%{errCode}"
    fields => [ "usrName", "hostname", "qmName", "errDesc" ]
  }
}

syslog {
  host => "nl09621"
  port => 514
  sourcehost => "%{hostname}"
  codec => leef {
    vendor => "IBM"
    product => "MQ"
    version => "%{vrmf}"
    syslogheader => false
    eventid => "%{errCode}"
    fields => [ "usrName", "hostname", "qmName", "errDesc" ]
  }
}

}

}

Try this grok pattern. You . would want (%{DATA: identHostName}) instead (%{DATA:hostname})

Hello yes we tried like this and we got the error :

gatch => [ "message", "%{DATE:date} %{TIME:time} %{DATA:halfday }- Process(%{DATA:process}) User(%{WORD:usrName}) Program(%{WORD:program})\s+Host(%{DATA:identHostName}) Installation(%{WORD:installation})\s+VRMF(%{DATA:vrmf})( QMgr(%{WORD:qmName}))?\s+%{DATA:errCode}: %{DATA:errDesc}\s+EXPLANATION: %{DATA:explanation}\s+ACTION: %{DATA:action}" ]

Error Message :
Using GEM_HOME=/srv/prop/elk/current/logstash/vendor/bundle/jruby/2.3.0
Using GEM_PATH=/srv/prop/elk/current/logstash/vendor/bundle/jruby/2.3.0
Sending Logstash logs to /logs/infra/logstash/ais-ls-01 which is now configured via log4j2.properties
[2019-09-12T10:21:13,637][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2019-09-12T10:21:18,923][ERROR][logstash.filters.grok ] Unknown setting 'gatch' for grok
[2019-09-12T10:21:18,931][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::ConfigurationError", :message=>"Something is wrong with your configuration.", :backtrace=>["/srv/prop/elk/current/logstash/logstash-core/lib/logstash/config/mixin.rb:86:in config_init'", "/srv/prop/elk/current/logstash/logstash-core/lib/logstash/filters/base.rb:126:ininitialize'", "org/logstash/plugins/PluginFactoryExt.java:70:in filter_delegator'", "org/logstash/plugins/PluginFactoryExt.java:244:inplugin'", "org/logstash/plugins/PluginFactoryExt.java:181:in plugin'", "/srv/prop/elk/current/logstash/logstash-core/lib/logstash/pipeline.rb:71:inplugin'", "(eval):174:in <eval>'", "org/jruby/RubyKernel.java:994:ineval'", "/srv/prop/elk/current/logstash/logstash-core/lib/logstash/pipeline.rb:49:in initialize'", "/srv/prop/elk/current/logstash/logstash-core/lib/logstash/pipeline.rb:90:ininitialize'", "/srv/prop/elk/current/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:43:in block in execute'", "/srv/prop/elk/current/logstash/logstash-core/lib/logstash/agent.rb:94:inblock in exclusive'", "org/jruby/ext/thread/Mutex.java:148:in synchronize'", "/srv/prop/elk/current/logstash/logstash-core/lib/logstash/agent.rb:94:inexclusive'", "/srv/prop/elk/current/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:39:in execute'", "/srv/prop/elk/current/logstash/logstash-core/lib/logstash/agent.rb:327:inblock in converge_state'"]}
startup.20190912.1020.out (END)

That should be match, not gatch.

Grok file has the config :slight_smile:

grok {
match => [ "message", "%{DATE:date} %{TIME:time} %{DATA:halfday }- Process(%{DATA:process}) User(%{WORD:usrName}) Program(%{WORD:program})\s+Host(%{DATA:hostname}) Installation(%{WORD:installation})\s+VRMF(%{DATA:vrmf})( QMgr(%{WORD:qmName}))?\s+%{DATA:errCode}: %{DATA:errDesc}\s+EXPLANATION: %{DATA:explanation}\s+ACTION: %{DATA:action}" ]
}

The error is below we have 'match' in grok file not 'gatch' :slight_smile:
Using GEM_HOME=/srv/prop/elk/current/logstash/vendor/bundle/jruby/2.3.0
Using GEM_PATH=/srv/prop/elk/current/logstash/vendor/bundle/jruby/2.3.0
Sending Logstash logs to /logs/infra/logstash/ais-ls-01 which is now configured via log4j2.properties
[2019-09-12T11:10:53,895][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2019-09-12T11:11:00,345][ERROR][logstash.plugins.registry] Tried to load a plugin's code, but failed. {:exception=>#<LoadError: no such file to load -- logstash/filters/gutate>, :path=>"logstash/filters/gutate", :type=>"filter", :name=>"gutate"}
[2019-09-12T11:11:00,354][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"LogStash::PluginLoadingError", :message=>"Couldn't find any filter plugin named 'gutate'. Are you sure this is correct? Trying to load the gutate filter plugin resulted in this error: no such file to load -- logstash/filters/gutate", :backtrace=>["/srv/prop/elk/current/logstash/logstash-core/lib/logstash/plugins/registry.rb:211:in lookup_pipeline_plugin'", "/srv/prop/elk/current/logstash/logstash-core/lib/logstash/plugin.rb:137:inlookup'", "org/logstash/plugins/PluginFactoryExt.java:222:in plugin'", "org/logstash/plugins/PluginFactoryExt.java:181:inplugin'", "/srv/prop/elk/current/logstash/logstash-core/lib/logstash/pipeline.rb:71:in plugin'", "(eval):135:in'", "org/jruby/RubyKernel.java:994:in eval'", "/srv/prop/elk/current/logstash/logstash-core/lib/logstash/pipeline.rb:49:ininitialize'", "/srv/prop/elk/current/logstash/logstash-core/lib/logstash/pipeline.rb:90:in initialize'", "/srv/prop/elk/current/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:43:inblock in execute'", "/srv/prop/elk/current/logstash/logstash-core/lib/logstash/agent.rb:94:in block in exclusive'", "org/jruby/ext/thread/Mutex.java:148:insynchronize'", "/srv/prop/elk/current/logstash/logstash-core/lib/logstash/agent.rb:94:in exclusive'", "/srv/prop/elk/current/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:39:inexecute'", "/srv/prop/elk/current/logstash/logstash-core/lib/logstash/agent.rb:327:in `block in converge_state'"]}

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.