Logstash 2.0 and Zabbix plugin

Elastic Stack Logstash
1

Hi,

I'm trying to run Logstash 2.0.0 with Zabbix output plugin but, unfortunately, receiving an error:

    ←[33mField referenced by [@metadata][zabbix_key_quikserv] is missing {:level=>:w
    arn}←[0m
    ←[33mField referenced by [@metadata][zabbix_key_fixinfe] is missing {:level=>:wa
    rn}←[0m

←[33mZabbix server at 10.1.110.71 rejected all items sent. {:zabbix_host=>"s-msk
11-tst01", :level=>:warn}←[0m

{
       "message" => "1: 12 Nov 15 (Thu) 12:47:09.470 (5408:6120:GATE): [Sta] Qui
kApi allocated: 1.83 Mb, orders: 12, trades: 15, ob: 0, param: 1027\r",
      "@version" => "1",
    "@timestamp" => "2015-11-12T09:47:09.501Z",
          "host" => "s-msk11-tst01",
          "path" => "C:\\QUIK\\FIX Adapters\\FixIn_FIXEdge_dev\\Import\\QuikFix.
log",
          "type" => "FIXINFE-log"
}

On Zabbix server I've configured items (zabbix trapper): quikserv, fixin.fixedgedev.
My logstash.conf is below:

input {
file {
	path => ["C:\QUIK\Server\events.log"]
	type => "QUIKServ-log"
	codec => plain { charset => "Windows-1252" }
	start_position => "end"
	sincedb_path => "C:\Progra~1\logstash\sincedb\sincedb_quikserv"
	}
file {
	path => ["C:\QUIK\Multihub_Curr_RB_ROL\multihub.log"]
	type => "MHUBCURR-log"
	codec => plain { charset => "UTF-8" }
	start_position => "end"
	sincedb_path => "C:\Progra~1\logstash\sincedb\sincedb_mhubcurr"
	}
file {
	path => ["C:\QUIK\FIX Adapters\FixIn_FIXEdge_dev\Import\QuikFix.log"]
	type => "FIXINFE-log"
	codec => plain { charset => "Windows-1252" }
	start_position => "end"
	sincedb_path => "C:\Progra~1\logstash\sincedb\sincedb_fixinfe"
	}
}

filter {
	if [type] == "MHUBCURR-log" and [message] !~ /Error|No gate for/ {
		drop { }
		mutate {
			add_field => { "[@metadata][zabbix_key_mhubcurr]" => "mhub.curr" }
		}
	}
	if [type] == "QUIKServ-log" and [message] !~ /Error|Disconnect|Exception|Stop QUIK work/ {
		drop { }
		mutate {
			add_field => { "[@metadata][zabbix_key_quikserv]" => "quiksrv" }
		}
	}
	if [type] == "FIXINFE-log" and [message] !~ /Disconnect|down|150=8|35=9|rejected|Exit|ERROR|Destroy|allocated/ {
		drop { }
		mutate {
			add_field => { "[@metadata][zabbix_key_fixinfe]" => "fixin.fixedgedev" }
		}
	}
}

output {
	if [type] == "QUIKServ-log" {
	zabbix {
		zabbix_host => "host"
		zabbix_key => "[@metadata][zabbix_key_quikserv]"
		zabbix_server_host => "10.1.110.71"
		zabbix_value => "message"
	}
	}
	if [type] == "MHUBCURR-log" {
	zabbix {
		zabbix_host => "host"
		zabbix_key => "[@metadata][zabbix_key_mhubcurr]"
		zabbix_server_host => "10.1.110.71"
		zabbix_value => "message"
	}
	}
	if [type] == "FIXINFE-log" {
	zabbix {
		zabbix_host => "host"
		zabbix_key => "[@metadata][zabbix_key_fixinfe]"
		zabbix_server_host => "10.1.110.71"
		zabbix_value => "message"
	}
	}
	stdout { codec => rubydebug }
}

Could you please advise what might be wrong in my configuration and how to fix it?
Thanks!

2

Here's the flow for the message that's in your error:

  1. Event gets picked up from host s-msk11-tst01 and tagged with type = FIXINFE-log
  2. The last conditional in the filter block matches the type.
  3. The message field is evaluated and it matches your regular expression, so it continues on without the drop and the add_field being performed. This means that @metadata does not get populated with anything.
  4. Your output conditional checks for type to be FIXINFE-log
  5. It attempts to send all matching events to zabbix with the provided parameters
  6. Because your event did not get [@metadata][zabbix_key_fixinfe] in the filter block, it is missing, and the zabbix output plugin yields the error you see.

I'm also perplexed to see that you're dropping the event with every instance of drop {}. There's nothing to do an add_field to once the event has been dropped. Barring that, it may create a completely new, but empty, event with just that value in @metadata. This would still fail to be sent as the message field and host field would be dropped.

1 Like