If condition, with csv filter not working

Elastic Stack Logstash
1

i am trying to apply 'if' condition on the bases of field which i have defined in filebeat.yml file,
and inside 'if' condition i have placed a csv filter which is supposed to filter the data and those data should be displayed in Kibana as a independent fields. But now all the data is displayed as a message.

Below is my Filebeat.yml conf

filebeat.prospectors:
- input_type: log
  paths:
    - /var/opt/statistics/ServerIf.stat
  fields:
    document: ServerIf
- input_type: log
  paths:
    - /var/opt/statistics/ClientIf.stat
  fields:
    document: ClientIf
   output.logstash:
   hosts: ["localhost:5043"]

Below is my logstash pipeline config

 input {
	beats {
        port => "5043"
    }
}
 filter {
	
	if[document] == "ServerIf"
	{
		csv {
			autodetect_column_names => true
			separator => ","
			convert => {
				"DATE/TIME" => "date_time"
				"EnquiryIn" => "integer"
				"EnquiryOut" => "integer"
				"BalanceUsed" => "integer"
				"BalanceLeft" => "integer"
			   }

		
			}
	}
	else if[document] == "ClientIf"
	{
		csv {
			autodetect_column_names => true
			separator => ","
			}
	}
	}

output {
	elasticsearch {
        hosts => [ "localhost:9200" ]
    }
}
2

Please show an example event that Logstash has processed (but incorrectly so). Use a stdout { codec => rubydebug } output.

3

Hi magnusbaeck

I change the output to stdout { codec => rebydebug } , after that i started to receive error in logstash.

[2017-09-05T06:54:21,108][FATAL][logstash.runner ] An unexpected error occurred! {:error=>#<NameError: undefined local variable or methoddotfile' for #AwesomePrint::Inspector:0x5f447a57>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/1.9/gems/awesome_print-1.8.0/lib/awesome_print/inspector.rb:163:in merge_custom_defaults!'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/awesome_print-1.8.0/lib/awesome_print/inspector.rb:50:ininitialize'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/awesome_print-1.8.0/lib/awesome_print/core_ext/kernel.rb:9:in ai'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-codec-rubydebug-3.0.3/lib/logstash/codecs/rubydebug.rb:39:inencode_default'", "org/jruby/RubyMethod.java:120:in call'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-codec-rubydebug-3.0.3/lib/logstash/codecs/rubydebug.rb:35:inencode'", "/usr/share/logstash/logstash-core/lib/logstash/codecs/base.rb:50:in multi_encode'", "org/jruby/RubyArray.java:1613:ineach'", "/usr/share/logstash/logstash-core/lib/logstash/codecs/base.rb:50:in multi_encode'", "/usr/share/logstash/logstash-core/lib/logstash/outputs/base.rb:90:inmulti_receive'", "/usr/share/logstash/logstash-core/lib/logstash/output_delegator_strategies/single.rb:15:in multi_receive'", "org/jruby/ext/thread/Mutex.java:149:insynchronize'", "/usr/share/logstash/logstash-core/lib/logstash/output_delegator_strategies/single.rb:14:in multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/output_delegator.rb:47:inmulti_receive'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:420:in output_batch'", "org/jruby/RubyHash.java:1342:ineach'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:419:in output_batch'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:365:inworker_loop'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:330:in start_workers'"]}

Thanks,
Vipin Kumar

4

Hmm. Appears to be a bug. Unless you have particular dependencies you can try downgrading Logstash.

5

I have find the issue , the "if" condition which i was using was not used in proper way.
The correct way is below

input {
	beats {
        port => "5043"
    }
}
 filter {
	
	if[fields][document] == "ServerIf"
	{
		csv {
			autodetect_column_names => true
                       separator => ","
			convert => {
				"DATE/TIME" => "date_time"
				"EnquiryIn" => "integer"
				"EnquiryOut" => "integer"
				"BalanceUsed" => "integer"
				"BalanceLeft" => "integer"
			   }

		
			}
	}
	else if[fields][document] == "ClientIf"
        {
		csv {
			autodetect_column_names => true
			separator => ","
			}
	}
	}

output {
           elasticsearch {
        hosts => [ "localhost:9200" ]
    }
}
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.