Remove fields with value: nil

You don't mention which version of Logstash you are using, but as of LS 2.1.1 which includes logstash-filter-csv-2.1.0 there is a 'skip_empty_columns' config option.

From filter csv.rb ( ./logstash-2.1.1/vendor/bundle/jruby/1.9/gems/logstash-filter-csv-2.1.0/lib/logstash/filters/csv.rb):

Define whether empty columns should be skipped.

Defaults to false. If set to true, columns containing no value will not get set.

config :skip_empty_columns, :validate => :boolean, :default => false

This setting is listed in Logstash 2.2 reference for csv filter (but not listed in LS-2.1 reference [even though it's in LS-2.1.1)

If you are running an earlier version of Logstash (than 2.1.1), maybe updating the csv gem itself is possibility(??) :confused: -- I really don't know.