Add a csv input for every batch


I am using http_poller input plugin and elasticsearch output plugin.. I want to use CSV output plugin also for logging logstash success in a CSV file..But in my CSV all the events are noted but not only once.
I want to note the log in csv for only once for every plugin batch. How can I do that..

I tried doing it using aggregate filter but it is not working properly.

  aggregate {
      task_id => "%{[@metadata][fingerprint]}"
      code => "event.set('is_batch_complete', true)"
      push_previous_map_as_event => true
      timeout => 300
      timeout_tags => ['_aggregatetimeout']
      timeout_code => "event.set('is_batch_complete', true)"
if [is_batch_complete] {
		  csv {
			fields => ["batch_timestamp"]
			path => "success-log\se-logstash-success-log-%{+YYYY-MM}.csv"

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.