# Aggregate filter for mixed data lines

**URL:** <https://discuss.elastic.co/t/aggregate-filter-for-mixed-data-lines/236110>\
**Category:** Logstash\
**Created:** [June 8, 2020, 1:19am UTC](https://discuss.elastic.co/t/aggregate-filter-for-mixed-data-lines/236110 "2020-06-08T01:19:50Z")\
**Posts on this page:** 1\
**Showing post:** 4

<div class="post-metadata">

**Author:** ![mskadu](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mskadu/32/69520_2.png) [@mskadu](https://discuss.elastic.co/u/mskadu)\
**Post date:** [June 8, 2020, 6:20pm UTC](https://discuss.elastic.co/t/aggregate-filter-for-mixed-data-lines/236110/4 "2020-06-08T18:20:35Z")

</div>

In the meanwhile, I spotted [this post](https://discuss.elastic.co/t/import-csv-with-different-column-names-to-same-field/223725/4) which allows me to use the Elasticsearch output plugin in upsert mode - which pretty near does what i need. Here's what my output section now looks like

```auto
...
output {

  if "source1" in [tags] {
       elasticsearch { ..} # write to source1 specific index
  }
  else if "source2" in [tags] {
       elasticsearch { ..} # write to source2 specific index
  }
  # and ultimately the combined index
   elasticsearch{
      hosts => ["my-es-host:9200"]
      index => ["my-combined-index"]
      action => "update"
      document_id => "%{Operation}_%{RefData1}_%{RefData1}"
      doc_as_upsert => true
   }
}

```

This gives me two choices - wicked!

---

_[View the full topic](https://discuss.elastic.co/t/aggregate-filter-for-mixed-data-lines/236110)._