# Indexing question related to pipeline outputs

**URL:** <https://discuss.elastic.co/t/indexing-question-related-to-pipeline-outputs/218417>\
**Category:** Logstash\
**Created:** [February 7, 2020, 9:35pm UTC](https://discuss.elastic.co/t/indexing-question-related-to-pipeline-outputs/218417 "2020-02-07T21:35:21Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![\_finack](https://avatars.discourse-cdn.com/v4/letter/_/e68b1a/32.png) [@\_finack](https://discuss.elastic.co/u/_finack)\
**Post date:** [February 7, 2020, 9:35pm UTC](https://discuss.elastic.co/t/indexing-question-related-to-pipeline-outputs/218417/1 "2020-02-07T21:35:21Z")

</div>

**Issue** : I have a Logstash pipeline output testing `if [agent][type] == "metricbeat"`. I am expecting such documents to end up in indices matching the pattern metricbeat-\*; however, some are ending up in indices matching the pattern indexme-\*, despite the documents in indexme-\* having an agent.type field set to "metricbeat". I don't understand why this is occurring.

**The Scenario** : I am testing Elastic Stack and right now I just have Metricbeat feeding data from the ES server and Winlogbeat feeding data from a Windows Event Collector. Both are going into Logstash.

I was messing around with the pipeline filters to create new document fields for Windows events and discovered [the pipeline configurations from the HELK project](https://github.com/Cyb3rWard0g/HELK/tree/master/docker/helk-logstash/pipeline). I loaded all of these plugins (except 0002-kafka-input.conf), fixed the output plugins to not use authentication and go to the correct ES server (localhost).

This resulted in all of the Metricbeat documents going to indexme-\* indices (due to 9998-catch\_all-output.conf). I created a new pipeline output titled 9930-metricbeat-output.conf that uses the basic output configuration from the ES getting started guide (see below) except with the addition of checking if `[agent][type] == "metricbeat"`. Now I am getting some metricbeat-related documents in indexme-\* and others in metricbeat-\*. Documents in both indices contain "metricbeat" in a field named agent.type.

Please help me understand why this is happening.

I'm new to Elastic Search.

**Supporting Information** :

9930-metricbeat-output.conf

```
output {
  if [agent][type] == "metricbeat" {
    elasticsearch {
      hosts => "localhost:9200"
      manage_template => false
      index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
      #user => "username"
      #password => "password"
    }
  }
}

```

9998-catch\_all-output.conf

```
output {
  if [@metadata][helk_parsed] != "yes" and [source] != "/var/log/osquery/osqueryd.results.log" and [@metadata][kafka][topic] != "SYSMON_JOIN" and [@metadata][helk_input_source] != "mitre_attack"{
    elasticsearch {
      hosts => ["localhost:9200"]
      index => "indexme-%{+YYYY.MM.dd}"
      # document_id => "%{[@metadata][log_hash]}"
      #user => 'elastic'
      #password => 'elasticpassword'
    }
  }
}
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 6, 2020, 9:35pm UTC](https://discuss.elastic.co/t/indexing-question-related-to-pipeline-outputs/218417/2 "2020-03-06T21:35:38Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.