# Netflow codec writes to multiple indexes

**URL:** https://discuss.elastic.co/t/netflow-codec-writes-to-multiple-indexes/191356
**Category:** Logstash
**Created:** [July 19, 2019, 9:08am UTC](https://discuss.elastic.co/t/netflow-codec-writes-to-multiple-indexes/191356 "2019-07-19T09:08:42Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![rysiornia](https://avatars.discourse-cdn.com/v4/letter/r/8dc957/32.png) [@rysiornia](https://discuss.elastic.co/u/rysiornia)
#### Post date: [July 19, 2019, 9:08am UTC](https://discuss.elastic.co/t/netflow-codec-writes-to-multiple-indexes/191356/1 "2019-07-19T09:08:42Z")

</div>

Hello,  
I have logstash in 6.8.1 version my problem is that when I'm using two pipelines one for metricbeat and other for netflow I can't see metricbeat data. In elasticsearch I can see two indexes metricbeat-\* and netflow\_logstash-\* but looks like both of them are netflow indexes.  
My beast.conf file:

```
 input {
      beats {
        port => 5044
      }
    }

    output {
      elasticsearch {
        hosts => "localhost:9200"
        manage_template => false
    # index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
         index => "metricbeat-%{+YYYY.MM.dd}"
      }
    } 

```

My netflow.conf:

```
input {
      udp {
        port => 19000
        codec => netflow {
                netflow_definitions => "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-codec-netflow-3.14.1/lib/logstash/codecs/netflow/netflow.yaml"
                versions => [9]
            }
      }
    }

    output {
      elasticsearch {
        hosts => "localhost:9200"
        index => "netflow_logstash-%{+YYYY.MM.dd}"
      }
    }

```

When I used `index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"` I had metricbeat index, netflow index and index named %{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd} with netflow data in it.

What is wrong with my pipelines?

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 19, 2019, 12:54pm UTC](https://discuss.elastic.co/t/netflow-codec-writes-to-multiple-indexes/191356/2 "2019-07-19T12:54:24Z")

</div>

Are you running them in separate pipelines? If they are both in the main pipeline then events will be read from both inputs and every event will be written to both outputs. Please see [this](https://discuss.elastic.co/t/logstash-setting-up-multiple-config-files-in-one-pipeline/190643/2) post.

---

<div class="post-metadata">

### Author: ![rysiornia](https://avatars.discourse-cdn.com/v4/letter/r/8dc957/32.png) [@rysiornia](https://discuss.elastic.co/u/rysiornia)
#### Post date: [July 19, 2019, 2:38pm UTC](https://discuss.elastic.co/t/netflow-codec-writes-to-multiple-indexes/191356/3 "2019-07-19T14:38:15Z")

</div>

Thanks for your help  
I misunderstood the concept of the pipeline. I thought that separate files = separate pipelines.  
In pipelines.yml I had :

```
 peline.id: main
      path.config: "/etc/logstash/conf.d/*.conf"

```

After splitting pipelines (creating different pipelines IDs) everything is working fine.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 16, 2019, 2:38pm UTC](https://discuss.elastic.co/t/netflow-codec-writes-to-multiple-indexes/191356/4 "2019-08-16T14:38:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
