# Duplicate data Logstash
**URL:** https://discuss.elastic.co/t/duplicate-data-logstash/269635
**Category:** Logstash
**Created:** [April 8, 2021, 6:21pm UTC](https://discuss.elastic.co/t/duplicate-data-logstash/269635 "2021-04-08T18:21:21Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@Juan\_David\_Jaramillo](https://discuss.elastic.co/u/Juan_David_Jaramillo)
#### Post date: [April 8, 2021, 6:21pm UTC](https://discuss.elastic.co/t/duplicate-data-logstash/269635/1 "2021-04-08T18:21:21Z")
Hi, guy's
i need your help, please with this...
I have a problem with the indexing of data from logstash to elastic, I have two pipelines and when I run the service I duplicate the data from one pipeline to the other, and does not let me create an individualized one, what I need is to create an index with the correct data and not pass me the other data from the other pipeline, thanks, it should be noted that the first pipeline I have with snmp and the other, which shows me the data from snmp is syslog.
---
### Author:  [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)
#### Post date: [April 8, 2021, 6:28pm UTC](https://discuss.elastic.co/t/duplicate-data-logstash/269635/2 "2021-04-08T18:28:24Z")
Logstash by default concatenates config files and do not handle them separately. This means that data from all inputs go to all outputs unless you implement control through [conditionals](https://www.elastic.co/guide/en/logstash/7.12/event-dependent-configuration.html) or specify the configs as [distinct pipelines using pipelines.yml](https://www.elastic.co/guide/en/logstash/7.12/multiple-pipelines.html).
---
### Author:  [@Juan\_David\_Jaramillo](https://discuss.elastic.co/u/Juan_David_Jaramillo)
#### Post date: [April 8, 2021, 8:09pm UTC](https://discuss.elastic.co/t/duplicate-data-logstash/269635/3 "2021-04-08T20:09:59Z")
good, thanks, but now I have a problem with the configuration of multiple pipelines, and is that I do not work the configuration in the pipeline.yml, the pipeline\_id remains the same by default, how can I configure this correctly use the command: /usr/share/logstash/bin/logstash -f efecty --config.reload.automatic , but according to the documentation tells me that with the -f ignores the pipeline.yml, but here is the problem if I remove the -f error comes out what can I do?
---
### Author:  [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)
#### Post date: [April 8, 2021, 9:02pm UTC](https://discuss.elastic.co/t/duplicate-data-logstash/269635/4 "2021-04-08T21:02:03Z")
What is the error? You are running logstash in the command line with the parameter `-f` pointing to a config file, it will ignore the `pipelines.yml` file and the `pipeline_id` will be called `main`, this is what is happening in your first screenshot.
If you want to run two different pipelines you need to configure the `pipeline.yml` file and point to the config files.
Based in your screenshots, you should use this `pipelines.yml`
```auto
- pipeline.id: efecty
path.config: "/etc/logstash/conf.d/efecty.conf"
pipeline.workers: 3
- pipeline.id: syslog
path.config: "/etc/logstash/conf.d/syslog.conf"
queue.type: persisted
```
Try to use this config and run logstash without the `-f` parameter.
Besides that, please avoid posting screenshots, sometimes it is not possible to read, share your code or logs using the `` button.
---
### Author:  [@Juan\_David\_Jaramillo](https://discuss.elastic.co/u/Juan_David_Jaramillo)
#### Post date: [April 8, 2021, 9:13pm UTC](https://discuss.elastic.co/t/duplicate-data-logstash/269635/5 "2021-04-08T21:13:53Z")
ok, thanks I am going to try it , ok i get it
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [May 6, 2021, 9:14pm UTC](https://discuss.elastic.co/t/duplicate-data-logstash/269635/6 "2021-05-06T21:14:23Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.