# Multiple Config Files - Duplicate data
**URL:** https://discuss.elastic.co/t/multiple-config-files-duplicate-data/294898
**Category:** Logstash
**Created:** [January 20, 2022, 7:09am UTC](https://discuss.elastic.co/t/multiple-config-files-duplicate-data/294898 "2022-01-20T07:09:13Z")
**Posts on this page:** 3
**Page:** 1
### Author:  [@sujeetkp](https://discuss.elastic.co/u/sujeetkp)
#### Post date: [January 20, 2022, 7:09am UTC](https://discuss.elastic.co/t/multiple-config-files-duplicate-data/294898/1 "2022-01-20T07:09:13Z")
I am new to logstash and filebeat. I am trying to set up multiple config files for my logstash instance.
Using filebeat to send data to logstash. Even if I have filters created for both the logstash config files, I am getting duplicate data.
Logstash config file - 1:
```auto
input {
beats {
port => 5045
}
}
filter {
if [fields][env] == "prod" {
grok { match => { "message" => "%{LOGLEVEL:loglevel}] %{GREEDYDATA:message}$" }
overwrite => ["message"]
}
}
}
output {
stdout {
codec => rubydebug
}
elasticsearch {
hosts => ["https://172.17.0.2:9200"]
index => "logstash-myapp-%{+YYYY.MM.dd}"
user => "elastic"
password => "password"
ssl => true
cacert => "/usr/share/logstash/certs/http_ca.crt"
}
}
```
logstash config file-2
```auto
input {
beats {
port => 5044
}
}
filter {
if [fields][env] == "dev" {
grok { match => { "message" => "%{LOGLEVEL:loglevel}] %{GREEDYDATA:message}$" }
overwrite => ["message"]
}
}
}
output {
stdout {
codec => rubydebug
}
elasticsearch {
hosts => ["https://172.17.0.2:9200"]
index => "logstash-myapp-%{+YYYY.MM.dd}"
user => "elastic"
password => "password"
ssl => true
cacert => "/usr/share/logstash/certs/http_ca.crt"
}
}
```
Logfile Content:
```auto
[INFO] First Line
[INFO] Second Line
[INFO] Third Line
```
Filebeat config:
```auto
filebeat.inputs:
- type: filestream
enabled: true
paths:
- /root/data/logs/*.log
fields:
app: test
env: dev
output.logstash:
# The Logstash hosts
hosts: ["172.17.0.4:5044"]
```
I know that even if we have multiple files for config, logstash processes each and every line of the data against all the filters present in all the config files. Hence we have put filters in each of the config files for "fields.env".
I am expecting 3 lines to be sent to Elasticsearch because "fields.env" is "dev", but it is sending 6 lines to Elasticsearch and duplicate data.
Pleas help.
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [January 20, 2022, 3:31pm UTC](https://discuss.elastic.co/t/multiple-config-files-duplicate-data/294898/2 "2022-01-20T15:31:01Z")
The conditional only controls which grok filters are applied. All events are written to all outputs unconditionally.
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 17, 2022, 3:31pm UTC](https://discuss.elastic.co/t/multiple-config-files-duplicate-data/294898/3 "2022-02-17T15:31:20Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.