# Logstash with input from file and input from beats (port 5044) create two indices with the same data

**URL:** <https://discuss.elastic.co/t/logstash-with-input-from-file-and-input-from-beats-port-5044-create-two-indices-with-the-same-data/191348>\
**Category:** Logstash\
**Created:** [July 19, 2019, 8:28am UTC](https://discuss.elastic.co/t/logstash-with-input-from-file-and-input-from-beats-port-5044-create-two-indices-with-the-same-data/191348 "2019-07-19T08:28:34Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![izhael](https://avatars.discourse-cdn.com/v4/letter/i/b19c9b/32.png) [@izhael](https://discuss.elastic.co/u/izhael)\
**Post date:** [July 19, 2019, 8:28am UTC](https://discuss.elastic.co/t/logstash-with-input-from-file-and-input-from-beats-port-5044-create-two-indices-with-the-same-data/191348/1 "2019-07-19T08:28:35Z")

</div>

Good morning, I have two configuration files in Logstash conf directory:

First file:

**File: firewall.conf**  
\*\* Logs from Firewall \*\*

input {  
file {  
path =\> "/var/log/remote/firewall/\*-firewall.log"  
ignore\_older =\> 60  
tags =\> ["firewall-log"]  
}  
}  
...  
...  
...  
output {  
if "firewall-log" in [tags] {  
elasticsearch {  
index =\> "firewall-log-%{+YYYY.MM}"  
hosts =\> ["localhost: 9200"]  
}  
}

Second file:

\*\* File: beats.conf \*\*  
\*\* Logs from auditlogbeat and winlogbeat \*\*

input {  
beats {  
port =\> 5044  
}  
}

output {  
elasticsearch {  
hosts =\> ["[http://localhost:9200](http://localhost:9200)"]  
index =\> "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM}"  
document\_type =\> "%{[@metadata][type]}"  
}  
}

With this configuration, two indexes for the firewall data appear in Kibana. The first of them firewall-log-2019-07 with the data of the log file specified in the input section of firewall.conf. The funny thing is that a second index called %{[@metadata] [beat]}-{[@metadata] [version]}-%{2019-07} is created and with the same data as firewall-log-2019.07 index. I don't undestand the reason because in the input section of beats.conf I specified beats and listen from port 5044 and however Logstash is taking data from the firewall log file.  
To solve the problem I have changed the output section in beats.conf file and I have added a if clause:

output {  
if "beats\_input\_codec\_plain\_applied" in [tags] {  
elasticsearch {  
hosts =\> ["[http://localhost:9200](http://localhost:9200)"]  
index =\> "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM}"  
document\_type =\> "%{[@metadata][type]}"  
}  
}  
}

I don't know If this is a good solution, but this way the index %{[@metadata] [beat]}-{[@metadata] [version]}-% {2019-07} is not created and the data from the firewall log file is loaded only in the firewall-log-2019.07 index.

Thank you!!

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 19, 2019, 12:56pm UTC](https://discuss.elastic.co/t/logstash-with-input-from-file-and-input-from-beats-port-5044-create-two-indices-with-the-same-data/191348/2 "2019-07-19T12:56:31Z")

</div>

Please read [this](https://discuss.elastic.co/t/logstash-setting-up-multiple-config-files-in-one-pipeline/190643/2) post.

---

<div class="post-metadata">

**Author:** ![izhael](https://avatars.discourse-cdn.com/v4/letter/i/b19c9b/32.png) [@izhael](https://discuss.elastic.co/u/izhael)\
**Post date:** [July 25, 2019, 9:38pm UTC](https://discuss.elastic.co/t/logstash-with-input-from-file-and-input-from-beats-port-5044-create-two-indices-with-the-same-data/191348/3 "2019-07-25T21:38:26Z")

</div>

Now I got it. Thank you for your help.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 22, 2019, 9:38pm UTC](https://discuss.elastic.co/t/logstash-with-input-from-file-and-input-from-beats-port-5044-create-two-indices-with-the-same-data/191348/4 "2019-08-22T21:38:37Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.