# Multiple filebeat input in logstash but how to create different set of index for them **URL:** \ **Category:** Logstash\ **Created:** [January 30, 2019, 12:53pm UTC](https://discuss.elastic.co/t/multiple-filebeat-input-in-logstash-but-how-to-create-different-set-of-index-for-them/166358 "2019-01-30T12:53:45Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![MiddlewareTeam](https://avatars.discourse-cdn.com/v4/letter/m/8e8cbc/32.png) [@MiddlewareTeam](https://discuss.elastic.co/u/MiddlewareTeam)\ **Post date:** [January 30, 2019, 12:53pm UTC](https://discuss.elastic.co/t/multiple-filebeat-input-in-logstash-but-how-to-create-different-set-of-index-for-them/166358/1 "2019-01-30T12:53:46Z")
I have two set of environment consider as A and B. I have configured A env in logstash.conf and its working as expected but now I have to add B in the same file but I have to create different index for the A and B. How to achieve it? > [@Create multiple indexes with same logstash config file](https://discuss.elastic.co/t/create-multiple-indexes-with-same-logstash-config-file/97456): > > Hello, I am looking for some help in creating a muliple indexes in ES with just single logstash config file. All dev logs should go to dev index , all sys logs should to sys index and uat logs to uat index. The logstash input will be just running on single port example 5043 and the filter remains same for all env, Is there way we can configure only the output sections and route the logs based on hosts ? or any other way ? . I am putting logstash config below , any help will greatly appreciated… I have gone through above link but the index is not working with the same. Below is the logstash.conf I am using \< input { beats { port =\> "5044" } } filter { if [fields][log\_type] in ["apache\_access" , "apache-access"] and [fields][application] == "application" and [fields][env] == "A" { grok { match =\> [ "message" , "%{IP:access-ip1} %{IP:access-ip2} - - [%{NOTSPACE:access-timestamp} +%{INT}] "%{WORD:access-httpmethod} %{NOTSPACE:access-request} %{WORD:access-protocol}/%{NUMBER:access-protocolversion}" %{INT:access-status} %{INT:access-responsesize} %{INT:access-responsetime} "-" "%{WORD} %{WORD} %{NOTSPACE}" [ %{WORD} %{WORD} %{WORD}= %{INT:access-responsetimeinmicrosec}%{GREEDYDATA}" ] overwrite =\> ["message"] } ``` if [fields][log_type] in ["apache_access" , "apache-access"] and [fields][application] == "application" and [fields][env] == "B" { grok { "message" , "%{IP:access-ip1} %{IP:access-ip2} \- \- \[%{NOTSPACE:access-timestamp} \+%{INT}\] \"%{WORD:access-httpmethod} %{NOTSPACE:access-request} %{WORD:access-protocol}/%{NUMBER:access-protocolversion}\" %{INT:access-status} %{INT:access-responsesize} %{INT:access-responsetime} \"\-\" \"%{WORD} %{WORD} %{NOTSPACE}\" \[ %{WORD} %{WORD} %{WORD}\= %{INT:access-responsetimeinmicrosec}%{GREEDYDATA}" overwrite => ["message"] } } ``` } output { if [fields][log\_type] in ["apache\_access" , "apache-access"] and [fields][application] == "application" and [fields][env] == "A" { elasticsearch { ... index =\> "A" } } else { elasticsearch { ... index =\> "B" } } } /\> ---
**Author:** ![guyboertje](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/guyboertje/32/31592_2.png) [@guyboertje](https://discuss.elastic.co/u/guyboertje)\ **Post date:** [January 30, 2019, 2:30pm UTC](https://discuss.elastic.co/t/multiple-filebeat-input-in-logstash-but-how-to-create-different-set-of-index-for-them/166358/2 "2019-01-30T14:30:58Z")
You can use interpolation in the index setting of the Elasticsearch output - meaning that some field or metadata field in your event holds part of the index name. So those events from "A" could have a field (add\_field in beats input) called say `index_suffix` with value "a" and events from "B" have the same field with value "b". `index => "logstash-%{[index_suffix]}"` ---
**Author:** ![MiddlewareTeam](https://avatars.discourse-cdn.com/v4/letter/m/8e8cbc/32.png) [@MiddlewareTeam](https://discuss.elastic.co/u/MiddlewareTeam)\ **Post date:** [January 31, 2019, 12:07pm UTC](https://discuss.elastic.co/t/multiple-filebeat-input-in-logstash-but-how-to-create-different-set-of-index-for-them/166358/3 "2019-01-31T12:07:11Z")
Thanks for your reply guyboertje. Can you or someone please suggest how can we implement below scenario (master slave in logstash): Create one A.conf for env A, create B.conf for env B with required filter and output details. And then will create one master.conf which will be having input configuration of logstash along with source of both env.conf over there. Is it possible to have this kind of configuration? and how can we achieve this? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [January 31, 2019, 1:13pm UTC](https://discuss.elastic.co/t/multiple-filebeat-input-in-logstash-but-how-to-create-different-set-of-index-for-them/166358/4 "2019-01-31T13:13:25Z")
If you point -f at a directory then logstash will concatente all the files in it to build the configuration. So you might have a set of files.... ```auto 01input.conf 10env-qa.conf 20env-dev.conf ``` The input would be common and all the processing and output for qa/dev would have to be conditional based on tags or some other field. ---
**Author:** ![MiddlewareTeam](https://avatars.discourse-cdn.com/v4/letter/m/8e8cbc/32.png) [@MiddlewareTeam](https://discuss.elastic.co/u/MiddlewareTeam)\ **Post date:** [February 1, 2019, 11:12am UTC](https://discuss.elastic.co/t/multiple-filebeat-input-in-logstash-but-how-to-create-different-set-of-index-for-them/166358/5 "2019-02-01T11:12:15Z")
Thanks Badger, I tried that and got successful as well. One more query 1. Do we need different port in input tag to use multiple indexes? like below \< - pipeline.id: my-pipeline\_1 path.config: "/etc/path/to/A.config" - pipeline.id: my-other-pipeline path.config: "/etc/different/path/B.cfg" /\> and then ``` < #A.cfg input { beats { port => 5044 } } filter { dissect { ... } } output { elasticsearch { IP:port} index A} #B.cfg input { tcp { port => 5045 } } filter { grok { ... } } output { elasticsearch { ... } index B} ``` /\> ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [February 1, 2019, 1:52pm UTC](https://discuss.elastic.co/t/multiple-filebeat-input-in-logstash-but-how-to-create-different-set-of-index-for-them/166358/6 "2019-02-01T13:52:15Z")
You cannot have two pipelines listening on the same port, so yes, you need to use different ports. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [March 6, 2019, 1:46pm UTC](https://discuss.elastic.co/t/multiple-filebeat-input-in-logstash-but-how-to-create-different-set-of-index-for-them/166358/8 "2019-03-06T13:46:41Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.