# How to configure different indexes in logstash **URL:** \ **Category:** Logstash\ **Created:** [August 23, 2016, 9:55am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665 "2016-08-23T09:55:53Z")\ **Posts on this page:** 19\ **Page:** 2
**Author:** ![Vijayant\_Panda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vijayant_panda/32/18963_2.png) [@Vijayant\_Panda](https://discuss.elastic.co/u/Vijayant_Panda)\ **Post date:** [June 12, 2017, 7:49am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/21 "2017-06-12T07:49:09Z")
@magnus. the host in output used here is coming from logs parsed from filebeats?and the index "beats\_sit-%{+YYYY.MM.dd}" in output does this index template beats\_sit-\* has to be first created ? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 12, 2017, 8:01am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/22 "2017-06-12T08:01:04Z")
> the host in output used here is coming from logs parsed from filebeats? You mean the `host` field? Yes, that should be the Filebeat host that sends the data. > and the index "beats\_sit-%{+YYYY.MM.dd}" in output does this index template beats\_sit-\* has to be first created ? Yes, you should probably have an index template that matches the name of your indexes. ---
**Author:** ![Vijayant\_Panda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vijayant_panda/32/18963_2.png) [@Vijayant\_Panda](https://discuss.elastic.co/u/Vijayant_Panda)\ **Post date:** [June 12, 2017, 9:00am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/23 "2017-06-12T09:00:36Z")
Thanks magnus for [reply.so](http://reply.so) in this case "beats\_dev-%{+YYYY.MM.dd}" index name . What index template I should create and how ? Can you post the index for this -beats\_dev-%{+YYYY.MM.dd} and its path location in logstash? ---
**Author:** ![Vijayant\_Panda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vijayant_panda/32/18963_2.png) [@Vijayant\_Panda](https://discuss.elastic.co/u/Vijayant_Panda)\ **Post date:** [June 12, 2017, 9:15am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/24 "2017-06-12T09:15:15Z")
if [host] in ["[example1.sit.com](http://example1.sit.com)", "[example2.sit.com](http://example2.sit.com)"] ... the host here is coming after filebeat parsing? so here host contains the host name where the respective event log is coming from filebeat client ?my confusion is in my json logs itself i have a field called hostname .any way I can use the hostname from the logs ? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 12, 2017, 9:39am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/25 "2017-06-12T09:39:00Z")
> What index template I should create and how ? You can make a copy of the existing template file (which you'll find in the Logstash installation directory), adjust the `template` field that controls which indexes it applies to, and configure Logstash's elasticsearch output to use it. Another way of obtaining the current template is by fetching it directly from ES via the get index template REST API. The template uploaded by Logstash is by default named "logstash". Since you are a beginner it would be easier if you just stuck to the defaults and called your index logstash-\*, e.g. logstash-beats\_dev-2017.06.12. Once you learn more you can start customizing things. > the host here is coming after filebeat parsing? so here host contains the host name where the respective event log is coming from filebeat client ? I believe so, yes. That should be easy to verify, shouldn't it? > my confusion is in my json logs itself i have a field called hostname .any way I can use the hostname from the logs ? Yes, of course. ---
**Author:** ![Vijayant\_Panda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vijayant_panda/32/18963_2.png) [@Vijayant\_Panda](https://discuss.elastic.co/u/Vijayant_Panda)\ **Post date:** [June 12, 2017, 11:45am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/26 "2017-06-12T11:45:56Z")
ok thanks for your reply magnus. Here is what i am going to do- 1.i took the default template from \logstash-5.4.1\vendor\bundle\jruby\1.9\gems\logstash-output-elasticsearch-7.3.1-java\lib\logstash\outputs\elasticsearch\elasticsearch template es 2x.json 1. i am going to use this default template and rename this json file as mytemplate.json and inside that going to rename existing "template" : "logstash-_" to "template" :"logstash-beats\_dev-_". 2. then in logstash output { elasticsearch { hosts =\> ["elasticserver:9200"] template =\> "/logstash/logstash-5.4.1/templates/mytemplate.json" template\_name =\> "logstash-beats\_dev-\*" template\_overwrite =\> true } stdout{} } Please tell me if i am missing i any steps here or I need to do anything else.thanks a lot for patience ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 12, 2017, 12:42pm UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/27 "2017-06-12T12:42:02Z")
> i am going to use this default template and rename this json file as mytemplate.json and inside that going to rename existing "template" : "logstash-" to "template" :"logstash-beats\_dev-". If your index names match logstash-\* then you don't need to create your own index template. > then in logstash output > { > elasticsearch { > hosts =\> ["elasticserver:9200"] > template =\> "/logstash/logstash-5.4.1/templates/mytemplate.json" > template\_name =\> "logstash-beats\_dev-\*" > template\_overwrite =\> true > } But you're not setting the index name to anything so you're getting the default, logstash-%{+YYYY.MM.DD}. ---
**Author:** ![Vijayant\_Panda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vijayant_panda/32/18963_2.png) [@Vijayant\_Panda](https://discuss.elastic.co/u/Vijayant_Panda)\ **Post date:** [June 12, 2017, 1:21pm UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/28 "2017-06-12T13:21:18Z")
ok .where should i set the index name as logstash-beats\_dev ? my requirement is that I should redirect all events to index logstash-beats\_dev\* based on hostname ?if different hostname=sit or hostname=test it should create respective index like logstash-beats\_sit\* or logstash-beats\_test\*.....? Can you please elaborate how it works?thanks for your patience ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 12, 2017, 2:07pm UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/29 "2017-06-12T14:07:38Z")
> ok .where should i set the index name as logstash-beats\_dev ? The elasticsearch output's `index` option sets the index name. > my requirement is that I should redirect all events to index logstash-beats\_dev\* based on hostname ?if different hostname=sit or hostname=test it should create respective index like logstash-beats\_sit\* or logstash-beats\_test\*.....? ```plaintext elasticsearch { ... index => "logstash-beats_%{host}-%{+YYYY.MM.dd}" } ``` This assumes that - the field `host` contains the name of the host that generated the event and - that you have a very small number of hosts since you otherwise will create a large number of indexes. ---
**Author:** ![Vijayant\_Panda](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vijayant_panda/32/18963_2.png) [@Vijayant\_Panda](https://discuss.elastic.co/u/Vijayant_Panda)\ **Post date:** [June 12, 2017, 5:32pm UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/30 "2017-06-12T17:32:41Z")
ok thanks for reply.Please correct me -or let me know if anything more is needed Will the logstash conf get the {host} field from filebeats automatically ?if yes then the index would be logstash-beats\_dev.com-2017.12.06 ? What index name I have to use in kibana UI so as to separate the logs viewing based on logstash-beats\_dev.com-\* or logstash-beats\_test.com-\* ..... So my logstash.conf is input { beats { port =\> 5044 } } filter{ json{ source =\> "message" target =\> "tweet" } } output { elasticsearch { hosts =\> ["elasticserver:9200"] index =\> "logstash-beats\_%{host}-%{+YYYY.MM.dd}" } stdout { codec =\> rubydebug } } filebeats.yml from various hosts - - input\_type: log #----------------------------- Logstash output -------------------------------- output.logstash: # The Logstash hosts hosts: ["localhost:5044"] ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [June 12, 2017, 8:09pm UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/31 "2017-06-12T20:09:11Z")
> Will the logstash conf get the {host} field from filebeats automatically ? I think so, but as I said you can easily verify it yourself. > if yes then the index would be logstash-beats\_dev.com-2017.12.06 ? Yes, if `host` contains "[dev.com](http://dev.com)" and the event is from the future (Dec 6, 2017). > What index name I have to use in kibana UI so as to separate the logs viewing based on logstash-beats\_dev.com-\* or logstash-beats\_test.com-\* ..... You can set up multiple index patterns that each match a different index name prefix. ---
**Author:** ![sbampa](https://avatars.discourse-cdn.com/v4/letter/s/8e7dd6/32.png) [@sbampa](https://discuss.elastic.co/u/sbampa)\ **Post date:** [October 12, 2017, 8:56am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/32 "2017-10-12T08:56:48Z")
Hi all, this is an old topic, but i need an help on it. I have the following configuration: input { beats { port =\> "5043" } } filter { grok { match =\> ["message", "%{TIMESTAMP\_ISO8601:timestamp} %{LOGLEVEL:level} "] match =\> ["message", "%{HTTPDERROR\_DATE:timestamp} (%{NUMBER:bytes}) "] match =\> ["message", "%{TIME:timestamp} %{LOGLEVEL:level}"] } date { match =\> ["timestamp", "YYYY-MM-dd HH:mm:ss,SSS", "HH:mm:ss,SSS", "EEE MMM dd HH:mm:ss YYYY"] timezone =\> "Europe/Rome" target =\> "@timestamp" } mutate { remove\_field =\> ["timestamp","offset","level","@version","input\_type","beat","type","tags","sort","fields"] } } output { if [host] in ["olinux7-template.farm.eurobet-italia.local"]{ elasticsearch { hosts =\> ["172.16.19.216:9200", "172.16.19.217:9200", "172.16.19.218:9200", "172.16.19.220:9200"] user =\> elastic password =\> changeme manage\_template =\> false index =\> "prova-%{+YYYY.MM.dd}" document\_type =\> "%{[@metadata][type]}" } } else { elasticsearch { hosts =\> ["172.16.19.216:9200", "172.16.19.217:9200", "172.16.19.218:9200", "172.16.19.220:9200"] user =\> elastic password =\> changeme manage\_template =\> false index =\> "sito-gioco-%{+YYYY.MM.dd}" document\_type =\> "%{[@metadata][type]}" } } } But, the index prova-%{........} is never created. can i use the "host" keyword in the if statement? If not, can you help me with some tips? Thanks, BR ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 12, 2017, 9:46am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/33 "2017-10-12T09:46:30Z")
Please start a new thread for your problem. ---
**Author:** ![sbampa](https://avatars.discourse-cdn.com/v4/letter/s/8e7dd6/32.png) [@sbampa](https://discuss.elastic.co/u/sbampa)\ **Post date:** [October 12, 2017, 9:58am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/34 "2017-10-12T09:58:10Z")
Hi, solved adding "fields" in filebeat config and using it in the logstash output with if statement, BR ---
**Author:** ![Swaroop\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/swaroop_chand/32/45812_2.png) [@Swaroop\_Chand](https://discuss.elastic.co/u/Swaroop_Chand)\ **Post date:** [May 26, 2019, 8:50am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/35 "2019-05-26T08:50:02Z")
input { beats { port =\> 5044 } } filter { if [type] == "syslog" { grok { match =\> { "message" =\> "%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_hostname} %{DATA:syslog\_program}(?:[%{POSINT:syslog\_pid}])?: %{GREEDYDATA:syslog\_message}" } } date { match =\> ["syslog\_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"] } } } output { elasticsearch { hosts =\> ["[https://localhost:9200](https://localhost:9200)"] index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}" document\_type =\> "%{[@metadata][type]}" } } After running "service logstash start" new index is not created.Please Help. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [May 26, 2019, 8:56am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/36 "2019-05-26T08:56:39Z")
Please open a new thread if your own and provide more details, e.g. output from log files. ---
**Author:** ![Swaroop\_Chand](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/swaroop_chand/32/45812_2.png) [@Swaroop\_Chand](https://discuss.elastic.co/u/Swaroop_Chand)\ **Post date:** [May 26, 2019, 9:50am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/37 "2019-05-26T09:50:10Z")
Thank you.created in thread.Please have look. > [@How to configure indexes in logstash](https://discuss.elastic.co/t/how-to-configure-indexes-in-logstash/182792): > > input { beats { port =\> 5044 } } filter { if [type] == "syslog" { grok { match =\> { "message" =\> "%{SYSLOGTIMESTAMP:syslog\_timestamp} %{SYSLOGHOST:syslog\_hostname} %{DATA:syslog\_program}(?:[%{POSINT:syslog\_pid}])?: %{GREEDYDATA:syslog\_message}" } } date { match =\> ["syslog\_timestamp", "MMM d HH:mm:ss", "MMM dd HH:mm:ss"] } } } output { elasticsearch { hosts =\> ["[https://localhost:9200](https://localhost:9200/)"] index =\> "%{[@metadata][beat]}-%{+YYYY.MM.dd}" document\_type =\> "%{[@metadata][type]}" } … ---
**Author:** ![rajan.kumar](https://avatars.discourse-cdn.com/v4/letter/r/278dde/32.png) [@rajan.kumar](https://discuss.elastic.co/u/rajan.kumar)\ **Post date:** [May 23, 2020, 3:07pm UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/38 "2020-05-23T15:07:11Z")
@magnusbaeck : I wanted to put an "if-else" block in my logstash conf file to send data in discrete indices to ES on the basis of IPs,but its not working in my case.Please have a look into my conf file: input { beats { port =\> 5044 } } output { if [host] == "[http://192.168.43.154:9200](http://192.168.43.154:9200)" { elasticsearch { index =\> "travlogs" ``` } ``` } } ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 4, 2022, 4:26am UTC](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665/39 "2022-11-04T04:26:54Z")
[Previous page](https://discuss.elastic.co/t/how-to-configure-different-indexes-in-logstash/58665.md?page=1)