# Multiple indices in elastic search with logstash output **URL:** \ **Category:** Elasticsearch\ **Created:** [May 12, 2013, 5:01am UTC](https://discuss.elastic.co/t/multiple-indices-in-elastic-search-with-logstash-output/11923 "2013-05-12T05:01:56Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![f61](https://avatars.discourse-cdn.com/v4/letter/f/82dd89/32.png) [@f61](https://discuss.elastic.co/u/f61)\ **Post date:** [May 12, 2013, 5:01am UTC](https://discuss.elastic.co/t/multiple-indices-in-elastic-search-with-logstash-output/11923/1 "2013-05-12T05:01:56Z")
I am using logstash+ elastic-search+kibana framework for my project. I want to create per tenant index in Elastic search. Is this a efficient way to do? Can i customize elastic search to look for a specific tag in logstash output and create and store the data in particular index? -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![radu\_gheorghe](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/radu_gheorghe/32/556_2.png) [@radu\_gheorghe](https://discuss.elastic.co/u/radu_gheorghe)\ **Post date:** [May 13, 2013, 6:46am UTC](https://discuss.elastic.co/t/multiple-indices-in-elastic-search-with-logstash-output/11923/2 "2013-05-13T06:46:47Z")
Hello, On Sun, May 12, 2013 at 8:01 AM, [mailtov16@gmail.com](mailto:mailtov16@gmail.com) wrote: > I am using logstash+ elastic-search+kibana framework for my project. I > want to create per tenant index in Elastic search. Is this a efficient way > to do? I think it depends on how many indices you'll end up having. If you'll have a huge amount of indices, they'll use up more space, more memory and more file descriptors. Because you'll have more shards, and therefore more segments. Although, you can tune the number of shards and the merge policy[http://www.elasticsearch.org/guide/reference/index-modules/merge/](http://www.elasticsearch.org/guide/reference/index-modules/merge/)to have less segments. However, when you search for data from a single tenant, it should be faster than if you mix all data up and do the same search by filtering on a field. Because there's less data to search on. > Can i customize Elasticsearch to look for a specific tag in logstash > output and create and store the data in particular index? I'm not sure exactly what you're after, but it sounds like you want to index different types of data in different places. If that's so, I think logstash is the place where you can configure, for example, the destination index name to depend on a variable. People on the Logstash mailing list will definitely know more about what you can do here. ## Best regards, Radu [http://sematext.com/](http://sematext.com/) -- Elasticsearch -- Solr -- Lucene -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![f61](https://avatars.discourse-cdn.com/v4/letter/f/82dd89/32.png) [@f61](https://discuss.elastic.co/u/f61)\ **Post date:** [May 14, 2013, 4:31am UTC](https://discuss.elastic.co/t/multiple-indices-in-elastic-search-with-logstash-output/11923/3 "2013-05-14T04:31:22Z")
Thnx Radu On Mon, May 13, 2013 at 12:16 PM, Radu Gheorghe [radu.gheorghe@sematext.com](mailto:radu.gheorghe@sematext.com)wrote: > Hello, > > On Sun, May 12, 2013 at 8:01 AM, [mailtov16@gmail.com](mailto:mailtov16@gmail.com) wrote: > > > I am using logstash+ elastic-search+kibana framework for my project. I > > want to create per tenant index in Elastic search. Is this a efficient way > > to do? > > I think it depends on how many indices you'll end up having. If you'll > have a huge amount of indices, they'll use up more space, more memory and > more file descriptors. Because you'll have more shards, and therefore more > segments. Although, you can tune the number of shards and the merge policy[http://www.elasticsearch.org/guide/reference/index-modules/merge/](http://www.elasticsearch.org/guide/reference/index-modules/merge/)to have less segments. > > However, when you search for data from a single tenant, it should be > faster than if you mix all data up and do the same search by filtering on a > field. Because there's less data to search on. > > > Can i customize Elasticsearch to look for a specific tag in logstash > > output and create and store the data in particular index? > > I'm not sure exactly what you're after, but it sounds like you want to > index different types of data in different places. If that's so, I think > logstash is the place where you can configure, for example, the destination > index name to depend on a variable. People on the Logstash mailing list > will definitely know more about what you can do here. > > ## Best regards, Radu > > [http://sematext.com/](http://sematext.com/) -- Elasticsearch -- Solr -- Lucene > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/zss4kvLMkNw/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/zss4kvLMkNw/unsubscribe?hl=en-US) > . > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![Jaap\_H](https://avatars.discourse-cdn.com/v4/letter/j/c57346/32.png) [@Jaap\_H](https://discuss.elastic.co/u/Jaap_H)\ **Post date:** [December 9, 2013, 2:31pm UTC](https://discuss.elastic.co/t/multiple-indices-in-elastic-search-with-logstash-output/11923/4 "2013-12-09T14:31:57Z")
I was just wondering if you have succeeded in creating the multi tenant indices? What I'd want to avoid is to run as many agents/indexers as I have tenants... Op dinsdag 14 mei 2013 06:31:22 UTC+2 schreef freak 62: > Thnx Radu > > On Mon, May 13, 2013 at 12:16 PM, Radu Gheorghe \<[radu.g...@sematext.com](mailto:radu.g...@sematext.com)\ > > > wrote: > > > Hello, > > > > On Sun, May 12, 2013 at 8:01 AM, \<[mail...@gmail.com](mailto:mail...@gmail.com) \\> wrote: > > > > > I am using logstash+ elastic-search+kibana framework for my project. I > > > want to create per tenant index in Elastic search. Is this a efficient way > > > to do? > > > > I think it depends on how many indices you'll end up having. If you'll > > have a huge amount of indices, they'll use up more space, more memory and > > more file descriptors. Because you'll have more shards, and therefore more > > segments. Although, you can tune the number of shards and the merge > > policy[http://www.elasticsearch.org/guide/reference/index-modules/merge/](http://www.elasticsearch.org/guide/reference/index-modules/merge/)to have less segments. > > > > However, when you search for data from a single tenant, it should be > > faster than if you mix all data up and do the same search by filtering on a > > field. Because there's less data to search on. > > > > > Can i customize Elasticsearch to look for a specific tag in logstash > > > output and create and store the data in particular index? > > > > I'm not sure exactly what you're after, but it sounds like you want to > > index different types of data in different places. If that's so, I think > > logstash is the place where you can configure, for example, the destination > > index name to depend on a variable. People on the Logstash mailing list > > will definitely know more about what you can do here. > > > > ## Best regards, Radu > > > > [http://sematext.com/](http://sematext.com/) -- Elasticsearch -- Solr -- Lucene > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/zss4kvLMkNw/unsubscribe?hl=en-US](https://groups.google.com/d/topic/elasticsearch/zss4kvLMkNw/unsubscribe?hl=en-US) > > . > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/2ecfe37f-141a-4b3c-80e0-fb6176ff5a66%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/2ecfe37f-141a-4b3c-80e0-fb6176ff5a66%40googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:02am UTC](https://discuss.elastic.co/t/multiple-indices-in-elastic-search-with-logstash-output/11923/5 "2017-07-06T02:02:27Z")