I am using logstash+ elastic-search+kibana framework for my project. I want
to create per tenant index in Elastic search. Is this a efficient way to
do?
Can i customize elastic search to look for a specific tag in logstash
output and create and store the data in particular index?
I am using logstash+ elastic-search+kibana framework for my project. I
want to create per tenant index in Elastic search. Is this a efficient way
to do?
I think it depends on how many indices you'll end up having. If you'll have
a huge amount of indices, they'll use up more space, more memory and more
file descriptors. Because you'll have more shards, and therefore more
segments. Although, you can tune the number of shards and the merge
policyhttp://www.elasticsearch.org/guide/reference/index-modules/merge/to
have less segments.
However, when you search for data from a single tenant, it should be faster
than if you mix all data up and do the same search by filtering on a field.
Because there's less data to search on.
Can i customize Elasticsearch to look for a specific tag in logstash
output and create and store the data in particular index?
I'm not sure exactly what you're after, but it sounds like you want to
index different types of data in different places. If that's so, I think
logstash is the place where you can configure, for example, the destination
index name to depend on a variable. People on the Logstash mailing list
will definitely know more about what you can do here.
I am using logstash+ elastic-search+kibana framework for my project. I
want to create per tenant index in Elastic search. Is this a efficient way
to do?
I think it depends on how many indices you'll end up having. If you'll
have a huge amount of indices, they'll use up more space, more memory and
more file descriptors. Because you'll have more shards, and therefore more
segments. Although, you can tune the number of shards and the merge policyhttp://www.elasticsearch.org/guide/reference/index-modules/merge/to have less segments.
However, when you search for data from a single tenant, it should be
faster than if you mix all data up and do the same search by filtering on a
field. Because there's less data to search on.
Can i customize Elasticsearch to look for a specific tag in logstash
output and create and store the data in particular index?
I'm not sure exactly what you're after, but it sounds like you want to
index different types of data in different places. If that's so, I think
logstash is the place where you can configure, for example, the destination
index name to depend on a variable. People on the Logstash mailing list
will definitely know more about what you can do here.
I was just wondering if you have succeeded in creating the multi tenant
indices?
What I'd want to avoid is to run as many agents/indexers as I have
tenants...
Op dinsdag 14 mei 2013 06:31:22 UTC+2 schreef freak 62:
On Sun, May 12, 2013 at 8:01 AM, <mail...@gmail.com <javascript:>> wrote:
I am using logstash+ elastic-search+kibana framework for my project. I
want to create per tenant index in Elastic search. Is this a efficient way
to do?
I think it depends on how many indices you'll end up having. If you'll
have a huge amount of indices, they'll use up more space, more memory and
more file descriptors. Because you'll have more shards, and therefore more
segments. Although, you can tune the number of shards and the merge
policyhttp://www.elasticsearch.org/guide/reference/index-modules/merge/to have less segments.
However, when you search for data from a single tenant, it should be
faster than if you mix all data up and do the same search by filtering on a
field. Because there's less data to search on.
Can i customize Elasticsearch to look for a specific tag in logstash
output and create and store the data in particular index?
I'm not sure exactly what you're after, but it sounds like you want to
index different types of data in different places. If that's so, I think
logstash is the place where you can configure, for example, the destination
index name to depend on a variable. People on the Logstash mailing list
will definitely know more about what you can do here.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.