We are planning to use ELK for our log analysis. We have multiple data
centers. Since it is not recommended to have across data center cluster, we
are going to have one ES cluster per data center, here are the three
design options we have:
Use snapshot & restore to replicate data across clusters.
Use tribe node to achieve across cluster queries
Ship and index logs to each cluster
Here are our questions, and any comments will be appreciated:
How complex is snapshot & restore, anyone has experience on this purpose?
Would the performance of only one tribe node be a concern or bottleneck,
is it possible to have multiple tribe nodes for scale up or load balancing?
Is it possible to customize Kibana so that it can go to different
cluster to query data depends on the query?
1 - It's pretty simple and has been used before.
2 - it can be yes. You can have multiple tribe nodes though.
3 - This may be possible but you'd have to hack a fair bit of code, so it's
not really practical.
We are planning to use ELK for our log analysis. We have multiple data
centers. Since it is not recommended to have across data center cluster, we
are going to have one ES cluster per data center, here are the three
design options we have:
Use snapshot & restore to replicate data across clusters.
Use tribe node to achieve across cluster queries
Ship and index logs to each cluster
Here are our questions, and any comments will be appreciated:
How complex is snapshot & restore, anyone has experience on this
purpose?
Would the performance of only one tribe node be a concern or
bottleneck, is it possible to have multiple tribe nodes for scale up or
load balancing?
Is it possible to customize Kibana so that it can go to different
cluster to query data depends on the query?
Thank you for your reply. Is there any existing approach for kibana to
communicate with multiple tribe nodes? Or is it something we should
implement by ourselves by customizing kibana?
Thank you!
Abigail
On Tuesday, March 10, 2015 at 8:56:25 PM UTC-4, Mark Walkom wrote:
1 - It's pretty simple and has been used before.
2 - it can be yes. You can have multiple tribe nodes though.
3 - This may be possible but you'd have to hack a fair bit of code, so
it's not really practical.
On 10 March 2015 at 13:00, Alex <nay...@gmail.com <javascript:>> wrote:
Hi all,
We are planning to use ELK for our log analysis. We have multiple data
centers. Since it is not recommended to have across data center cluster, we
are going to have one ES cluster per data center, here are the three
design options we have:
Use snapshot & restore to replicate data across clusters.
Use tribe node to achieve across cluster queries
Ship and index logs to each cluster
Here are our questions, and any comments will be appreciated:
How complex is snapshot & restore, anyone has experience on this
purpose?
Would the performance of only one tribe node be a concern or
bottleneck, is it possible to have multiple tribe nodes for scale up or
load balancing?
Is it possible to customize Kibana so that it can go to different
cluster to query data depends on the query?
Why not load balance multiple tribe nodes, if you need multiple.
On Wednesday, March 11, 2015 at 9:41:39 AM UTC-6, Abigail wrote:
Hi Mark,
Thank you for your reply. Is there any existing approach for kibana to
communicate with multiple tribe nodes? Or is it something we should
implement by ourselves by customizing kibana?
Thank you!
Abigail
On Tuesday, March 10, 2015 at 8:56:25 PM UTC-4, Mark Walkom wrote:
1 - It's pretty simple and has been used before.
2 - it can be yes. You can have multiple tribe nodes though.
3 - This may be possible but you'd have to hack a fair bit of code, so
it's not really practical.
We are planning to use ELK for our log analysis. We have multiple data
centers. Since it is not recommended to have across data center cluster, we
are going to have one ES cluster per data center, here are the three
design options we have:
Use snapshot & restore to replicate data across clusters.
Use tribe node to achieve across cluster queries
Ship and index logs to each cluster
Here are our questions, and any comments will be appreciated:
How complex is snapshot & restore, anyone has experience on this
purpose?
Would the performance of only one tribe node be a concern or
bottleneck, is it possible to have multiple tribe nodes for scale up or
load balancing?
Is it possible to customize Kibana so that it can go to different
cluster to query data depends on the query?
Why not load balance multiple tribe nodes, if you need multiple.
On Wednesday, March 11, 2015 at 9:41:39 AM UTC-6, Abigail wrote:
Hi Mark,
Thank you for your reply. Is there any existing approach for kibana to
communicate with multiple tribe nodes? Or is it something we should
implement by ourselves by customizing kibana?
Thank you!
Abigail
On Tuesday, March 10, 2015 at 8:56:25 PM UTC-4, Mark Walkom wrote:
1 - It's pretty simple and has been used before.
2 - it can be yes. You can have multiple tribe nodes though.
3 - This may be possible but you'd have to hack a fair bit of code, so
it's not really practical.
We are planning to use ELK for our log analysis. We have multiple data
centers. Since it is not recommended to have across data center cluster, we
are going to have one ES cluster per data center, here are the three
design options we have:
Use snapshot & restore to replicate data across clusters.
Use tribe node to achieve across cluster queries
Ship and index logs to each cluster
Here are our questions, and any comments will be appreciated:
How complex is snapshot & restore, anyone has experience on this
purpose?
Would the performance of only one tribe node be a concern or
bottleneck, is it possible to have multiple tribe nodes for scale up or
load balancing?
Is it possible to customize Kibana so that it can go to different
cluster to query data depends on the query?
Why not load balance multiple tribe nodes, if you need multiple.
On Wednesday, March 11, 2015 at 9:41:39 AM UTC-6, Abigail wrote:
Hi Mark,
Thank you for your reply. Is there any existing approach for kibana to
communicate with multiple tribe nodes? Or is it something we should
implement by ourselves by customizing kibana?
Thank you!
Abigail
On Tuesday, March 10, 2015 at 8:56:25 PM UTC-4, Mark Walkom wrote:
1 - It's pretty simple and has been used before.
2 - it can be yes. You can have multiple tribe nodes though.
3 - This may be possible but you'd have to hack a fair bit of code, so
it's not really practical.
We are planning to use ELK for our log analysis. We have multiple data
centers. Since it is not recommended to have across data center cluster, we
are going to have one ES cluster per data center, here are the three
design options we have:
Use snapshot & restore to replicate data across clusters.
Use tribe node to achieve across cluster queries
Ship and index logs to each cluster
Here are our questions, and any comments will be appreciated:
How complex is snapshot & restore, anyone has experience on this
purpose?
Would the performance of only one tribe node be a concern or
bottleneck, is it possible to have multiple tribe nodes for scale up or
load balancing?
Is it possible to customize Kibana so that it can go to different
cluster to query data depends on the query?
Perhaps you are misunderstanding me. Elasticsearch does not provide a load
balancer for this purpose. You would use a typical HTTP load balancer
which could be anything as simple as Nginx, to something costly and
expensive like a NetScalar. Configuring such a loadbalancer I believe is
outside the scope of this list.
On Thursday, March 12, 2015 at 11:16:12 AM UTC-6, Abigail wrote:
Yes, that is what I meant. Is there any reference for set up the load
balance for Kibana 4? Or if it is easier for Kibana 3?
On Thu, Mar 12, 2015 at 12:26 PM, <aa...@definemg.com <javascript:>>
wrote:
Why not load balance multiple tribe nodes, if you need multiple.
On Wednesday, March 11, 2015 at 9:41:39 AM UTC-6, Abigail wrote:
Hi Mark,
Thank you for your reply. Is there any existing approach for kibana to
communicate with multiple tribe nodes? Or is it something we should
implement by ourselves by customizing kibana?
Thank you!
Abigail
On Tuesday, March 10, 2015 at 8:56:25 PM UTC-4, Mark Walkom wrote:
1 - It's pretty simple and has been used before.
2 - it can be yes. You can have multiple tribe nodes though.
3 - This may be possible but you'd have to hack a fair bit of code, so
it's not really practical.
We are planning to use ELK for our log analysis. We have multiple data
centers. Since it is not recommended to have across data center cluster, we
are going to have one ES cluster per data center, here are the three
design options we have:
Use snapshot & restore to replicate data across clusters.
Use tribe node to achieve across cluster queries
Ship and index logs to each cluster
Here are our questions, and any comments will be appreciated:
How complex is snapshot & restore, anyone has experience on this
purpose?
Would the performance of only one tribe node be a concern or
bottleneck, is it possible to have multiple tribe nodes for scale up or
load balancing?
Is it possible to customize Kibana so that it can go to different
cluster to query data depends on the query?
Perhaps you are misunderstanding me. Elasticsearch does not provide a
load balancer for this purpose. You would use a typical HTTP load balancer
which could be anything as simple as Nginx, to something costly and
expensive like a NetScalar. Configuring such a loadbalancer I believe is
outside the scope of this list.
On Thursday, March 12, 2015 at 11:16:12 AM UTC-6, Abigail wrote:
Yes, that is what I meant. Is there any reference for set up the load
balance for Kibana 4? Or if it is easier for Kibana 3?
Why not load balance multiple tribe nodes, if you need multiple.
On Wednesday, March 11, 2015 at 9:41:39 AM UTC-6, Abigail wrote:
Hi Mark,
Thank you for your reply. Is there any existing approach for kibana to
communicate with multiple tribe nodes? Or is it something we should
implement by ourselves by customizing kibana?
Thank you!
Abigail
On Tuesday, March 10, 2015 at 8:56:25 PM UTC-4, Mark Walkom wrote:
1 - It's pretty simple and has been used before.
2 - it can be yes. You can have multiple tribe nodes though.
3 - This may be possible but you'd have to hack a fair bit of code, so
it's not really practical.
We are planning to use ELK for our log analysis. We have multiple
data centers. Since it is not recommended to have across data center
cluster, we are going to have one ES cluster per data center, here are the
three design options we have:
Use snapshot & restore to replicate data across clusters.
Use tribe node to achieve across cluster queries
Ship and index logs to each cluster
Here are our questions, and any comments will be appreciated:
How complex is snapshot & restore, anyone has experience on this
purpose?
Would the performance of only one tribe node be a concern or
bottleneck, is it possible to have multiple tribe nodes for scale up or
load balancing?
Is it possible to customize Kibana so that it can go to different
cluster to query data depends on the query?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.