A Question on Plugin redundancy


(Roy Russo) #1

Hello all,

Curious here on what is the recommended way of plugin deployment in a
cluster, for redundancy on that plugin. Say I have 10 nodes... I deploy a
plugin on Node1. Node1 dies. Now what? I may have a new master node, but my
plugin is gone.

Is there an idea of deploying plugins on every node, and having only the
elected master be the "active" plugin?

Excuse me if it's a dumb question. I can't seem to find the answer
anywhere.

Regards,
Roy Russo
http://www.elastichq.org/

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8cd60da6-7101-4a0d-9d63-d8e6ceac5c5a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(David Pilato) #2

What's wrong with installing plugins in all nodes?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 21 janv. 2014 à 19:57, Roy Russo royrusso@gmail.com a écrit :

Hello all,

Curious here on what is the recommended way of plugin deployment in a cluster, for redundancy on that plugin. Say I have 10 nodes... I deploy a plugin on Node1. Node1 dies. Now what? I may have a new master node, but my plugin is gone.

Is there an idea of deploying plugins on every node, and having only the elected master be the "active" plugin?

Excuse me if it's a dumb question. I can't seem to find the answer anywhere.

Regards,
Roy Russo
http://www.elastichq.org/

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/8cd60da6-7101-4a0d-9d63-d8e6ceac5c5a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/FB20901E-BC7D-49E9-8CCE-4584555AC3F2%40pilato.fr.
For more options, visit https://groups.google.com/groups/opt_out.


(Roy Russo) #3

Hi David,

Thanks for replying. The problem is that if you have a plugin that lets say
is responsible for polling ES _/node/stats and pushing the data
"somewhere", you don't want 100 nodes doing the same thing (that's a rough
example). It just seems like it's a central point of failure, but if
deployed per-node, you end up with duplicate data or duplicate service
calls with #X-simultaneously-running plugins.

You have written a lot of plugins/rivers. Are your meant to be deployed
singularly?

On Tuesday, January 21, 2014 2:34:49 PM UTC-5, David Pilato wrote:

What's wrong with installing plugins in all nodes?

--
David :wink:
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

Le 21 janv. 2014 à 19:57, Roy Russo <royr...@gmail.com <javascript:>> a
écrit :

Hello all,

Curious here on what is the recommended way of plugin deployment in a
cluster, for redundancy on that plugin. Say I have 10 nodes... I deploy a
plugin on Node1. Node1 dies. Now what? I may have a new master node, but my
plugin is gone.

Is there an idea of deploying plugins on every node, and having only the
elected master be the "active" plugin?

Excuse me if it's a dumb question. I can't seem to find the answer
anywhere.

Regards,
Roy Russo
http://www.elastichq.org/

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/8cd60da6-7101-4a0d-9d63-d8e6ceac5c5a%40googlegroups.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/9df71edf-fb1e-4438-8ac3-1f9c21a8b666%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jörg Prante) #4

You must deploy plugins to all nodes, so you will still have the plugin
available when nodes come and go. You could add an action so that you can
turn on/off a plugin explicitly by remote command. Or you can define a
condition so a plugin can decide when to run. For example if you want a
singleton, you can check if you run on the master node, and execute only
then.

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFQUHak0-PQ%3DLW3C0_0GQy8EwJThOKC__RhGgzGJk4MvA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Lukáš Vlček) #5

Roy,

if you are in fact talking about implementation of monitoring system for ES
then the following are my 2 cents:

  • regarding how to poll data from ES nodes, I would not implement it as a
    "plugin" or anything similar that is dependent on ES platform itself. It
    should be an external process and possibly very light process (Java process
    is not light). The impact on ES should be as minimal as possible and it
    should not rely on plugin system of ES either. I would consider looking at
    something like RRD tools (and extensions on top of it). Really lightweight
    process that sits on each node, kicks once a while, grabs all metrics via
    REST API and stores it into local storage. It should then try to send these
    data into central store but it may not be always available thus storing it
    locally is important in case the connection to central store is restored at
    later time. The other argument why not to use plugin system of ES is that
    you should be able to upgrade monitoring system independently on ES cluster

  • you can not do this with ES plugins.

  • should it poll the same stats from every single node? I think it should!
    If ES cluster gets into trouble is can be because not every part of the
    cluster sees "the same picture". IMO the only way how to learn about this
    is collecting individual "pictures from every node". It is also the most
    simple way of doing it (and you should definitely aim for simplicity).
    Solving duplicities on the central storage side later should be possible
    but you might not even need to do it if the size of collected data is not a
    problem for you. The part where you need to be very careful when polling
    the same stats from each node is making sure this is not putting high load
    on the ES cluster. If it is possible to poll from the local node only (like
    using _local in URL) then opt for it. Though I am no sure if this option is
    still available post 1.0.0.RC1 release.

Regards,
Lukas

On Tue, Jan 21, 2014 at 10:09 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

You must deploy plugins to all nodes, so you will still have the plugin
available when nodes come and go. You could add an action so that you can
turn on/off a plugin explicitly by remote command. Or you can define a
condition so a plugin can decide when to run. For example if you want a
singleton, you can check if you run on the master node, and execute only
then.

Jörg

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFQUHak0-PQ%3DLW3C0_0GQy8EwJThOKC__RhGgzGJk4MvA%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAO9cvUbcy%2B-F1EdrOAJE%2Bh8Ud8JcbnpWpAf78ijZ4yO0jn6RYA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Roy Russo) #6

Hi Lukas,

Somewhat... The idea is a monitoring 'service'. Hosted. Where its hosted is
a separate conversation.

I agree with most everything you stated. Having a local agent running is
the way to go. I'm avoiding remote polling and little to no advantage in
distributing as a plugin. The one advantage is that it is very clean and
simple to install, but such a tight dependency has its downsides, as you
mentioned.

I don't agree on not using java. Folks using elasticsearch already have a
JVM installed obviously, so its an easy jar drop and run. I'm not sure why
you would consider it a heavy process. Have you been looking at my old
jboss code? :wink: rrd is lighter. No objection there. So is Perl.

At first the data from elasticsearch is meant to be condensed and sent
across the wire... Small payloads. At some point, I may move to storing the
full blobs, but that can task a system and certainly add to hardware costs
for retention. Across x,000 clusters, storing monitoring metrics form
histograms and analysis can get heavy.

On Jan 21, 2014 11:24 PM, "Lukáš Vlček" lukas.vlcek@gmail.com wrote:

Roy,

if you are in fact talking about implementation of monitoring system for
ES then the following are my 2 cents:

  • regarding how to poll data from ES nodes, I would not implement it as a
    "plugin" or anything similar that is dependent on ES platform itself. It
    should be an external process and possibly very light process (Java process
    is not light). The impact on ES should be as minimal as possible and it
    should not rely on plugin system of ES either. I would consider looking at
    something like RRD tools (and extensions on top of it). Really lightweight
    process that sits on each node, kicks once a while, grabs all metrics via
    REST API and stores it into local storage. It should then try to send these
    data into central store but it may not be always available thus storing it
    locally is important in case the connection to central store is restored at
    later time. The other argument why not to use plugin system of ES is that
    you should be able to upgrade monitoring system independently on ES cluster

  • you can not do this with ES plugins.

  • should it poll the same stats from every single node? I think it should!
    If ES cluster gets into trouble is can be because not every part of the
    cluster sees "the same picture". IMO the only way how to learn about this
    is collecting individual "pictures from every node". It is also the most
    simple way of doing it (and you should definitely aim for simplicity).
    Solving duplicities on the central storage side later should be possible
    but you might not even need to do it if the size of collected data is not a
    problem for you. The part where you need to be very careful when polling
    the same stats from each node is making sure this is not putting high load
    on the ES cluster. If it is possible to poll from the local node only (like
    using _local in URL) then opt for it. Though I am no sure if this option is
    still available post 1.0.0.RC1 release.

Regards,
Lukas

On Tue, Jan 21, 2014 at 10:09 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

You must deploy plugins to all nodes, so you will still have the plugin
available when nodes come and go. You could add an action so that you can
turn on/off a plugin explicitly by remote command. Or you can define a
condition so a plugin can decide when to run. For example if you want a
singleton, you can check if you run on the master node, and execute only
then.

Jörg

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFQUHak0-PQ%3DLW3C0_0GQy8EwJThOKC__RhGgzGJk4MvA%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/UWiNFuu6bw4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAO9cvUbcy%2B-F1EdrOAJE%2Bh8Ud8JcbnpWpAf78ijZ4yO0jn6RYA%40mail.gmail.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CANTOpgtZuTk%2BQkNii7SoZWtf%3DkQ%3DYJ5%2BxJP1i3kAa1Yw4M0k0Q%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Lukáš Vlček) #7

Roy,

if you want to kick start some process like every 2 seconds and all it does
is fire couple of HTTP requests and store responses to local file then
using Java for this might have higher cost compared to existing
alternatives. That is all I meant by "heavy" java. If such job can finish
in terms of millis then you can not even fully benefit from JVM goodies.

If, however, you mean long running agent then Java is valid option. JBoss
RHQ is a good example of monitoring implemented in Java (as you surely know
:-)).

Regards,
Lukáš
Dne 22.1.2014 6:41 "Roy Russo" royrusso@gmail.com napsal(a):

Hi Lukas,

Somewhat... The idea is a monitoring 'service'. Hosted. Where its hosted
is a separate conversation.

I agree with most everything you stated. Having a local agent running is
the way to go. I'm avoiding remote polling and little to no advantage in
distributing as a plugin. The one advantage is that it is very clean and
simple to install, but such a tight dependency has its downsides, as you
mentioned.

I don't agree on not using java. Folks using elasticsearch already have a
JVM installed obviously, so its an easy jar drop and run. I'm not sure why
you would consider it a heavy process. Have you been looking at my old
jboss code? :wink: rrd is lighter. No objection there. So is Perl.

At first the data from elasticsearch is meant to be condensed and sent
across the wire... Small payloads. At some point, I may move to storing the
full blobs, but that can task a system and certainly add to hardware costs
for retention. Across x,000 clusters, storing monitoring metrics form
histograms and analysis can get heavy.

On Jan 21, 2014 11:24 PM, "Lukáš Vlček" lukas.vlcek@gmail.com wrote:

Roy,

if you are in fact talking about implementation of monitoring system for
ES then the following are my 2 cents:

  • regarding how to poll data from ES nodes, I would not implement it as a
    "plugin" or anything similar that is dependent on ES platform itself. It
    should be an external process and possibly very light process (Java process
    is not light). The impact on ES should be as minimal as possible and it
    should not rely on plugin system of ES either. I would consider looking at
    something like RRD tools (and extensions on top of it). Really lightweight
    process that sits on each node, kicks once a while, grabs all metrics via
    REST API and stores it into local storage. It should then try to send these
    data into central store but it may not be always available thus storing it
    locally is important in case the connection to central store is restored at
    later time. The other argument why not to use plugin system of ES is that
    you should be able to upgrade monitoring system independently on ES cluster

  • you can not do this with ES plugins.

  • should it poll the same stats from every single node? I think it
    should! If ES cluster gets into trouble is can be because not every part of
    the cluster sees "the same picture". IMO the only way how to learn about
    this is collecting individual "pictures from every node". It is also the
    most simple way of doing it (and you should definitely aim for simplicity).
    Solving duplicities on the central storage side later should be possible
    but you might not even need to do it if the size of collected data is not a
    problem for you. The part where you need to be very careful when polling
    the same stats from each node is making sure this is not putting high load
    on the ES cluster. If it is possible to poll from the local node only (like
    using _local in URL) then opt for it. Though I am no sure if this option is
    still available post 1.0.0.RC1 release.

Regards,
Lukas

On Tue, Jan 21, 2014 at 10:09 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

You must deploy plugins to all nodes, so you will still have the plugin
available when nodes come and go. You could add an action so that you can
turn on/off a plugin explicitly by remote command. Or you can define a
condition so a plugin can decide when to run. For example if you want a
singleton, you can check if you run on the master node, and execute only
then.

Jörg

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFQUHak0-PQ%3DLW3C0_0GQy8EwJThOKC__RhGgzGJk4MvA%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/UWiNFuu6bw4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAO9cvUbcy%2B-F1EdrOAJE%2Bh8Ud8JcbnpWpAf78ijZ4yO0jn6RYA%40mail.gmail.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANTOpgtZuTk%2BQkNii7SoZWtf%3DkQ%3DYJ5%2BxJP1i3kAa1Yw4M0k0Q%40mail.gmail.com
.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAO9cvUZ1CcV6JC4YvVjCre4FqQHqfLScoWpsgEmmjzWq8fGNWA%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jörg Prante) #8

Roy, from what I understand, you want plugins that are somehow coordinated
and do not require to get installed on every node.

A similar situation is possible in the HTTP area. Some ES nodes may provide
HTTP, some not, by disabling HTTP.

The deploy of a set of coordinated plugins is possible by adding a
coordination service within a "uber plugin" architecture where a group of
node dynamically serve custom requests within a cluster. The idea is to
deploy an uber plugin once to several nodes, not all. From then on, they
accept REST commands for distributed job executing on the nodes that can
serve the job type. Each node can examine the cluster nodes and decide
where to forward the job to. Nodes that have no plugin installed - or
having the plugin disabled - are not affected.

With a trick, the uber plugin nodes can receive additional jars live, for
serving different job types.

This can be useful for monitoring services, but also for scalable rivers.

All the ES admin have to do is to select the appropriate nodes for the uber
plugin.

I started his work can called it gatherer plugin, see

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHOJgZ_mao8myiESwOex6oy%3Dm53rUn32e%2Bha83b2u2Fbw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Roy Russo) #9

Hi Jorg,

I think gatherer may be a little heavy for what I'm trying to achieve. What
I'm working on can be compared to
MMS: https://mms.mongodb.com/learn-more#monitoring

Simple idea. No magic. I like to delude myself in to thinking its the next
phase for ElasticHQ. :wink:

Lukas has a point in that distribution as a plugin is not ideal, as a
monitoring agent should not be affected by the system it is monitoring -
not intimately tied.

Lukas,

Didn't know of RHQ. I was there for JBossON.

On Tuesday, January 21, 2014 1:57:18 PM UTC-5, Roy Russo wrote:

Hello all,

Curious here on what is the recommended way of plugin deployment in a
cluster, for redundancy on that plugin. Say I have 10 nodes... I deploy a
plugin on Node1. Node1 dies. Now what? I may have a new master node, but my
plugin is gone.

Is there an idea of deploying plugins on every node, and having only the
elected master be the "active" plugin?

Excuse me if it's a dumb question. I can't seem to find the answer
anywhere.

Regards,
Roy Russo
http://www.elastichq.org/

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/f69b698e-47cd-48d6-b738-76f7a059a5e3%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Jörg Prante) #10

Yes, you could ramp up one (or more) Elasticsearch node(s) without data,
without http, not master eligible - but with gatherer plugin :wink:

Jörg

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoF5%2By0MoFMWrmkE-rkVjiTXHXq%2B7G5yk4RBCg%2BezxNzKg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Lukáš Vlček) #11

Roy,

Sounds a bit like similar approach to Sematext SPM to me (except they use
CollectD - which is based on RRD is I am not mistaken) - but that shouldn't
stop you.

As for RHQ it is upstream for JON (JBossON). See
http://planet.jboss.org/post/jboss_operations_network_jbosson_jon_rhq_whats_in_a_name
I think it might have been renamed (this happens in JBoss world :-)).

Regards,
Lukáš

On Wed, Jan 22, 2014 at 4:43 PM, Roy Russo royrusso@gmail.com wrote:

Hi Jorg,

I think gatherer may be a little heavy for what I'm trying to achieve.
What I'm working on can be compared to MMS:
https://mms.mongodb.com/learn-more#monitoring

Simple idea. No magic. I like to delude myself in to thinking its the next
phase for ElasticHQ. :wink:

Lukas has a point in that distribution as a plugin is not ideal, as a
monitoring agent should not be affected by the system it is monitoring -
not intimately tied.

Lukas,

Didn't know of RHQ. I was there for JBossON.

On Tuesday, January 21, 2014 1:57:18 PM UTC-5, Roy Russo wrote:

Hello all,

Curious here on what is the recommended way of plugin deployment in a
cluster, for redundancy on that plugin. Say I have 10 nodes... I deploy a
plugin on Node1. Node1 dies. Now what? I may have a new master node, but my
plugin is gone.

Is there an idea of deploying plugins on every node, and having only the
elected master be the "active" plugin?

Excuse me if it's a dumb question. I can't seem to find the answer
anywhere.

Regards,
Roy Russo
http://www.elastichq.org/

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/f69b698e-47cd-48d6-b738-76f7a059a5e3%40googlegroups.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAO9cvUZo2bV9xo7%3DvUmCWcm%2BpA7UNfqUo4-3nPnWf2pnZX7J2A%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Roy Russo) #12

Lukas,

Yes, very similar approach in tech, but different in distribution model.
The use of RRD makes sense for them, as I believe their offering monitors
more than just ES clusters. It's a New Relic type of model. Alternatively,
New Relic has ES monitoring plugins available as well.

On Wednesday, January 22, 2014 12:00:11 PM UTC-5, Lukáš Vlček wrote:

Roy,

Sounds a bit like similar approach to Sematext SPM to me (except they use
CollectD - which is based on RRD is I am not mistaken) - but that shouldn't
stop you.

As for RHQ it is upstream for JON (JBossON). See
http://planet.jboss.org/post/jboss_operations_network_jbosson_jon_rhq_whats_in_a_name
I think it might have been renamed (this happens in JBoss world :-)).

Regards,
Lukáš

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/a07a40ac-e1fa-4554-834f-6ab8a268ec24%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Ivan Brusic) #13

There have been various discussion over the years regarding monitoring. I
use graphite, but the elasticsearch team also built an a similar service
with an elasticsearch backend:
https://github.com/elasticsearch/elasticsearch-metrics-reporter-java.

Of course, this would just gather the data, with no reporting capabilities.

Ivan

On Wed, Jan 22, 2014 at 9:54 AM, Roy Russo royrusso@gmail.com wrote:

Lukas,

Yes, very similar approach in tech, but different in distribution model.
The use of RRD makes sense for them, as I believe their offering monitors
more than just ES clusters. It's a New Relic type of model. Alternatively,
New Relic has ES monitoring plugins available as well.

On Wednesday, January 22, 2014 12:00:11 PM UTC-5, Lukáš Vlček wrote:

Roy,

Sounds a bit like similar approach to Sematext SPM to me (except they use
CollectD - which is based on RRD is I am not mistaken) - but that shouldn't
stop you.

As for RHQ it is upstream for JON (JBossON). See http://planet.jboss.org/
post/jboss_operations_network_jbosson_jon_rhq_whats_in_a_name
I think it might have been renamed (this happens in JBoss world :-)).

Regards,
Lukáš

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a07a40ac-e1fa-4554-834f-6ab8a268ec24%40googlegroups.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDF-G7%2BqYDK5cK5DxZgK6c5NMGwTFDRk%3Dowbi9we8A6Uw%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Alexander Reelsen) #14

Hey Roy,

another simple solution might be to simply have the plugin run every n
seconds/minutes and check if the node it runs on is currently the master
node. This ensure, that data is collected only on one node in your cluster
and it works in case of outages after the master node reelection... of
course you have dedicated master nodes and those never fail :slight_smile:

In summary, it still might be a better idea to have an external process,
which can be upgraded anytime, even without a cluster outage or another
rolling upgrade.

--Alex

On Thu, Jan 23, 2014 at 5:14 PM, Ivan Brusic ivan@brusic.com wrote:

There have been various discussion over the years regarding monitoring. I
use graphite, but the elasticsearch team also built an a similar service
with an elasticsearch backend:
https://github.com/elasticsearch/elasticsearch-metrics-reporter-java.

Of course, this would just gather the data, with no reporting capabilities.

Ivan

On Wed, Jan 22, 2014 at 9:54 AM, Roy Russo royrusso@gmail.com wrote:

Lukas,

Yes, very similar approach in tech, but different in distribution model.
The use of RRD makes sense for them, as I believe their offering monitors
more than just ES clusters. It's a New Relic type of model. Alternatively,
New Relic has ES monitoring plugins available as well.

On Wednesday, January 22, 2014 12:00:11 PM UTC-5, Lukáš Vlček wrote:

Roy,

Sounds a bit like similar approach to Sematext SPM to me (except they
use CollectD - which is based on RRD is I am not mistaken) - but that
shouldn't stop you.

As for RHQ it is upstream for JON (JBossON). See
http://planet.jboss.org/post/jboss_operations_network_
jbosson_jon_rhq_whats_in_a_name
I think it might have been renamed (this happens in JBoss world :-)).

Regards,
Lukáš

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a07a40ac-e1fa-4554-834f-6ab8a268ec24%40googlegroups.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDF-G7%2BqYDK5cK5DxZgK6c5NMGwTFDRk%3Dowbi9we8A6Uw%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAGCwEM--nG0NYRwtOQThKBj78GQyGHqynod58oT4c%2BKzXrTuMg%40mail.gmail.com.
For more options, visit https://groups.google.com/groups/opt_out.


(Roy Russo) #15

Hi Alex,

Yep. I think Jorg (or someone in this thread) suggested this approach. It's
likely the approach I will be taking at first swing with the hope of moving
later to a hybrid, where the the same core code can be used to install as
either a plugin or standalone/separated process running in a different jvm.

On Monday, January 27, 2014 4:13:32 AM UTC-5, Alexander Reelsen wrote:

Hey Roy,

another simple solution might be to simply have the plugin run every n
seconds/minutes and check if the node it runs on is currently the master
node. This ensure, that data is collected only on one node in your cluster
and it works in case of outages after the master node reelection... of
course you have dedicated master nodes and those never fail :slight_smile:

In summary, it still might be a better idea to have an external process,
which can be upgraded anytime, even without a cluster outage or another
rolling upgrade.

--Alex

On Thu, Jan 23, 2014 at 5:14 PM, Ivan Brusic <iv...@brusic.com<javascript:>

wrote:

There have been various discussion over the years regarding monitoring. I
use graphite, but the elasticsearch team also built an a similar service
with an elasticsearch backend:
https://github.com/elasticsearch/elasticsearch-metrics-reporter-java.

Of course, this would just gather the data, with no reporting
capabilities.

Ivan

On Wed, Jan 22, 2014 at 9:54 AM, Roy Russo <royr...@gmail.com<javascript:>

wrote:

Lukas,

Yes, very similar approach in tech, but different in distribution model.
The use of RRD makes sense for them, as I believe their offering monitors
more than just ES clusters. It's a New Relic type of model. Alternatively,
New Relic has ES monitoring plugins available as well.

On Wednesday, January 22, 2014 12:00:11 PM UTC-5, Lukáš Vlček wrote:

Roy,

Sounds a bit like similar approach to Sematext SPM to me (except they
use CollectD - which is based on RRD is I am not mistaken) - but that
shouldn't stop you.

As for RHQ it is upstream for JON (JBossON). See
http://planet.jboss.org/post/jboss_operations_network_
jbosson_jon_rhq_whats_in_a_name
I think it might have been renamed (this happens in JBoss world :-)).

Regards,
Lukáš

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/a07a40ac-e1fa-4554-834f-6ab8a268ec24%40googlegroups.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CALY%3DcQDF-G7%2BqYDK5cK5DxZgK6c5NMGwTFDRk%3Dowbi9we8A6Uw%40mail.gmail.com
.

For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/d3ce9dc9-d5a3-45f8-bed5-c2d511d84379%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.


(system) #16