# Keeping JMX and REST metrics APIs in sync **URL:** \ **Category:** Elasticsearch\ **Created:** [February 3, 2012, 10:08pm UTC](https://discuss.elastic.co/t/keeping-jmx-and-rest-metrics-apis-in-sync/6590 "2012-02-03T22:08:46Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)\ **Post date:** [February 3, 2012, 10:08pm UTC](https://discuss.elastic.co/t/keeping-jmx-and-rest-metrics-apis-in-sync/6590/1 "2012-02-03T22:08:46Z")
Hi, Shay - what are the chances of metrics exposed in JMX getting (and remaining!) in sync with metrics exposed via REST APIs? I know you don't like JMX because of the overhead, but JMX is already there (so already has whatever overhead it adds), it's just that it's not useful because it doesn't have all metrics. So if one/you were to bring metrics in JMX up to date, what are the chances of you making sure to keep that in sync with metrics in REST APIs? (basically, if new metric becomes available, it should be exposed via JMX, too) ## Thanks, Otis > **[Sematext – Solr / Elasticsearch Experts – Search & Big Data Analytics](https://sematext.com/)** > > Building innovative Cloud and On Premise solutions for performance monitoring, alerting and anomaly detection (SPM), and log management and analytics (Logsene) ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [February 5, 2012, 4:26pm UTC](https://discuss.elastic.co/t/keeping-jmx-and-rest-metrics-apis-in-sync/6590/2 "2012-02-05T16:26:04Z")
On Saturday, February 4, 2012 at 12:08 AM, Otis Gospodnetic wrote: > Hi, > > Shay - what are the chances of metrics exposed in JMX getting (and > remaining!) in sync with metrics exposed via REST APIs? > I know you don't like JMX because of the overhead, but JMX is already > there (so already has whatever overhead it adds), it's just that it's > not useful because it doesn't have all metrics. First, the overhead of JMX mainly comes from RMI, and thats only "started" when you start to monitor it remotely (and the annoying GC aspects of it as well, like full GC unless configured properly). More monitoring systems today have the ability to call a simple REST service and extract the relevant metrics from json. This option is vastly superior to JMX (over RMI) because you can get proper "objects" back and not ping each attribute separately, already uses HTTP so it works well with things like load balancer and security, and the like. > So if one/you were to bring metrics in JMX up to date, what are the > chances of you making sure to keep that in sync with metrics in REST > APIs? (basically, if new metric becomes available, it should be > exposed via JMX, too) Its a matter of being diligent, and whenever a new stat is added, expose it. > ## Thanks, Otis > > [http://sematext.com/](http://sematext.com/) ---
**Author:** ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)\ **Post date:** [February 7, 2012, 3:10am UTC](https://discuss.elastic.co/t/keeping-jmx-and-rest-metrics-apis-in-sync/6590/3 "2012-02-07T03:10:58Z")
Hi, On Feb 5, 11:26 am, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > On Saturday, February 4, 2012 at 12:08 AM, Otis Gospodnetic wrote: > > > Hi, > > > Shay - what are the chances of metrics exposed in JMX getting (and > > remaining!) in sync with metrics exposed via REST APIs? > > I know you don't like JMX because of the overhead, but JMX is already > > there (so already has whatever overhead it adds), it's just that it's > > not useful because it doesn't have all metrics. > > First, the overhead of JMX mainly comes from RMI, and thats only "started" when you start to monitor it remotely (and the annoying GC aspects of it as well, like full GC unless configured properly). Ah, right, lovely RMI. I can't believe that's still in use! In my particular case, I am/was interested in connected to JMX from within ES Node's JVM. > More monitoring systems today have the ability to call a simple REST service and extract the relevant metrics from json. This option is vastly superior to JMX (over RMI) because you can get proper "objects" back and not ping each attribute separately, already uses HTTP so it works well with things like load balancer and security, and the like. Right. We ended up going against HTTP and parsing JSON. One problem with this approach, however, is that JSON response changed between some versions, so in order for a monitoring tool to support multiple ES versions, it has to have version-specific parsers for that JSON. Is there an easy work-around or suggested way for dealing with this that I'm missing? We've also tried getting to the stats via the Java API, but that has the same problem - if you are not ES version-aware, you may call a method that does not exist in the version of ES being monitored. > > So if one/you were to bring metrics in JMX up to date, what are the > > chances of you making sure to keep that in sync with metrics in REST > > APIs? (basically, if new metric becomes available, it should be > > exposed via JMX, too) > > Its a matter of being diligent, and whenever a new stat is added, expose it. Right. I'll open an issue for initial REST+JSON/JMX syncing and cross my fingers and toes in hopes of that remaining in sync in future versions. Here it is: [JMX and REST+JSON stats should be in sync · Issue #1675 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/1675) I wonder if one could write unit tests that catch situations where REST stats and JMX stats don't match 100% in terms of coverage... ## Thanks, Otis > **[Sematext | IT System Monitoring Tools for DevOps](https://sematext.com/)** > > IT system monitoring and management tools for DevOps who need 24x7 live visibility into their infrastructure. ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [February 7, 2012, 10:25am UTC](https://discuss.elastic.co/t/keeping-jmx-and-rest-metrics-apis-in-sync/6590/4 "2012-02-07T10:25:57Z")
I suggest you use the Java API to get the stats, if you are running embedded in ES. It can be provided as a plugin, and yes, if those change (though hopefully we nailed to structure now), you can version your plugin based on elasticsearch versions supported (as other plugins do). On Tuesday, February 7, 2012 at 5:10 AM, Otis Gospodnetic wrote: > Hi, > > On Feb 5, 11:26 am, Shay Banon \<[kim...@gmail.com](mailto:kim...@gmail.com) ([http://gmail.com](http://gmail.com))\> wrote: > > > On Saturday, February 4, 2012 at 12:08 AM, Otis Gospodnetic wrote: > > > > > Hi, > > > > > Shay - what are the chances of metrics exposed in JMX getting (and > > > remaining!) in sync with metrics exposed via REST APIs? > > > I know you don't like JMX because of the overhead, but JMX is already > > > there (so already has whatever overhead it adds), it's just that it's > > > not useful because it doesn't have all metrics. > > > > First, the overhead of JMX mainly comes from RMI, and thats only "started" when you start to monitor it remotely (and the annoying GC aspects of it as well, like full GC unless configured properly). > > Ah, right, lovely RMI. I can't believe that's still in use! > In my particular case, I am/was interested in connected to JMX from > within ES Node's JVM. > > > More monitoring systems today have the ability to call a simple REST service and extract the relevant metrics from json. This option is vastly superior to JMX (over RMI) because you can get proper "objects" back and not ping each attribute separately, already uses HTTP so it works well with things like load balancer and security, and the like. > > Right. We ended up going against HTTP and parsing JSON. > > One problem with this approach, however, is that JSON response changed > between some versions, so in order for a monitoring tool to support > multiple ES versions, it has to have version-specific parsers for that > JSON. > > Is there an easy work-around or suggested way for dealing with this > that I'm missing? > > We've also tried getting to the stats via the Java API, but that has > the same problem - if you are not ES version-aware, you may call a > method that does not exist in the version of ES being monitored. > > > > So if one/you were to bring metrics in JMX up to date, what are the > > > chances of you making sure to keep that in sync with metrics in REST > > > APIs? (basically, if new metric becomes available, it should be > > > exposed via JMX, too) > > > > Its a matter of being diligent, and whenever a new stat is added, expose it. > > Right. I'll open an issue for initial REST+JSON/JMX syncing and cross > my fingers and toes in hopes of that remaining in sync in future > versions. Here it is: > > [JMX and REST+JSON stats should be in sync · Issue #1675 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/1675) > > I wonder if one could write unit tests that catch situations where > REST stats and JMX stats don't match 100% in terms of coverage... > > ## Thanks, Otis > > [http://sematext.com/](http://sematext.com/) ---
**Author:** ![Lukas\_Vlcek1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lukas_vlcek1/32/819_2.png) [@Lukas\_Vlcek1](https://discuss.elastic.co/u/Lukas_Vlcek1)\ **Post date:** [February 7, 2012, 1:34pm UTC](https://discuss.elastic.co/t/keeping-jmx-and-rest-metrics-apis-in-sync/6590/5 "2012-02-07T13:34:44Z")
Otis, I think the only way how to get ES version via REST API is request to "/" point. However, I am not sure if there is any Java API for that. Shay, May be it would be useful to add such Java API or more focused REST API. IMO "/" endpoint is weakly defined, something like "/\_cluster/nodes/\_local/version" or "/\_nodes/\_local/version" would be better. Or may be even better include version information into Nodes info API. That way it would be easy to check that all nodes run on the same version using a single request, which might be a basic troubleshooting check for example. Regards, Lukas On Tue, Feb 7, 2012 at 11:25 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > I suggest you use the Java API to get the stats, if you are running > embedded in ES. It can be provided as a plugin, and yes, if those change > (though hopefully we nailed to structure now), you can version your plugin > based on elasticsearch versions supported (as other plugins do). > > On Tuesday, February 7, 2012 at 5:10 AM, Otis Gospodnetic wrote: > > Hi, > > On Feb 5, 11:26 am, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > On Saturday, February 4, 2012 at 12:08 AM, Otis Gospodnetic wrote: > > Hi, > > Shay - what are the chances of metrics exposed in JMX getting (and > remaining!) in sync with metrics exposed via REST APIs? > I know you don't like JMX because of the overhead, but JMX is already > there (so already has whatever overhead it adds), it's just that it's > not useful because it doesn't have all metrics. > > First, the overhead of JMX mainly comes from RMI, and thats only "started" > when you start to monitor it remotely (and the annoying GC aspects of it as > well, like full GC unless configured properly). > > Ah, right, lovely RMI. I can't believe that's still in use! > In my particular case, I am/was interested in connected to JMX from > within ES Node's JVM. > > More monitoring systems today have the ability to call a simple REST > service and extract the relevant metrics from json. This option is vastly > superior to JMX (over RMI) because you can get proper "objects" back and > not ping each attribute separately, already uses HTTP so it works well > with things like load balancer and security, and the like. > > Right. We ended up going against HTTP and parsing JSON. > > One problem with this approach, however, is that JSON response changed > between some versions, so in order for a monitoring tool to support > multiple ES versions, it has to have version-specific parsers for that > JSON. > > Is there an easy work-around or suggested way for dealing with this > that I'm missing? > > We've also tried getting to the stats via the Java API, but that has > the same problem - if you are not ES version-aware, you may call a > method that does not exist in the version of ES being monitored. > > So if one/you were to bring metrics in JMX up to date, what are the > chances of you making sure to keep that in sync with metrics in REST > APIs? (basically, if new metric becomes available, it should be > exposed via JMX, too) > > Its a matter of being diligent, and whenever a new stat is added, expose > it. > > Right. I'll open an issue for initial REST+JSON/JMX syncing and cross > my fingers and toes in hopes of that remaining in sync in future > versions. Here it is: > > [JMX and REST+JSON stats should be in sync · Issue #1675 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/1675) > > I wonder if one could write unit tests that catch situations where > REST stats and JMX stats don't match 100% in terms of coverage... > > ## Thanks, Otis > > [http://sematext.com/](http://sematext.com/) ---
**Author:** ![otisg](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/otisg/32/492_2.png) [@otisg](https://discuss.elastic.co/u/otisg)\ **Post date:** [February 7, 2012, 6:08pm UTC](https://discuss.elastic.co/t/keeping-jmx-and-rest-metrics-apis-in-sync/6590/6 "2012-02-07T18:08:48Z")
Yeah, +1 for exposing a version explicitly. ## Otis > **[Sematext | IT System Monitoring Tools for DevOps](https://sematext.com/)** > > IT system monitoring and management tools for DevOps who need 24x7 live visibility into their infrastructure. On Feb 7, 8:34 am, Lukáš Vlček [lukas.vl...@gmail.com](mailto:lukas.vl...@gmail.com) wrote: > Otis, > > I think the only way how to get ES version via REST API is request to "/" > point. However, I am not sure if there is any Java API for that. > > Shay, > > May be it would be useful to add such Java API or more focused REST API. > IMO "/" endpoint is weakly defined, something like > "/\_cluster/nodes/\_local/version" or "/\_nodes/\_local/version" would be > better. Or may be even better include version information into Nodes info > API. That way it would be easy to check that all nodes run on the same > version using a single request, which might be a basic troubleshooting > check for example. > > Regards, > Lukas > > On Tue, Feb 7, 2012 at 11:25 AM, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > I suggest you use the Java API to get the stats, if you are running > > embedded in ES. It can be provided as a plugin, and yes, if those change > > (though hopefully we nailed to structure now), you can version your plugin > > based on elasticsearch versions supported (as other plugins do). > > > On Tuesday, February 7, 2012 at 5:10 AM, Otis Gospodnetic wrote: > > > Hi, > > > On Feb 5, 11:26 am, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > On Saturday, February 4, 2012 at 12:08 AM, Otis Gospodnetic wrote: > > > Hi, > > > Shay - what are the chances of metrics exposed in JMX getting (and > > remaining!) in sync with metrics exposed via REST APIs? > > I know you don't like JMX because of the overhead, but JMX is already > > there (so already has whatever overhead it adds), it's just that it's > > not useful because it doesn't have all metrics. > > > First, the overhead of JMX mainly comes from RMI, and thats only "started" > > when you start to monitor it remotely (and the annoying GC aspects of it as > > well, like full GC unless configured properly). > > > Ah, right, lovely RMI. I can't believe that's still in use! > > In my particular case, I am/was interested in connected to JMX from > > within ES Node's JVM. > > > More monitoring systems today have the ability to call a simple REST > > service and extract the relevant metrics from json. This option is vastly > > superior to JMX (over RMI) because you can get proper "objects" back and > > not ping each attribute separately, already uses HTTP so it works well > > with things like load balancer and security, and the like. > > > Right. We ended up going against HTTP and parsing JSON. > > > One problem with this approach, however, is that JSON response changed > > between some versions, so in order for a monitoring tool to support > > multiple ES versions, it has to have version-specific parsers for that > > JSON. > > > Is there an easy work-around or suggested way for dealing with this > > that I'm missing? > > > We've also tried getting to the stats via the Java API, but that has > > the same problem - if you are not ES version-aware, you may call a > > method that does not exist in the version of ES being monitored. > > > So if one/you were to bring metrics in JMX up to date, what are the > > chances of you making sure to keep that in sync with metrics in REST > > APIs? (basically, if new metric becomes available, it should be > > exposed via JMX, too) > > > Its a matter of being diligent, and whenever a new stat is added, expose > > it. > > > Right. I'll open an issue for initial REST+JSON/JMX syncing and cross > > my fingers and toes in hopes of that remaining in sync in future > > versions. Here it is: > > > [JMX and REST+JSON stats should be in sync · Issue #1675 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/1675) > > > I wonder if one could write unit tests that catch situations where > > REST stats and JMX stats don't match 100% in terms of coverage... > > > ## Thanks, Otis > > > > [http://sematext.com/](http://sematext.com/) ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [February 7, 2012, 6:43pm UTC](https://discuss.elastic.co/t/keeping-jmx-and-rest-metrics-apis-in-sync/6590/7 "2012-02-07T18:43:54Z")
The nodes info Java API already includes the version associated with each node. Each NodeInfo element within the NodesInfoResponse include the node, which has the version associated with it. On Tuesday, February 7, 2012 at 8:08 PM, Otis Gospodnetic wrote: > Yeah, +1 for exposing a version explicitly. > > ## Otis > > [http://sematext.com/](http://sematext.com/) > > On Feb 7, 8:34 am, Lukáš Vlček \<[lukas.vl...@gmail.com](mailto:lukas.vl...@gmail.com) ([http://gmail.com](http://gmail.com))\> wrote: > > > Otis, > > > > I think the only way how to get ES version via REST API is request to "/" > > point. However, I am not sure if there is any Java API for that. > > > > Shay, > > > > May be it would be useful to add such Java API or more focused REST API. > > IMO "/" endpoint is weakly defined, something like > > "/\_cluster/nodes/\_local/version" or "/\_nodes/\_local/version" would be > > better. Or may be even better include version information into Nodes info > > API. That way it would be easy to check that all nodes run on the same > > version using a single request, which might be a basic troubleshooting > > check for example. > > > > Regards, > > Lukas > > > > On Tue, Feb 7, 2012 at 11:25 AM, Shay Banon \<[kim...@gmail.com](mailto:kim...@gmail.com) ([http://gmail.com](http://gmail.com))\> wrote: > > > > > I suggest you use the Java API to get the stats, if you are running > > > embedded in ES. It can be provided as a plugin, and yes, if those change > > > (though hopefully we nailed to structure now), you can version your plugin > > > based on elasticsearch versions supported (as other plugins do). > > > > > On Tuesday, February 7, 2012 at 5:10 AM, Otis Gospodnetic wrote: > > > > > Hi, > > > > > On Feb 5, 11:26 am, Shay Banon \<[kim...@gmail.com](mailto:kim...@gmail.com) ([http://gmail.com](http://gmail.com))\> wrote: > > > > > On Saturday, February 4, 2012 at 12:08 AM, Otis Gospodnetic wrote: > > > > > Hi, > > > > > Shay - what are the chances of metrics exposed in JMX getting (and > > > remaining!) in sync with metrics exposed via REST APIs? > > > I know you don't like JMX because of the overhead, but JMX is already > > > there (so already has whatever overhead it adds), it's just that it's > > > not useful because it doesn't have all metrics. > > > > > First, the overhead of JMX mainly comes from RMI, and thats only "started" > > > when you start to monitor it remotely (and the annoying GC aspects of it as > > > well, like full GC unless configured properly). > > > > > Ah, right, lovely RMI. I can't believe that's still in use! > > > In my particular case, I am/was interested in connected to JMX from > > > within ES Node's JVM. > > > > > More monitoring systems today have the ability to call a simple REST > > > service and extract the relevant metrics from json. This option is vastly > > > superior to JMX (over RMI) because you can get proper "objects" back and > > > not ping each attribute separately, already uses HTTP so it works well > > > with things like load balancer and security, and the like. > > > > > Right. We ended up going against HTTP and parsing JSON. > > > > > One problem with this approach, however, is that JSON response changed > > > between some versions, so in order for a monitoring tool to support > > > multiple ES versions, it has to have version-specific parsers for that > > > JSON. > > > > > Is there an easy work-around or suggested way for dealing with this > > > that I'm missing? > > > > > We've also tried getting to the stats via the Java API, but that has > > > the same problem - if you are not ES version-aware, you may call a > > > method that does not exist in the version of ES being monitored. > > > > > So if one/you were to bring metrics in JMX up to date, what are the > > > chances of you making sure to keep that in sync with metrics in REST > > > APIs? (basically, if new metric becomes available, it should be > > > exposed via JMX, too) > > > > > Its a matter of being diligent, and whenever a new stat is added, expose > > > it. > > > > > Right. I'll open an issue for initial REST+JSON/JMX syncing and cross > > > my fingers and toes in hopes of that remaining in sync in future > > > versions. Here it is: > > > > > [JMX and REST+JSON stats should be in sync · Issue #1675 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/1675) > > > > > I wonder if one could write unit tests that catch situations where > > > REST stats and JMX stats don't match 100% in terms of coverage... > > > > > ## Thanks, Otis > > > > > > [http://sematext.com/](http://sematext.com/) ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:40am UTC](https://discuss.elastic.co/t/keeping-jmx-and-rest-metrics-apis-in-sync/6590/8 "2017-07-06T03:40:12Z")