# Writing a ES plugin for AWS CloudWatch **URL:** \ **Category:** Elasticsearch\ **Created:** [March 7, 2013, 10:24am UTC](https://discuss.elastic.co/t/writing-a-es-plugin-for-aws-cloudwatch/11052 "2013-03-07T10:24:32Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![flavia](https://avatars.discourse-cdn.com/v4/letter/f/a87d85/32.png) [@flavia](https://discuss.elastic.co/u/flavia)\ **Post date:** [March 7, 2013, 10:24am UTC](https://discuss.elastic.co/t/writing-a-es-plugin-for-aws-cloudwatch/11052/1 "2013-03-07T10:24:32Z")
Hi all, I'm writing an elasticsearch plugin which periodically (e.g. every minute) checks the health and state information of a cluster, and posts this information in Amazon Web Service's CloudWatch. In this way we can monitor what is going on with our database, and we are able to act as soon as some of these values are out of the normal range. I'm pretty much a newbie in elasticsearch, and I read somewhere in another post[https://groups.google.com/forum/?hl=en&fromgroups=#!searchin/elasticsearch/writing$20plugin/elasticsearch/fsTVZg3Hyr8/IQfmGI1jMqYJ](https://groups.google.com/forum/?hl=en&fromgroups=#!searchin/elasticsearch/writing%2420plugin/elasticsearch/fsTVZg3Hyr8/IQfmGI1jMqYJ)that there are these kinds of plugins: - "river plugins" operating at cluster level - "generic plugins" working at node level - "index plugins" working at indices level, having access to mappings and indices within a cluster - "shard plugins" working at shard level, having access to Lucene index structures - "transport client plugins" working out-of-scope of a cluster So what should be the kind of plugin I should use? I need to be able to: - Run periodically. I plan to use a java Timer for that. - Access information about the cluster state and cluster health. Which other plugin can I look at, to get an example of how to do the second? Thanks!! Flavia -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![flavia](https://avatars.discourse-cdn.com/v4/letter/f/a87d85/32.png) [@flavia](https://discuss.elastic.co/u/flavia)\ **Post date:** [March 8, 2013, 11:47am UTC](https://discuss.elastic.co/t/writing-a-es-plugin-for-aws-cloudwatch/11052/2 "2013-03-08T11:47:33Z")
Hi! I made progress with the plugin, it's nearly done. I wrote it using a service (inheriting AbstractLifecycleComponent), like this one: [GitHub - elastic/elasticsearch-transport-thrift: Thrift Transport for elasticsearch (STOPPED)](https://github.com/elasticsearch/elasticsearch-transport-thrift). The authors of this plugin also wrote a guide/intro to writing a plugin, see here: [http://jfarrell.github.com/](http://jfarrell.github.com/), very nice. Still it's not clear to me what kind of plugins you can write. It seems every plugin I look at is implemented in a different way. Perhaps there is some documentation about it that I didn't find. Thanks! Flavia On Thursday, March 7, 2013 11:24:32 AM UTC+1, [fla...@9apps.net](mailto:fla...@9apps.net) wrote: > Hi all, > > I'm writing an elasticsearch plugin which periodically (e.g. every minute) > checks the health and state information of a cluster, and posts this > information in Amazon Web Service's CloudWatch. In this way we can monitor > what is going on with our database, and we are able to act as soon as some > of these values are out of the normal range. > > I'm pretty much a newbie in elasticsearch, and I read somewhere in another > post[https://groups.google.com/forum/?hl=en&fromgroups=#!searchin/elasticsearch/writing$20plugin/elasticsearch/fsTVZg3Hyr8/IQfmGI1jMqYJ](https://groups.google.com/forum/?hl=en&fromgroups=#!searchin/elasticsearch/writing$20plugin/elasticsearch/fsTVZg3Hyr8/IQfmGI1jMqYJ)that there are these kinds of plugins: > > - "river plugins" operating at cluster level > - "generic plugins" working at node level > - "index plugins" working at indices level, having access to mappings and > indices within a cluster > - "shard plugins" working at shard level, having access to Lucene index > structures > - "transport client plugins" working out-of-scope of a cluster > > So what should be the kind of plugin I should use? I need to be able to: > > - Run periodically. I plan to use a java Timer for that. > - Access information about the cluster state and cluster health. > > Which other plugin can I look at, to get an example of how to do the > second? > > Thanks!! > > Flavia -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [March 8, 2013, 12:21pm UTC](https://discuss.elastic.co/t/writing-a-es-plugin-for-aws-cloudwatch/11052/3 "2013-03-08T12:21:08Z")
You already have found out about plugin types. Because ES is so flexible, you can write plugins in many different ways, but there are some method usages plugins share. In the class org.elasticsearch.plugins.AbstractPlugin you can see methods a plugin can activate. The simplest type is the "generic" plugin, I call it "generic" just because it does not care about indices, shards, or transport operations, so, it does not use indexModules/indexServices or shardModules/shardServices methods. For your task of reading the cluster state and post it somewhere, a generic plugin is ok. Jörg Am 08.03.13 12:47, schrieb [flavia@9apps.net](mailto:flavia@9apps.net): > Still it's not clear to me what kind of plugins you can write. It > seems every plugin I look at is implemented in a different way. > Perhaps there is some documentation about it that I didn't find. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![spinscale](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spinscale/32/25011_2.png) [@spinscale](https://discuss.elastic.co/u/spinscale)\ **Post date:** [March 8, 2013, 12:26pm UTC](https://discuss.elastic.co/t/writing-a-es-plugin-for-aws-cloudwatch/11052/4 "2013-03-08T12:26:22Z")
Hey there, take a look at [GitHub - spinscale/elasticsearch-graphite-plugin: [UNMAINTAINED] Elasticsearch plugin which pushes data to a graphite server](https://github.com/spinscale/elasticsearch-graphite-plugin) This plugin already does something similar you seem to try. It is polling some cluster information every n seconds and pushes them somewhere. It seems you are trying to do the same, so you might copy some code off it. --Alex On Fri, Mar 8, 2013 at 12:47 PM, [flavia@9apps.net](mailto:flavia@9apps.net) wrote: > Hi! > > I made progress with the plugin, it's nearly done. I wrote it using a > service (inheriting AbstractLifecycleComponent), like this one: > [GitHub - elastic/elasticsearch-transport-thrift: Thrift Transport for elasticsearch (STOPPED)](https://github.com/elasticsearch/elasticsearch-transport-thrift). The > authors of this plugin also wrote a guide/intro to writing a plugin, see > here: [http://jfarrell.github.com/](http://jfarrell.github.com/), very nice. > > Still it's not clear to me what kind of plugins you can write. It seems > every plugin I look at is implemented in a different way. Perhaps there is > some documentation about it that I didn't find. > > Thanks! > Flavia > > On Thursday, March 7, 2013 11:24:32 AM UTC+1, [fla...@9apps.net](mailto:fla...@9apps.net) wrote: > > > Hi all, > > > > I'm writing an elasticsearch plugin which periodically (e.g. every > > minute) checks the health and state information of a cluster, and posts > > this information in Amazon Web Service's CloudWatch. In this way we can > > monitor what is going on with our database, and we are able to act as soon > > as some of these values are out of the normal range. > > > > I'm pretty much a newbie in elasticsearch, and I read somewhere in another > > post[https://groups.google.com/forum/?hl=en&fromgroups=#!searchin/elasticsearch/writing$20plugin/elasticsearch/fsTVZg3Hyr8/IQfmGI1jMqYJ](https://groups.google.com/forum/?hl=en&fromgroups=#!searchin/elasticsearch/writing$20plugin/elasticsearch/fsTVZg3Hyr8/IQfmGI1jMqYJ)that there are these kinds of plugins: > > > > - "river plugins" operating at cluster level > > - "generic plugins" working at node level > > - "index plugins" working at indices level, having access to mappings and > > indices within a cluster > > - "shard plugins" working at shard level, having access to Lucene index > > structures > > - "transport client plugins" working out-of-scope of a cluster > > > > So what should be the kind of plugin I should use? I need to be able to: > > > > - Run periodically. I plan to use a java Timer for that. > > - Access information about the cluster state and cluster health. > > > > Which other plugin can I look at, to get an example of how to do the > > second? > > > > Thanks!! > > > > Flavia > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![flavia](https://avatars.discourse-cdn.com/v4/letter/f/a87d85/32.png) [@flavia](https://discuss.elastic.co/u/flavia)\ **Post date:** [March 8, 2013, 1:09pm UTC](https://discuss.elastic.co/t/writing-a-es-plugin-for-aws-cloudwatch/11052/5 "2013-03-08T13:09:54Z")
Ah, ok, I guess I am using the generic one. I am overriding the services() method. Thanks for your answer! Flavia On Friday, March 8, 2013 1:21:08 PM UTC+1, Jörg Prante wrote: > You already have found out about plugin types. Because ES is so > flexible, you can write plugins in many different ways, but there are > some method usages plugins share. > > In the class org.elasticsearch.plugins.AbstractPlugin you can see > methods a plugin can activate. The simplest type is the "generic" > plugin, I call it "generic" just because it does not care about indices, > shards, or transport operations, so, it does not use > indexModules/indexServices or shardModules/shardServices methods. > > For your task of reading the cluster state and post it somewhere, a > generic plugin is ok. > > Jörg > > Am 08.03.13 12:47, schrieb [fla...@9apps.net](mailto:fla...@9apps.net) \: > > > Still it's not clear to me what kind of plugins you can write. It > > seems every plugin I look at is implemented in a different way. > > Perhaps there is some documentation about it that I didn't find. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![flavia](https://avatars.discourse-cdn.com/v4/letter/f/a87d85/32.png) [@flavia](https://discuss.elastic.co/u/flavia)\ **Post date:** [March 8, 2013, 1:10pm UTC](https://discuss.elastic.co/t/writing-a-es-plugin-for-aws-cloudwatch/11052/6 "2013-03-08T13:10:31Z")
thanks! I will take a look On Friday, March 8, 2013 1:26:22 PM UTC+1, Alexander Reelsen wrote: > Hey there, > > take a look at [GitHub - spinscale/elasticsearch-graphite-plugin: [UNMAINTAINED] Elasticsearch plugin which pushes data to a graphite server](https://github.com/spinscale/elasticsearch-graphite-plugin) > > This plugin already does something similar you seem to try. It is polling > some cluster information every n seconds and pushes them somewhere. It > seems you are trying to do the same, so you might copy some code off it. > > --Alex > > On Fri, Mar 8, 2013 at 12:47 PM, \<[fla...@9apps.net](mailto:fla...@9apps.net) \\> wrote: > > > Hi! > > > > I made progress with the plugin, it's nearly done. I wrote it using a > > service (inheriting AbstractLifecycleComponent), like this one: > > [GitHub - elastic/elasticsearch-transport-thrift: Thrift Transport for elasticsearch (STOPPED)](https://github.com/elasticsearch/elasticsearch-transport-thrift). The > > authors of this plugin also wrote a guide/intro to writing a plugin, see > > here: [http://jfarrell.github.com/](http://jfarrell.github.com/), very nice. > > > > Still it's not clear to me what kind of plugins you can write. It seems > > every plugin I look at is implemented in a different way. Perhaps there is > > some documentation about it that I didn't find. > > > > Thanks! > > Flavia > > > > On Thursday, March 7, 2013 11:24:32 AM UTC+1, [fla...@9apps.net](mailto:fla...@9apps.net) wrote: > > > > > Hi all, > > > > > > I'm writing an elasticsearch plugin which periodically (e.g. every > > > minute) checks the health and state information of a cluster, and posts > > > this information in Amazon Web Service's CloudWatch. In this way we can > > > monitor what is going on with our database, and we are able to act as soon > > > as some of these values are out of the normal range. > > > > > > I'm pretty much a newbie in elasticsearch, and I read somewhere in another > > > post[https://groups.google.com/forum/?hl=en&fromgroups=#!searchin/elasticsearch/writing$20plugin/elasticsearch/fsTVZg3Hyr8/IQfmGI1jMqYJ](https://groups.google.com/forum/?hl=en&fromgroups=#!searchin/elasticsearch/writing$20plugin/elasticsearch/fsTVZg3Hyr8/IQfmGI1jMqYJ)that there are these kinds of plugins: > > > > > > - "river plugins" operating at cluster level > > > - "generic plugins" working at node level > > > - "index plugins" working at indices level, having access to mappings > > > and indices within a cluster > > > - "shard plugins" working at shard level, having access to Lucene index > > > structures > > > - "transport client plugins" working out-of-scope of a cluster > > > > > > So what should be the kind of plugin I should use? I need to be able to: > > > > > > - Run periodically. I plan to use a java Timer for that. > > > - Access information about the cluster state and cluster health. > > > > > > Which other plugin can I look at, to get an example of how to do the > > > second? > > > > > > Thanks!! > > > > > > Flavia > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). For more options, visit [https://groups.google.com/groups/opt\_out](https://groups.google.com/groups/opt_out). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:47am UTC](https://discuss.elastic.co/t/writing-a-es-plugin-for-aws-cloudwatch/11052/7 "2017-07-06T02:47:34Z")