# Jclouds provider **URL:** \ **Category:** Elasticsearch\ **Created:** [September 9, 2011, 1:29pm UTC](https://discuss.elastic.co/t/jclouds-provider/5337 "2011-09-09T13:29:06Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![Josh\_Devins](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/josh_devins/32/2733_2.png) [@Josh\_Devins](https://discuss.elastic.co/u/Josh_Devins)\ **Post date:** [September 9, 2011, 1:29pm UTC](https://discuss.elastic.co/t/jclouds-provider/5337/1 "2011-09-09T13:29:06Z")
What happened to the jclouds provider that was in ES pre-0.8? I'm facing deploying ES to RackSpace cloud or SoftLayer and would like to use my cloud providers blob storage and discovery. Is there still a plugin available for jclouds or for cloud providers other than AWS? Thanks, Josh ---
**Author:** ![James\_Cook](https://avatars.discourse-cdn.com/v4/letter/j/898d66/32.png) [@James\_Cook](https://discuss.elastic.co/u/James_Cook)\ **Post date:** [September 9, 2011, 1:49pm UTC](https://discuss.elastic.co/t/jclouds-provider/5337/2 "2011-09-09T13:49:04Z")
> ---
**Author:** ![James\_Cook](https://avatars.discourse-cdn.com/v4/letter/j/898d66/32.png) [@James\_Cook](https://discuss.elastic.co/u/James_Cook)\ **Post date:** [September 9, 2011, 1:51pm UTC](https://discuss.elastic.co/t/jclouds-provider/5337/3 "2011-09-09T13:51:46Z")
I know that doesn't answer your question directly, but Shay dropped jClouds because it was having some problems early in its life. I agree that it would be nice to take another look at it. ---
**Author:** ![Josh\_Devins](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/josh_devins/32/2733_2.png) [@Josh\_Devins](https://discuss.elastic.co/u/Josh_Devins)\ **Post date:** [September 9, 2011, 1:53pm UTC](https://discuss.elastic.co/t/jclouds-provider/5337/4 "2011-09-09T13:53:37Z")
The issue mentions that RackSpace is supported but I don't see any documentation or reference to this support. Is it hiding somewhere? ๐Ÿ™‚ @James, nice AWS post too. Helps to describe how ES should integrate with clouds along with providing easy to follow steps for getting started on AWS. On Sep 9, 3:51 pm, James Cook [jc...@tracermedia.com](mailto:jc...@tracermedia.com) wrote: > I know that doesn't answer your question directly, but Shay dropped jClouds > because it was having some problems early in its life. I agree that it would > be nice to take another look at it. ---
**Author:** ![James\_Cook](https://avatars.discourse-cdn.com/v4/letter/j/898d66/32.png) [@James\_Cook](https://discuss.elastic.co/u/James_Cook)\ **Post date:** [September 9, 2011, 2:06pm UTC](https://discuss.elastic.co/t/jclouds-provider/5337/5 "2011-09-09T14:06:56Z")
Thanks, I'm glad it helped. No, Rackspace is not supported. The idea was that it was "supported" back in pre 0.8 builds because jClouds was used over the current cloud discovery process which only supports EC2 currently. Since Shay has externalized the cloud discovery process and gateway support for AWS specifically, it shouldn't be too hard to change it back over to jClouds support. I'm sure he would appreciate a pull request. I wish I had the time to work on it, because AWS is $$$, and I also would like other alternatives. The key would be to make sure that jClouds supports the necessary contracts for discovery (discover other nodes and provide region, group, tags), and the appropriate methods to support the cloud-based gateway (which I am unsure offhand how this works, i.e. sync/async transfer, auto chunking?, reading) ---
**Author:** ![Josh\_Devins](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/josh_devins/32/2733_2.png) [@Josh\_Devins](https://discuss.elastic.co/u/Josh_Devins)\ **Post date:** [September 9, 2011, 2:13pm UTC](https://discuss.elastic.co/t/jclouds-provider/5337/6 "2011-09-09T14:13:06Z")
We are in the same boat looking at growing AWS monthly bills ๐Ÿ˜‰ I will file a ticket and also if I have time, work on an implementation and a pull request. Would be great to get Shay's input as to why the jclouds-based integration was backed out, what sorts of problems he had, etc. At the very least, basic RackSpace support through their API should be possible. See: [https://github.com/elasticsearch/elasticsearch/issues/1319](https://github.com/elasticsearch/elasticsearch/issues/1319) ---
**Author:** ![James\_Cook](https://avatars.discourse-cdn.com/v4/letter/j/898d66/32.png) [@James\_Cook](https://discuss.elastic.co/u/James_Cook)\ **Post date:** [September 9, 2011, 2:14pm UTC](https://discuss.elastic.co/t/jclouds-provider/5337/7 "2011-09-09T14:14:11Z")
This might explain a bit: > ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [September 10, 2011, 10:19pm UTC](https://discuss.elastic.co/t/jclouds-provider/5337/8 "2011-09-10T22:19:07Z")
Heya, First, a quick explanation of what the aws cloud plugin provides. Basically, it provides two main features: 1. Automatic discovery based on cloud API. This is a nice extension to the zen discovery module, where the unicast list of hosts is built automatically using the AWS API. It would be nice to have it using jclouds or directly using other cloud APIs. The problem with jclouds back then was that it was doing far too much in order to just get the current running instances. Need to check if it improved. 2. If the cloud provider has support for blob store, then a blob store gateway can be implemented. This is again not necessarily required. You can run elasticsearch with the default local gateway which will work. Mentioning it again: both extensions, the auto disco based on the cloud API And blob store gateway are nice, but are not required in order to run elasticsearch on cloud providers. Without them, elasticsearch is similar to other "nosql" solutions, or even mysql (and the like) in terms of how you deploy it on your favorite cloud. Of course, it would be nice to have those extensions available to simplify even more the deployment on other clouds. On Fri, Sep 9, 2011 at 5:14 PM, James Cook [jcook@tracermedia.com](mailto:jcook@tracermedia.com) wrote: > This might explain a bit: > [Cloud Discovery does not appear to work ยท Issue #197 ยท elastic/elasticsearch ยท GitHub](https://github.com/elasticsearch/elasticsearch/issues/197) ---
**Author:** ![Josh\_Devins](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/josh_devins/32/2733_2.png) [@Josh\_Devins](https://discuss.elastic.co/u/Josh_Devins)\ **Post date:** [September 12, 2011, 8:27am UTC](https://discuss.elastic.co/t/jclouds-provider/5337/9 "2011-09-12T08:27:40Z")
Shay, thanks for the notes. Good to know the problems with jclouds in the past. I'm definitely more interested in getting the discovery and blob storage working as an optimization. I think it's clear that unicast discovery+local gateway works on any cloud provider. I hope to have some time to look into at least an initial jclouds solution, but could take some months given that it's an optimization ๐Ÿ˜‰ Cheers, Josh ---
**Author:** ![grkvlt](https://avatars.discourse-cdn.com/v4/letter/g/a183cd/32.png) [@grkvlt](https://discuss.elastic.co/u/grkvlt)\ **Post date:** [September 16, 2011, 4:59am UTC](https://discuss.elastic.co/t/jclouds-provider/5337/10 "2011-09-16T04:59:34Z")
Josh, I would be really interested in helping with the jclouds work, if I can. I work with the jclouds founder, Adrian, so that will hopefully make it easier to sort out any integration issues with jclouds and the blob store if they occur ๐Ÿ˜‰ I think the 1.1.1 release of jclouds is much improved, and as mentioned above it would greatly simplify deployment to a new cloud provider. As for 'doing too much to get the current running instances', I'm not sure what was happening there. It _should_ be a pretty simple task in jclouds... Cheers, Andrew. On Sep 12, 9:27 am, Josh Devins [i...@joshdevins.net](mailto:i...@joshdevins.net) wrote: > Shay, thanks for the notes. Good to know the problems withjcloudsin > the past. I'm definitely more interested in getting the discovery and > blob storage working as an optimization. I think it's clear that > unicast discovery+local gateway works on any cloud provider. > > I hope to have some time to look into at least an initialjclouds > solution, but could take some months given that it's an > optimization ๐Ÿ˜‰ > > Cheers, > > Josh ---
**Author:** ![Josh\_Devins\_2](https://avatars.discourse-cdn.com/v4/letter/j/bbce88/32.png) [@Josh\_Devins\_2](https://discuss.elastic.co/u/Josh_Devins_2)\ **Post date:** [September 16, 2011, 7:18am UTC](https://discuss.elastic.co/t/jclouds-provider/5337/11 "2011-09-16T07:18:33Z")
Hey Andrew, (As I mentioned in my email earlier) I haven't looked into starting the jclouds integration yet but probably will in the next 6-weeks. Let me know if you get started before that! Will update the ticket as I go. I agree that in principle/theory it should not be too complicated, but practice will always have some gotcha's! Hopefully any issues that arise can be dealt with my you/Adrian/jclouds. If you have time to work on this before I get to it, that would be great. It's low on my priority list right now since, as Shay mentioned, this is an optimization. You can still use local storage and unicast discovery on any cloud provider/DC/network setup. Josh ---
**Author:** ![James\_Cook](https://avatars.discourse-cdn.com/v4/letter/j/898d66/32.png) [@James\_Cook](https://discuss.elastic.co/u/James_Cook)\ **Post date:** [September 18, 2011, 11:09pm UTC](https://discuss.elastic.co/t/jclouds-provider/5337/12 "2011-09-18T23:09:16Z")
My brief glance at the API seemed to indicate that the concept of tags in (i.e. EC2) is not supported. Groups are supported. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:54am UTC](https://discuss.elastic.co/t/jclouds-provider/5337/13 "2017-07-06T03:54:11Z")