Cloud-aws for ES 2.1.1 in maven

Elastic Stack Elasticsearch
1

Hi,

I'm trying to upgrade a java client node from ES 1.7 to 2.1.1 and I'm stuck on the cloud-aws portion of the upgrade. Specifically,

My ES 1.7-compatible code successfully uses the elasticsearch-cloud-aws artifact:

<dependency>
  <groupId>org.elasticsearch</groupId>
  <artifactId>elasticsearch-cloud-aws</artifactId>
  <version>2.7.1</version>
</dependency>

I know that the plugin has moved into the main elasticsearch project but, unless I'm missing something, it's not included in the elasticsearch java client artifact.

Any guidance on how to incorporate cloud-aws in to a java node client would be greatly appreciated!

Currently, my client code is throwing the follow exception related to the absence of the discovery-ec2 code:

Caused by: java.lang.IllegalArgumentException: Unknown Discovery type [ec2]
at org.elasticsearch.discovery.DiscoveryModule.configure(DiscoveryModule.java:100)
at org.elasticsearch.common.inject.AbstractModule.configure(AbstractModule.java:61)
at org.elasticsearch.common.inject.spi.Elements$RecordingBinder.install(Elements.java:233)
at org.elasticsearch.common.inject.spi.Elements.getElements(Elements.java:105)
at org.elasticsearch.common.inject.InjectorShell$Builder.build(InjectorShell.java:143)
at org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.java:99)
at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:93)
at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:70)
at org.elasticsearch.common.inject.ModulesBuilder.createInjector(ModulesBuilder.java:46)
at org.elasticsearch.node.Node.<init>(Node.java:200)
at org.elasticsearch.node.Node.<init>(Node.java:128)
at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:145)
at org.elasticsearch.node.NodeBuilder.node(NodeBuilder.java:152)
2

The plugins are available via the org.elasticsearch.plugin groupId

<dependency>
    <groupId>org.elasticsearch.plugin</groupId>
    <artifactId>cloud-aws</artifactId>
    <version>2.1.1</version>
</dependency>
3

Does this work well? I'd expect that it does not load the cloud-aws plugin. Am I wrong?

4

You're correct -- it does not load the plugin and, as a result, it does not work.

I'm trying to figure out how to get the plugin to load -- any pointers?

(ps my client is a node client not a transport client)

5

Fwiw, I haven't found anything in the ES client library to support my use case "out of the box".

Out of sheer desperation, I've tried adding the CloudAwsPlugin via a custom Node class (see below). The plugin tries to load but then the application blows up with a ton of guice injection errors surrounding objects that have not yet been instantiated.

public class CloudAwsEnabledNode extends Node {
    public CloudAwsEnabledNode(Settings settings) {
        super(InternalSettingsPreparer.prepareEnvironment(settings, null),
                Version.CURRENT,
                ImmutableList.<Class<? extends Plugin>>of(CloudAwsPlugin.class));
    }
}

If you (or anyone) has any suggestions on approaches, I'm all ears! :slight_smile:

6

Ok, for the record, the hack described above for loading a plugin seems to work just fine for ES 2.1.1.

The guice injection problems I had stemmed from a version conflict between the jackson-databind artifact used by my project and the one used by the cloud-aws plugin (or, more precisely, by the underlying aws-java-sdk artifact that the cloud plugin uses). Once I resolved the dependency conflict, it worked.

7

Thank you for confirming that.

I think it deserves an issue in github which can be fixed either by add a addPlugin method to Node or by adding some documentation in the Java API guide.

Would you mind opening that?

8

Not at all, I had already started. Here it is: https://github.com/elastic/elasticsearch/issues/15927

9

I didn't find this when I initially searched on this topic but for anyone who stumbles across this thread, note the existence of Add plugins from classpath in embedded Elasticsearch client node

10

Looks like the Maven artifacts for this (and other?) plugins are no longer being updated (the latest published version is 5.0.0-alpha5, while elasticsearch is now at 5.4.0).

11

I believe it's not needed by the transport client?

12

Not needed, or does the transport client simply not support EC2 discovery (and other AWS-related functionality)?

Edit: Answered at EC2 discovery with TransportClient? - #2 by dadoonet