Installing custom Elasticsearch plugins in Elastic Cloud

Elastic Stack Elasticsearch
1

Hello,

I'm a trial user of Elastic cloud, and I can see it's technically possible to install custom plugins in Elasticsearch nodes:

image
image1920×1633 61.3 KB

However, as you can see, I'm not allowed to use this feature for my "subscription level".

What license would I need to do my tests?

The pricing page has no reference to this feature in any license level.

Extra permissions?

Another question is, when our plugin gets installed in Elasticsearch on premises, it asks user confirmation like so:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@     WARNING: plugin requires additional permissions     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
* java.lang.RuntimePermission accessClassInPackage.sun.misc
* java.lang.RuntimePermission accessDeclaredMembers
* java.lang.RuntimePermission getClassLoader
* java.lang.RuntimePermission setContextClassLoader
* java.lang.reflect.ReflectPermission suppressAccessChecks
* java.net.SocketPermission * connect,accept,resolve
* java.security.SecurityPermission insertProvider
* java.security.SecurityPermission putProviderProperty.BCFIPS
* java.security.SecurityPermission putProviderProperty.BCJSSE
* java.util.PropertyPermission * read,write
See https://docs.oracle.com/javase/8/docs/technotes/guides/security/permissions.html
for descriptions of what these permissions allow and the associated risks.

Will this ever work in Elastic Cloud?

Thank you!

2

Hi @Simone_Scarduzio

There are probably 2 things going on (and I see you have another question on Cloud)

Custom plugins can include the official Elasticsearch plugins not provided with Elasticsearch Service, any of the community-sourced plugins, or plugins that you write yourself. Uploading custom plugins is available only to Gold, Platinum and Enterprise subscriptions. For more information, check Upload custom plugins and bundles.

First exactly which plugin do you intend to use? I am not sure all are supported in the cloud.

You should review this

Second and and the actual issue.. one of the very few things you can not do with a Trial License is custom plugins. I think you can if you engage support / open a support ticket which I think you can do with a trial then I think you may be able to do that. Custom plugins are pretty tightly controlled.

Finally it looks like you are a bit of a sophisticated user if you would like to DM me with contact information (contact and company etc) I can have a solutions architect reach out to you for some additional help.

3

As a consultant, I'm trying to migrate a customer data from on premises to your managed service, without changing the way their systems integrate with Elasticsearch.

After further analysis, even if we managed get past the security permissions checks, we wouldn't anyway be able to integrate this huge plugin due to some extra modifications needed at the filesystem level to make it work.

So we opted for the second option:

  • A cluster on Elastic Cloud stores all their data
  • A container (external to Elastic Cloud, but running on the same Azure region) with an Elasticsearch client node (no shards, no master eligible)
  • We let the client node join the cloud cluster (once we manage to let them share the certs, see other topic)
  • We Install the plugin in the client node, so queries go through the plugin as usual

WDYT?

4

Perhaps semantics?

  • We let the client node join the cloud cluster (once we manage to let them share the certs, see other topic)

As far as I know,
An external self manage node cannot "join" an Elastic Cloud cluster, That breaks the security model.. If you mean you're setting it up as a single node cluster and then doing cross cluster search, that's okay.

The establishing of trust is between clusters not nodes from one cluster to nodes to another cluster... Even though part of that trust mentions node, it's about remote clusters... Not remote nodes.

As far as I know, you cannot have an external node join a cloud cluster.

1 Like
5

Yes, what you said makes total sense. And having two clusters in CCS is equally good, if not even better in term of separation of concerns.

We can keep all the data in the Elastic Cloud, and we can have a tiny ~empty cluster configured to cross-cluster search to the Cloud one, and from there we are just going to refer to indices as "cloud_cluster:index_xyz" instead of "index_xyz".

If this is ok for you, I think this is the way forward. So the only knot remaining here is the very "practical" format issue in the cert exchange, as I talked about in my other topic.

6

Not me ... its how it works... :slight_smile: But yes, this is the right path forward.

I am looking into the other question may take a day or so to get back, some people are OOO right now.

My default cert loaded right up into the cloud .... I will respond in that one when I get a chance...

For Clarity what version are you using?

1 Like
7

Thanks! At the moment I've simply spun up a random VPS, installed ES and Kibana 8.3.3 (latest) with the default configuration (SSL enabled both transport and http).

8

Excellent and just for the Record, you are on the bleeding edge CCS from Self Managed to ESS was just released and I suspect there are a few config items lurking in the edges.

BUT I did just upload the 8.3.3 self managed cert into ESS no issues... When I respond I will respond in that thread to help anyone else.. .it may take a couple days

Did you use the default certs or generate your own?

1 Like
9

Thank you @stephenb, much appreciated. I will update the thread with all the info in the meantime.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.