Keep my REST app as a plugin or not when migrating from 1.7 to 2.4+?

[Laurent_T]

Hi,

Some time ago I started migrating a plugin of mine from ES 1.7 to ES 2.4. The base code adaptation was easy and so was the the maven pom adaptations to build a proper plugin package.
However, once the plugin is deployed I'm facing a couple problems:

1. I didn't account for the added Java Security permissions in the code so now I'm facing potentially some hours of trial and error to find all permissions my code requires and adding them as well as encapsulating each part of the code in a doPrivileged. Those being required mostly by librairies I use, it doesn't seem right to have to adapt my code everywhere - I may be wrong though.
2. The resources from my jars do not seem to be available in the classpath anymore: I can see them at the root of my jar contained in the plugin zip but the app doesn't find them.

I've been integrating my app as a plugin for 2 reasons:

1. Being inside ES allows for really good performance when searching our index.
2. The REST layer seems much more optimized than anything I could code until now achieving higher throughput.

The app is already pluggable to any HTTP handler with no change to it's code. I just have to use it in an ES Plugin or in a standalone project for instance. As the idea was to someday leave the ES plugin environment, I thought now would be a good time. However, i'm having some really bad performances when I use my app with a transport client rather than with the embedded client the plugin mode provides.

Is the lower request throughput something expected ? If so, is there anyway I could leave the ES plugin environment and keep good performances ?
On the other hand, is there some way to bypass the issues I have as a plugin ? For instance tell ES not to check for permissions.

FYI, my plugin only reads the index and the index update is done asynchronously by an other app using new indices and read/write aliases. In time i'd like to have my index directly searchable in my app's JVM but without the whole ES stack to keep things as light as possible and focus on achieving high QPS.

Note: All that was before ES 5 was released but today i'd probably go for the latest version.

[jprante (Jörg Prante)]

@Laurent_T

However, i'm having some really bad performances when I use my app with a transport client rather than with the embedded client the plugin mode provides.

Is the lower request throughput something expected ?

How much lower?

I'm not sure what your plugin is executing inside ES. What search actions in transport client do not have a comparable performance to node client, and how do you measure the difference? It should be only a negligible difference at all.

If your plugin code really just searches the index, you should consider to move this search logic into your application.

Meanwhile, you can disable security manager in ES 2.x with

./bin/elasticsearch --security.manager.enabled false

Scripting and the Java Security Manager | Elasticsearch Guide [2.4] | Elastic

Note that disabling security manager is no longer possible in ES 5.