Running ES Client inside ES Plugin


(Vadim Voituk) #1

Hello,

I'm developing ES plugin that need to get some static data (let's call it
"list") from the ES-index.
What is best way to do this?

Of course it can be implemented the same way as "external" ES-client
(create client, connect to node, etc)
But i guess there are more effective way to do this.

For example, something like: get current node, send and some kind of
low-level request to the index needed.

Looking forward for any comments, ideas, considerations...

--
Vadim


(Alexander Reelsen) #2

Hi

On 20 Apr., 14:09, Vadim Voituk vadim.voi...@gmail.com wrote:

Of course it can be implemented the same way as "external" ES-client
(create client, connect to node, etc)
But i guess there are more effective way to do this.
No need to create a client, simply use the @Inject annotation in your
plugin to insert a client instance - see other sample plugins and
rivers how they do this.
Or am I missing something why this is not possible in your sample?

--Alexander


(Vadim Voituk) #3

On Friday, April 20, 2012 2:22:53 PM UTC+2, Alexander Reelsen wrote:

Of course it can be implemented the same way as "external" ES-client
(create client, connect to node, etc)
But i guess there are more effective way to do this.
No need to create a client, simply use the @Inject annotation in your
plugin to insert a client instance - see other sample plugins and
rivers how they do this.
Or am I missing something why this is not possible in your sample?

Alexander,
thanks for your answer.
Could you please point me to an example.

The only i found – injection of the client into child class
BaseRestController, but in my case i need to inject it
into NativeScriptFactory.

Something like:

public class XXXScriptFactory implements NativeScriptFactory
@Inject Client client;
....
}
But i'm getting error like:

"Initialization Failed ...

  1. Tried proxying org.elasticsearch.script.ScriptService to support a
    circular dependency, but it is not an interface."

(Shay Banon) #4

Add the Client to the constructor of your implementation of
NativeScriptFactory (and make sure to have @Inject on the constructor as
well).

On Fri, Apr 20, 2012 at 4:33 PM, Vadim Voituk vadim.voituk@gmail.comwrote:

On Friday, April 20, 2012 2:22:53 PM UTC+2, Alexander Reelsen wrote:

Of course it can be implemented the same way as "external" ES-client
(create client, connect to node, etc)
But i guess there are more effective way to do this.
No need to create a client, simply use the @Inject annotation in your
plugin to insert a client instance - see other sample plugins and
rivers how they do this.
Or am I missing something why this is not possible in your sample?

Alexander,
thanks for your answer.
Could you please point me to an example.

The only i found – injection of the client into child class
BaseRestController, but in my case i need to inject it
into NativeScriptFactory.

Something like:

public class XXXScriptFactory implements NativeScriptFactory
@Inject Client client;
....
}
But i'm getting error like:

"Initialization Failed ...

  1. Tried proxying org.elasticsearch.script.ScriptService to support a
    circular dependency, but it is not an interface."

(Vadim Voituk) #5

Hello, Shay

I'm trying to do in this way:

On Saturday, April 21, 2012 5:07:01 PM UTC+2, kimchy wrote:

Add the Client to the constructor of your implementation of
NativeScriptFactory (and make sure to have @Inject on the constructor as
well).

.. and here what i getting:

  1. Tried proxying org.elasticsearch.script.ScriptService to support a
    circular dependency, but it is not an interface.
  2. IllegalStateException[This is a proxy used to support circular
    references involving constructors. The object we're proxying is not
    constructed yet. Please wait until after injection has completed to use
    this object.]

And here is my code:

public class PreorderSortScriptFactory implements NativeScriptFactory {
@Inject public PreorderSortScriptFactory(Client client) {

    System.out.println("PreorderSortScriptFactory.PreorderSortScriptFactory() 
  • " + client );
    }

(avasilenko) #6

Vadim,

Looks like you encountered circular dependencies. Have you tried injecting
via setter?

Alexandr Vasilenko
Skype:menterr

2012/4/23 Vadim Voituk vadim.voituk@gmail.com

Hello, Shay

I'm trying to do in this way:

On Saturday, April 21, 2012 5:07:01 PM UTC+2, kimchy wrote:

Add the Client to the constructor of your implementation of
NativeScriptFactory (and make sure to have @Inject on the constructor as
well).

.. and here what i getting:

  1. Tried proxying org.elasticsearch.script.ScriptService to support a
    circular dependency, but it is not an interface.
  2. IllegalStateException[This is a proxy used to support circular
    references involving constructors. The object we're proxying is not
    constructed yet. Please wait until after injection has completed to use
    this object.]

And here is my code:

public class PreorderSortScriptFactory implements NativeScriptFactory {
@Inject public PreorderSortScriptFactory(Client client) {

    System.out.println("PreorderSortScriptFactory.PreorderSortScriptFactory()
  • " + client );
    }

(Vadim Voituk) #7

Hello, Alex,

On Monday, April 23, 2012 12:49:41 PM UTC+2, Alex Vasilenko wrote:

Vadim,

Looks like you encountered circular dependencies. Have you tried injecting
via setter?

Yeah, ai've tried in this way too:

@Inject public void setClient(Client client) { ... }

But the error is the same.

And i don't think that there are much of differences in injection
lifecycle there :slight_smile:


(Shay Banon) #8

Yea, I see why it happens... . Try and do the (not as nice solution) of
injecting the factory with Injector and keep it in a field. And then, on
newScript called on the factor, call: injector.getInstance(Client.class) to
get the client.

On Mon, Apr 23, 2012 at 3:44 PM, Vadim Voituk vadim.voituk@gmail.comwrote:

Hello, Alex,

On Monday, April 23, 2012 12:49:41 PM UTC+2, Alex Vasilenko wrote:

Vadim,

Looks like you encountered circular dependencies. Have you tried
injecting via setter?

Yeah, ai've tried in this way too:

@Inject public void setClient(Client client) { ... }

But the error is the same.

And i don't think that there are much of differences in injection
lifecycle there :slight_smile:


(Vadim Voituk) #9

Hello, Shay

Trying in this way:

@Inject public PreorderSortScriptFactory(Injector injector) {
super();
logger.info("" + injector);
}

but as result ES even not starts – java process silently dies.

Here is what i see from log

[2012-04-26 10:57:14,227][INFO ][node ] [Williams,
Simon] {0.19.2}[7659]: initializing ...

[2012-04-26 10:57:14,236][INFO ][plugins ] [Williams,
Simon] loaded [Pre-Ordered Sorting], sites []

[2012-04-26 10:57:16,512][INFO ][node ] [Bounty Hunter]
{0.19.2}[7659]: initializing ...
[2012-04-26 10:57:16,513][INFO ][plugins ] [Bounty Hunter]
loaded [Pre-Ordered Sorting], sites []

--
Voituk Vadim
vadim.voituk@gmail.com
SkypeID: voituk

On Wed, Apr 25, 2012 at 17:54, Shay Banon kimchy@gmail.com wrote:

Yea, I see why it happens... . Try and do the (not as nice solution) of
injecting the factory with Injector and keep it in a field. And then, on
newScript called on the factor, call: injector.getInstance(Client.class) to
get the client.

On Mon, Apr 23, 2012 at 3:44 PM, Vadim Voituk vadim.voituk@gmail.comwrote:

Hello, Alex,

On Monday, April 23, 2012 12:49:41 PM UTC+2, Alex Vasilenko wrote:

Vadim,

Looks like you encountered circular dependencies. Have you tried
injecting via setter?

Yeah, ai've tried in this way too:

@Inject public void setClient(Client client) { ... }

But the error is the same.

And i don't think that there are much of differences in injection
lifecycle there :slight_smile:


(Shay Banon) #10

I;ve ran a simpel test with injecting the Injector, and it seems to
work..., maybe you can share your code and I can have a look?

On Thu, Apr 26, 2012 at 1:58 PM, Vadim Voituk vadim.voituk@gmail.comwrote:

Hello, Shay

Trying in this way:

@Inject public PreorderSortScriptFactory(Injector injector) {
super();
logger.info("" + injector);
}

but as result ES even not starts – java process silently dies.

Here is what i see from log

[2012-04-26 10:57:14,227][INFO ][node ] [Williams,
Simon] {0.19.2}[7659]: initializing ...

[2012-04-26 10:57:14,236][INFO ][plugins ] [Williams,
Simon] loaded [Pre-Ordered Sorting], sites []

[2012-04-26 10:57:16,512][INFO ][node ] [Bounty
Hunter] {0.19.2}[7659]: initializing ...
[2012-04-26 10:57:16,513][INFO ][plugins ] [Bounty
Hunter] loaded [Pre-Ordered Sorting], sites []

--
Voituk Vadim
vadim.voituk@gmail.com
SkypeID: voituk

On Wed, Apr 25, 2012 at 17:54, Shay Banon kimchy@gmail.com wrote:

Yea, I see why it happens... . Try and do the (not as nice solution) of
injecting the factory with Injector and keep it in a field. And then, on
newScript called on the factor, call: injector.getInstance(Client.class) to
get the client.

On Mon, Apr 23, 2012 at 3:44 PM, Vadim Voituk vadim.voituk@gmail.comwrote:

Hello, Alex,

On Monday, April 23, 2012 12:49:41 PM UTC+2, Alex Vasilenko wrote:

Vadim,

Looks like you encountered circular dependencies. Have you tried
injecting via setter?

Yeah, ai've tried in this way too:

@Inject public void setClient(Client client) { ... }

But the error is the same.

And i don't think that there are much of differences in injection
lifecycle there :slight_smile:


(Vadim Voituk) #11

Hello, Shay

The code is pretty straightforward:

public class PreorderSortScriptFactory implements NativeScriptFactory {

private static final ESLogger logger =
Loggers.getLogger("com.voituk.PreorderSort");

private Injector injector;
@Inject
public PreorderSortScriptFactory(Injector injector) {
super();
this.injector = injector;
}

@Override
public ExecutableScript newScript(Map<String, Object> params) {
super();
logger.info("" + injector);

}

}

I'm using ES 0.19.2 on Debian with 4Gb specified as ES_HEAP_SIZE variable.
And the most strange thing that i can't see any exception or something like
that even in debug mode, so probably it's some kind of internal exception
happens that crashes JVM silently.

Anyway, i've managed the initial problem by creating separate client node
in this way:

Node node = NodeBuilder.nodeBuilder().client(true).node();

But it would be nice to to solve this in a right way - via @Injection.

Thanks.

On Friday, April 27, 2012 12:22:59 PM UTC+2, kimchy wrote:

I;ve ran a simpel test with injecting the Injector, and it seems to
work..., maybe you can share your code and I can have a look?


(Shay Banon) #12

Thats what I run and it works. If you can share a full project I can have a
look why it fails.

On Mon, Apr 30, 2012 at 10:54 AM, Vadim Voituk vadim.voituk@gmail.comwrote:

Hello, Shay

The code is pretty straightforward:

public class PreorderSortScriptFactory implements NativeScriptFactory {

private static final ESLogger logger =
Loggers.getLogger("com.voituk.PreorderSort");

private Injector injector;
@Inject
public PreorderSortScriptFactory(Injector injector) {
super();
this.injector = injector;
}

@Override
public ExecutableScript newScript(Map<String, Object> params) {
super();
logger.info("" + injector);

}

}

I'm using ES 0.19.2 on Debian with 4Gb specified as ES_HEAP_SIZE variable.
And the most strange thing that i can't see any exception or something
like that even in debug mode, so probably it's some kind of internal
exception happens that crashes JVM silently.

Anyway, i've managed the initial problem by creating separate client node
in this way:

Node node = NodeBuilder.nodeBuilder().client(true).node();

But it would be nice to to solve this in a right way - via @Injection.

Thanks.

On Friday, April 27, 2012 12:22:59 PM UTC+2, kimchy wrote:

I;ve ran a simpel test with injecting the Injector, and it seems to
work..., maybe you can share your code and I can have a look?


(Vadim Voituk) #13

Hello,

Will try to make empty project with this code only and see if it'll crash.

Will keep you updated.

Thanks,

Thats what I run and it works. If you can share a full project I can have a

look why it fails.

Hello, Shay

The code is pretty straightforward:

public class PreorderSortScriptFactory implements NativeScriptFactory {

private static final ESLogger logger =
Loggers.getLogger("com.voituk.PreorderSort");

private Injector injector;
@Inject
public PreorderSortScriptFactory(Injector injector) {
super();
this.injector = injector;
}

@Override
public ExecutableScript newScript(Map<String, Object> params) {
super();
logger.info("" + injector);

}

}

I'm using ES 0.19.2 on Debian with 4Gb specified as ES_HEAP_SIZE variable.
And the most strange thing that i can't see any exception or something
like that even in debug mode, so probably it's some kind of internal
exception happens that crashes JVM silently.

Anyway, i've managed the initial problem by creating separate client node
in this way:

Node node = NodeBuilder.nodeBuilder().client(true).node();

But it would be nice to to solve this in a right way - via @Injection.

Thanks.

On Friday, April 27, 2012 12:22:59 PM UTC+2, kimchy wrote:

I;ve ran a simpel test with injecting the Injector, and it seems to
work..., maybe you can share your code and I can have a look?


(Vadim Voituk) #14

Hello, Shay

Here is entire project code
https://github.com/voituk/elasticsearch-test-plugin

and that's what i see in log
[2012-05-07 13:57:06,944][INFO ][node ] [Sludge]
{0.19.3}[19516]: stopping ...
[2012-05-07 13:57:08,105][INFO ][node ] [Sludge]
{0.19.3}[19516]: stopped
[2012-05-07 13:57:08,105][INFO ][node ] [Sludge]
{0.19.3}[19516]: closing ...
[2012-05-07 13:57:08,177][INFO ][node ] [Sludge]
{0.19.3}[19516]: closed
[2012-05-07 13:57:13,942][INFO ][node ] [Cobra]
{0.19.3}[23642]: initializing ...
[2012-05-07 13:57:13,952][INFO ][plugins ] [Cobra] loaded
[Test Plugin], sites []

And nothing after that, and no even java process running.

java -version
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)

Here is now the ES is started (but don't think that it matters in this
case):
/usr/lib/jvm/java-6-sun/bin/java -Xms1g -Xmx1g -Xss256k -XX:+UseParNewGC
-XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError
-Delasticsearch -Des.pidfile=/var/run/elasticsearch.pid
-Des.path.home=/usr/share/elasticsearch -cp
:/usr/share/elasticsearch/lib/:/usr/share/elasticsearch/lib/sigar/
-Des.config=/etc/elasticsearch/elasticsearch.yml
-Des.path.home=/usr/share/elasticsearch
-Des.path.logs=/var/log/elasticsearch -Des.path.data=/var/lib/elasticsearch
-Des.path.work=/tmp/elasticsearch -Des.path.conf=/etc/elasticsearch
org.elasticsearch.bootstrap.ElasticSearch


(Vadim Voituk) #15

Hmm,

I've just removed everything not related to the problem described, and it
works fine.
So definitely it's not an ElasticSearch but my project setup problem.

Sorry for disturbing.

On Monday, May 7, 2012 4:06:28 PM UTC+2, Vadim Voituk wrote:

Hello, Shay

Here is entire project code
https://github.com/voituk/elasticsearch-test-plugin

and that's what i see in log
[2012-05-07 13:57:06,944][INFO ][node ] [Sludge]
{0.19.3}[19516]: stopping ...
[2012-05-07 13:57:08,105][INFO ][node ] [Sludge]
{0.19.3}[19516]: stopped
[2012-05-07 13:57:08,105][INFO ][node ] [Sludge]
{0.19.3}[19516]: closing ...
[2012-05-07 13:57:08,177][INFO ][node ] [Sludge]
{0.19.3}[19516]: closed
[2012-05-07 13:57:13,942][INFO ][node ] [Cobra]
{0.19.3}[23642]: initializing ...
[2012-05-07 13:57:13,952][INFO ][plugins ] [Cobra] loaded
[Test Plugin], sites []

And nothing after that, and no even java process running.

java -version
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)

Here is now the ES is started (but don't think that it matters in this
case):
/usr/lib/jvm/java-6-sun/bin/java -Xms1g -Xmx1g -Xss256k -XX:+UseParNewGC
-XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError
-Delasticsearch -Des.pidfile=/var/run/elasticsearch.pid
-Des.path.home=/usr/share/elasticsearch -cp
:/usr/share/elasticsearch/lib/:/usr/share/elasticsearch/lib/sigar/
-Des.config=/etc/elasticsearch/elasticsearch.yml
-Des.path.home=/usr/share/elasticsearch
-Des.path.logs=/var/log/elasticsearch -Des.path.data=/var/lib/elasticsearch
-Des.path.work=/tmp/elasticsearch -Des.path.conf=/etc/elasticsearch
org.elasticsearch.bootstrap.ElasticSearch


(Eric) #16

How to use this injection in 5.0.0 as now we don't register the factory but we return the object already created

How to inject this Injector in my plugin class ?


(system) #17