Elasticsearch in a .war: where to put the plugins directory?

Hi Everyone,
I'm deploying elasticsearch as part of my web application, packaged as a
.war file and using the elasticsearch-transport-wares to start up.

In the typical elasticsearch deployment, installing a plugin basically
amounts to copying a .jar file into the plugins directory, and you can
configure the location of that directory using "path.plugins". I'm
struggling to get this working in my deployment, and trying the following:

in elasticsearch.yml:
path.plugins: /plugins

jar -tvf ./target/my-webapp.war | grep plugins
68431 Tue Mar 05 17:15:58 EST 2013
WEB-INF/classes/plugins/elasticfacets/ElasticFacetsPlugin-0.2.2-SNAPSHOT.jar

is this not the right location? what am I missing?

thanks in advance!

--paul

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Just put elasticfacets plugin (and others) in WEB-INF/lib and it should
work. No need for the path.plugins setting, since the jar and the
es-plugin.properties can be picked up on the classpath by the webapp
classloader.

I have tried to improve plugin loading with an "Elasticsearch apps"
effort. This alternative approach to path.plugins replaces the
./bin/plugin tool and is loading the plugins together with their
dependencies ("the apps") from repositories at node startup time, also
using a separate classloader for the plugins. Setup of dependencies and
repositories is like in a Maven pom.xml. What is missing is classpath
isolation between plugins. See https://github.com/jprante/elasticsearch-apps

Jörg

Am 05.03.13 23:46, schrieb Paul Sanwald:

Hi Everyone,
I'm deploying elasticsearch as part of my web application, packaged
as a .war file and using the elasticsearch-transport-wares to start up.

In the typical elasticsearch deployment, installing a plugin basically
amounts to copying a .jar file into the plugins directory, and you can
configure the location of that directory using "path.plugins". I'm
struggling to get this working in my deployment, and trying the following:

in elasticsearch.yml:
path.plugins: /plugins

jar -tvf ./target/my-webapp.war | grep plugins
68431 Tue Mar 05 17:15:58 EST 2013
WEB-INF/classes/plugins/elasticfacets/ElasticFacetsPlugin-0.2.2-SNAPSHOT.jar

is this not the right location? what am I missing?

thanks in advance!

--paul

You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Jörg,
hmm, I tried putting the "plugins" directory in WEB-INF/lib, but no
dice, the logs don't show the plugin as being loaded, whereas in my
standalone instance, I see:
[2013-03-05 16:55:06,184][INFO
][org.leskes.elasticfacets.ElasticFacetsPlugin] ElasticFacets plugin
initialized

strangely, I don't seem to see es-plugin.properties anywhere. From your
message, it seems like the plugins loader scans the classpath for a
directory called "plugins", possibly the elasticsearch-transport-wares
NodeServlet isn't doing this? I will check the source.

--paul

On Tuesday, March 5, 2013 6:16:13 PM UTC-5, Jörg Prante wrote:

Just put elasticfacets plugin (and others) in WEB-INF/lib and it should
work. No need for the path.plugins setting, since the jar and the
es-plugin.properties can be picked up on the classpath by the webapp
classloader.

I have tried to improve plugin loading with an "Elasticsearch apps"
effort. This alternative approach to path.plugins replaces the
./bin/plugin tool and is loading the plugins together with their
dependencies ("the apps") from repositories at node startup time, also
using a separate classloader for the plugins. Setup of dependencies and
repositories is like in a Maven pom.xml. What is missing is classpath
isolation between plugins. See
https://github.com/jprante/elasticsearch-apps

Jörg

Am 05.03.13 23:46, schrieb Paul Sanwald:

Hi Everyone,
I'm deploying elasticsearch as part of my web application, packaged
as a .war file and using the elasticsearch-transport-wares to start up.

In the typical elasticsearch deployment, installing a plugin basically
amounts to copying a .jar file into the plugins directory, and you can
configure the location of that directory using "path.plugins". I'm
struggling to get this working in my deployment, and trying the
following:

in elasticsearch.yml:
path.plugins: /plugins

jar -tvf ./target/my-webapp.war | grep plugins
68431 Tue Mar 05 17:15:58 EST 2013

WEB-INF/classes/plugins/elasticfacets/ElasticFacetsPlugin-0.2.2-SNAPSHOT.jar

is this not the right location? what am I missing?

thanks in advance!

--paul

You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Do not worry about a "plugins" folder. Just put the JAR of the plugin
(or the JARs) in WEB-INF/lib, like you would do with any other in-WAR
dependency. es-plugin.properties is in a plugin JAR on the root level.
An App server is loading all the JARs and adds them to the class path of
the web app, so everything should work just by putting the JARs in place.

Jörg

Am 06.03.13 15:15, schrieb Paul Sanwald:

Jörg,
hmm, I tried putting the "plugins" directory in WEB-INF/lib, but no
dice, the logs don't show the plugin as being loaded, whereas in my
standalone instance, I see:
[2013-03-05 16:55:06,184][INFO
][org.leskes.elasticfacets.ElasticFacetsPlugin] ElasticFacets plugin
initialized

strangely, I don't seem to see es-plugin.properties anywhere. From
your message, it seems like the plugins loader scans the classpath for
a directory called "plugins", possibly the
elasticsearch-transport-wares NodeServlet isn't doing this? I will
check the source.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Jörg,
your help is much appreciated! I've added the jar to my WEB-INF/lib
directory, but it's still not loading. I feel as though I may be missing
some step to tell my node that it should scan for plugins.

I've confirmed that the plugin jar does contain the properties file:

$ jar -tvf
target/my-exploded-war/WEB-INF/lib/ElasticFacetsPlugin-0.2.2-SNAPSHOT.jar |
grep es-plugin.properties
51 Sat Feb 02 15:57:02 EST 2013 es-plugin.properties

but, nothing in the logs that indicate the plugin is being loaded.

--paul

On Wednesday, March 6, 2013 9:35:25 AM UTC-5, Jörg Prante wrote:

Do not worry about a "plugins" folder. Just put the JAR of the plugin
(or the JARs) in WEB-INF/lib, like you would do with any other in-WAR
dependency. es-plugin.properties is in a plugin JAR on the root level.
An App server is loading all the JARs and adds them to the class path of
the web app, so everything should work just by putting the JARs in place.

Jörg

Am 06.03.13 15:15, schrieb Paul Sanwald:

Jörg,
hmm, I tried putting the "plugins" directory in WEB-INF/lib, but no
dice, the logs don't show the plugin as being loaded, whereas in my
standalone instance, I see:
[2013-03-05 16:55:06,184][INFO
][org.leskes.elasticfacets.ElasticFacetsPlugin] ElasticFacets plugin
initialized

strangely, I don't seem to see es-plugin.properties anywhere. From
your message, it seems like the plugins loader scans the classpath for
a directory called "plugins", possibly the
elasticsearch-transport-wares NodeServlet isn't doing this? I will
check the source.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Not sure what you are doing, how does the code look like that starts the
node or the transport client in the web app? Can you gist an example?

Jörg

Am 06.03.13 17:44, schrieb Paul Sanwald:

Jörg,
your help is much appreciated! I've added the jar to my WEB-INF/lib
directory, but it's still not loading. I feel as though I may be
missing some step to tell my node that it should scan for plugins.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

I'm using the elasticsearch wares plugin to fire up the node:

I've configured it in my web.xml as follows:

node org.elasticsearch.wares.NodeServlet 1

--paul

On Wednesday, March 6, 2013 12:58:04 PM UTC-5, Jörg Prante wrote:

Not sure what you are doing, how does the code look like that starts the
node or the transport client in the web app? Can you gist an example?

Jörg

Am 06.03.13 17:44, schrieb Paul Sanwald:

Jörg,
your help is much appreciated! I've added the jar to my WEB-INF/lib
directory, but it's still not loading. I feel as though I may be
missing some step to tell my node that it should scan for plugins.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

I never used the NodeServlet but I will try to reproduce...

Jörg

Am 06.03.13 20:00, schrieb Paul Sanwald:

I'm using the elasticsearch wares plugin to fire up the node:

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Jörg, just got this working! it was a maven problem on my side, the jar I
was including had gotten corrupted somehow. thanks for your help, I'm using
the plugin successfully now.

--paul

On Wednesday, March 6, 2013 3:14:38 PM UTC-5, Jörg Prante wrote:

I never used the NodeServlet but I will try to reproduce...

Jörg

Am 06.03.13 20:00, schrieb Paul Sanwald:

I'm using the elasticsearch wares plugin to fire up the node:

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

FYI I just opened a pull request
https://github.com/elasticsearch/elasticsearch-transport-wares/pull/8 in
the hope the ES deployment experience as a web app can be improved, just
by providing a simple web.xml and a build routine for a simple es.war.
There are no plugins in WEB-INF/lib. Tested against JBoss AS 7.1.1

I hope this can be useful.

Jörg

Am 06.03.13 21:25, schrieb Paul Sanwald:

Jörg, just got this working! it was a maven problem on my side, the
jar I was including had gotten corrupted somehow. thanks for your
help, I'm using the plugin successfully now.

--paul

On Wednesday, March 6, 2013 3:14:38 PM UTC-5, Jörg Prante wrote:

I never used the NodeServlet but I will try to reproduce...

Jörg

Am 06.03.13 20:00, schrieb Paul Sanwald:
> I'm using the elasticsearch wares plugin to fire up the node:

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.