I'm considering replacing a Compass-based search in one of our
applications with an ElasticSearch-based equivalent. It's looking very
promising, however I'm concerned at the presence of Lucene classes
inside the elasticsearch-0.14.2 JAR. The Compass application
obviously has its own Lucene 2.x JARs, and there's a real danger of
these clashing.
My hope would be to ElasticSearch's java TransportClient to
communicate with the server, and I'd have thought there was no need
for it to have any knowledge of Lucene.
Would it be practical to repackage the ElasticSearch JARs to separate
the client code from the core ElasticSearch server code?
I could use the REST interface, I suppose, but I rather like the
java client, it'd be a shame to lose that.
The TransportClient should not require any Lucene classes, so you can use it simply without the Lucene classes being around. If you run into problems with it, ping me, and I will make it work. With the way the project is currently structured, its hard to test for it (having the TransportClient work without Lucene).
On Monday, January 17, 2011 at 5:09 PM, skaffman wrote:
Hi all,
I'm considering replacing a Compass-based search in one of our
applications with an Elasticsearch-based equivalent. It's looking very
promising, however I'm concerned at the presence of Lucene classes
inside the elasticsearch-0.14.2 JAR. The Compass application
obviously has its own Lucene 2.x JARs, and there's a real danger of
these clashing.
My hope would be to Elasticsearch's java TransportClient to
communicate with the server, and I'd have thought there was no need
for it to have any knowledge of Lucene.
Would it be practical to repackage the Elasticsearch JARs to separate
the client code from the core Elasticsearch server code?
I could use the REST interface, I suppose, but I rather like the
java client, it'd be a shame to lose that.
The TransportClient should not require any Lucene classes, so you can use it simply without the Lucene classes being around. If you run into problems with it, ping me, and I will make it work. With the way the project is currently structured, its hard to test for it (having the TransportClient work without Lucene).
On Monday, January 17, 2011 at 5:09 PM, skaffman wrote:
Hi all,
I'm considering replacing a Compass-based search in one of our
applications with an Elasticsearch-based equivalent. It's looking very
promising, however I'm concerned at the presence of Lucene classes
inside the elasticsearch-0.14.2 JAR. The Compass application
obviously has its own Lucene 2.x JARs, and there's a real danger of
these clashing.
My hope would be to Elasticsearch's java TransportClient to
communicate with the server, and I'd have thought there was no need
for it to have any knowledge of Lucene.
Would it be practical to repackage the Elasticsearch JARs to separate
the client code from the core Elasticsearch server code?
I could use the REST interface, I suppose, but I rather like the
java client, it'd be a shame to lose that.
Maybe I did not explain myself correctly. If you use the TransportClient, just use the elasticsearch jar without the Lucene jars, it should work.
On Monday, January 17, 2011 at 7:03 PM, skaffman wrote:
Hi Shay,
I'd really rather not have to repack the JAR myself, just to remove
the Lucene classes, it'd make upgrading a pain.
Is changing your distribution to formally separate client from server
really not an option?
The TransportClient should not require any Lucene classes, so you can use it simply without the Lucene classes being around. If you run into problems with it, ping me, and I will make it work. With the way the project is currently structured, its hard to test for it (having the TransportClient work without Lucene).
On Monday, January 17, 2011 at 5:09 PM, skaffman wrote:
Hi all,
I'm considering replacing a Compass-based search in one of our
applications with an Elasticsearch-based equivalent. It's looking very
promising, however I'm concerned at the presence of Lucene classes
inside the elasticsearch-0.14.2 JAR. The Compass application
obviously has its own Lucene 2.x JARs, and there's a real danger of
these clashing.
My hope would be to Elasticsearch's java TransportClient to
communicate with the server, and I'd have thought there was no need
for it to have any knowledge of Lucene.
Would it be practical to repackage the Elasticsearch JARs to separate
the client code from the core Elasticsearch server code?
I could use the REST interface, I suppose, but I rather like the
java client, it'd be a shame to lose that.
Yes, but the elasticsearch JAR itself contains Lucene classes (random
example - org.apache.lucene.search.FieldCache). That in itself is
rather confusing. Is this a packaging error?
Similarly, elasticsearch.jar contains a copy of
javax.annotation.Nullable. I suggest that this be moved into a
seperate JAR also, since sooner or later that's going to cause JavaEE
classloader problems.
The TransportClient should not require any Lucene classes, so you can use it simply without the Lucene classes being around. If you run into problems with it, ping me, and I will make it work. With the way the project is currently structured, its hard to test for it (having the TransportClient work without Lucene).
On Monday, January 17, 2011 at 5:09 PM, skaffman wrote:
Hi all,
I'm considering replacing a Compass-based search in one of our
applications with an Elasticsearch-based equivalent. It's looking very
promising, however I'm concerned at the presence of Lucene classes
inside the elasticsearch-0.14.2 JAR. The Compass application
obviously has its own Lucene 2.x JARs, and there's a real danger of
these clashing.
My hope would be to Elasticsearch's java TransportClient to
communicate with the server, and I'd have thought there was no need
for it to have any knowledge of Lucene.
Would it be practical to repackage the Elasticsearch JARs to separate
the client code from the core Elasticsearch server code?
I could use the REST interface, I suppose, but I rather like the
java client, it'd be a shame to lose that.
On Monday, January 17, 2011 at 7:16 PM, skaffman wrote:
Yes, but the elasticsearch JAR itself contains Lucene classes (random
example - org.apache.lucene.search.FieldCache). That in itself is
rather confusing. Is this a packaging error?
Thats not a packaging problem, its there to hack a feature into Lucene... .
Similarly, elasticsearch.jar contains a copy of
javax.annotation.Nullable. I suggest that this be moved into a
seperate JAR also, since sooner or later that's going to cause JavaEE
classloader problems.
It should not cause class loading problems, but I can move it.
The TransportClient should not require any Lucene classes, so you can use it simply without the Lucene classes being around. If you run into problems with it, ping me, and I will make it work. With the way the project is currently structured, its hard to test for it (having the TransportClient work without Lucene).
On Monday, January 17, 2011 at 5:09 PM, skaffman wrote:
Hi all,
I'm considering replacing a Compass-based search in one of our
applications with an Elasticsearch-based equivalent. It's looking very
promising, however I'm concerned at the presence of Lucene classes
inside the elasticsearch-0.14.2 JAR. The Compass application
obviously has its own Lucene 2.x JARs, and there's a real danger of
these clashing.
My hope would be to Elasticsearch's java TransportClient to
communicate with the server, and I'd have thought there was no need
for it to have any knowledge of Lucene.
Would it be practical to repackage the Elasticsearch JARs to separate
the client code from the core Elasticsearch server code?
I could use the REST interface, I suppose, but I rather like the
java client, it'd be a shame to lose that.
Thats not a packaging problem, its there to hack a feature into Lucene... .
Yeah, I thought as much. My point is that these lucene classes in the
elasticsearch.jar are going to clash with the "real" lucene 2.9 JARs
that are already in my application.
Can these hacked lucene classes not be extracted into an explicit,
separate JAR file?
Similarly, elasticsearch.jar contains a copy of
javax.annotation.Nullable. I suggest that this be moved into a
seperate JAR also, since sooner or later that's going to cause JavaEE
classloader problems.
It should not cause class loading problems, but I can move it.
Unfortunately, I've seen many cases where libraries have included
copies of javax.* classes that have caused problems. Many JavaEE
servers have strict classloading that forbids any attempt to replace
that package in an application's libraries.
On Monday, January 17, 2011 at 7:31 PM, skaffman wrote:
Thats not a packaging problem, its there to hack a feature into Lucene... .
Yeah, I thought as much. My point is that these lucene classes in the
elasticsearch.jar are going to clash with the "real" lucene 2.9 JARs
that are already in my application.
It might, though you can remove that class from ES jar. I am not going to provide another jar for it, since I aim at having a single ES jar (+lucene) for simplicity, and the common case is not having different Lucene versions in the same class loader.
The good news is that the hacked version will not be needed for long, as my enhancement should
Can these hacked lucene classes not be extracted into an explicit,
separate JAR file?
Similarly, elasticsearch.jar contains a copy of
javax.annotation.Nullable. I suggest that this be moved into a
seperate JAR also, since sooner or later that's going to cause JavaEE
classloader problems.
It should not cause class loading problems, but I can move it.
Unfortunately, I've seen many cases where libraries have included
copies of javax.* classes that have caused problems. Many JavaEE
servers have strict classloading that forbids any attempt to replace
that package in an application's libraries.
I'll change the location, no problem. I try to have ES jars be as little viral as possible when it comes to embedding it in another app (one of the main problem in Java) except for Lucene when applicable.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.