Running ES as a OSGi bundle

Elastic Stack Elasticsearch
1

Hello,
I am building an application on the OSGi platform and intending to use
ES for indexing and searching. My requirement is to make ES available
as a bundle and make it run on OSGi runtime along with my other
bundles. Any idea how I could go about achieving this?
Thanks!
Shruthi

2

Hello,

  As elasticsearch jar already bundles most of it's dependencies (look

at org.elasticsearch.common package) I think it's just a matter of using
felix's maven-bundle-plugin and exporting org.elasticsearch.*.

Cheers

On Mon, May 9, 2011 at 06:46, shruthi shruthiarjun@gmail.com wrote:

Hello,
I am building an application on the OSGi platform and intending to use
ES for indexing and searching. My requirement is to make ES available
as a bundle and make it run on OSGi runtime along with my other
bundles. Any idea how I could go about achieving this?
Thanks!
Shruthi

3

I'm trying to do the same using bnd to wrap lucene and the elasticsearch jars as osgi bundles.
There's one issue: elasticsearch has some org.apache.lucene packages that's overlap with the same packages exported by lucene itself. This is bad in osgi.

A quick solution is just not to export, from ES, those packages, that will become not available to other bundle. Maybe they're not intented to be exported so that is good.

It they're intented to be used outside ES, so the only solution is to change the name of the package in a way that it bocome unique.

4

You don't need to expose those classes from the osgi bundle. They are only there because some methods in Lucene are package protected.

On Saturday, May 28, 2011 at 12:58 PM, ildella wrote:

I'm trying to do the same using bnd to wrap lucene and the elasticsearch jars
as osgi bundles.
There's one issue: elasticsearch has some org.apache.lucene packages that's
overlap with the same packages exported by lucene itself. This is bad in
osgi.

A quick solution is just not to export, from ES, those packages, that will
become not available to other bundle. Maybe they're not intented to be
exported so that is good.

It they're intented to be used outside ES, so the only solution is to change
the name of the package in a way that it bocome unique.

--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Running-ES-as-a-OSGi-bundle-tp2918132p2995769.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com (http://Nabble.com).

5

You may merge jars from lucene and elasticsearch and create one big
bundle elasticsearch_all. I did it this way and works.

On May 28, 11:58 am, ildella ilde...@gmail.com wrote:

I'm trying to do the same using bnd to wrap lucene and the elasticsearch jars
as osgi bundles.
There's one issue: elasticsearch has some org.apache.lucene packages that's
overlap with the same packages exported by lucene itself. This is bad in
osgi.

A quick solution is just not to export, from ES, those packages, that will
become not available to other bundle. Maybe they're not intented to be
exported so that is good.

It they're intented to be used outside ES, so the only solution is to change
the name of the package in a way that it bocome unique.

--
View this message in context:http://elasticsearch-users.115913.n3.nabble.com/Running-ES-as-a-OSGi-...
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

6

I was planning to also use ES OSGI bundle for our application. Is there a standard bundle available ?

7

Any updates on this, I see similar questions making elasticsearch OSGI compliant remain open.

Currently I have following problems & resorted to some hacks to avoid the same temporarily. Any inputs on correct usage is greatly appreciated.

  1. While instantiating the TransportClient it relies on "Names.txt" & is present as part of the jars. It fails to load from classpath in OSGI envioronement.
    Work around:
    Currently copying the names.txt in the bundle and specifying the path explicitely.
    settingsBuilder.put("path.conf", dirPathHavingNamingDotTextFile);

  2. As elasticsearch also bundles few lucene classes "org.apache.index", OSGI complains saying it clashes with the classes present in the lucene.
    Work around:
    Remove the classes present in one of the jars and rebuild and deploy in the OSGI container.

Since elasticsearch configures the compression scheme in TransportClient CompressorFactory.configure(), there is no way to avoid the execution of static bloc of CompressionFactory class.
Suggest to make it optional through one more TransportClient constructor

Thanks,
Praveen

8

Hey,

I dont really know anything about OSGI, but you might be able to prevent
the first issue by setting the 'node.name' property (which you should do
anyway in production), as this prevents the file from being searched...

--Alex

On Mon, Jul 22, 2013 at 2:11 PM, pmanvi praveena.manvi@gmail.com wrote:

Any updates on this, I see similar questions making elasticsearch OSGI
compliant remain open.

Currently I have following problems & resorted to some hacks to avoid the
same temporarily. Any inputs on correct usage is greatly appreciated.

  1. While instantiating the TransportClient it relies on "Names.txt" & is
    present as part of the jars. It fails to load from classpath in OSGI
    envioronement.
    Work around:
    Currently copying the names.txt in the bundle and specifying the path
    explicitely.
    settingsBuilder.put("path.conf", dirPathHavingNamingDotTextFile);

  2. As elasticsearch also bundles few lucene classes "org.apache.index",
    OSGI
    complains saying it clashes with the classes present in the lucene.
    Work around:
    Remove the classes present in one of the jars and rebuild and deploy in the
    OSGI container.

Since elasticsearch configures the compression scheme in TransportClient
CompressorFactory.configure(), there is no way to avoid the execution of
static bloc of CompressionFactory class.
Suggest to make it optional through one more TransportClient constructor

Thanks,
Praveen

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Running-ES-as-a-OSGi-bundle-tp2918132p4038445.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

9

I assume you're talking just about the Java client in OSGi (I don't see how/why you would use ES with OSGi) and I'm
curious what's your use case?
Also any information about the OSGi framework that you're using?

On 22/07/2013 3:11 PM, pmanvi wrote:

Any updates on this, I see similar questions making elasticsearch OSGI
compliant remain open.

Currently I have following problems & resorted to some hacks to avoid the
same temporarily. Any inputs on correct usage is greatly appreciated.

  1. While instantiating the TransportClient it relies on "Names.txt" & is
    present as part of the jars. It fails to load from classpath in OSGI
    envioronement.
    Work around:
    Currently copying the names.txt in the bundle and specifying the path
    explicitely.
    settingsBuilder.put("path.conf", dirPathHavingNamingDotTextFile);

What's the exception that you're getting?

Accessing resources in OSGi can be tricky since the classpath is not always obvious. ES falls back to loading Names.txt
from the Environment class classloader.
My guess would be that since config/ is not an exported package, the bundle class loader doesn't allow access to it -
try adding config/ to your Export-Package directive and see whether that helps.

  1. As elasticsearch also bundles few lucene classes "org.apache.index", OSGI
    complains saying it clashes with the classes present in the lucene.
    Work around:
    Remove the classes present in one of the jars and rebuild and deploy in the
    OSGI container.

Is the start-up halted or is it just a warning that you're receiving? Note that it's perfectly fine for a bundle to have
'internal' packages - simply remove "org.apache.index" from your Export-Package (inside the Transport client jar) - thus
making the package private - and you should be fine.

Since elasticsearch configures the compression scheme in TransportClient
CompressorFactory.configure(), there is no way to avoid the execution of
static bloc of CompressionFactory class.
Suggest to make it optional through one more TransportClient constructor

Why is this a problem?

Thanks,
Praveen

--
View this message in context: http://elasticsearch-users.115913.n3.nabble.com/Running-ES-as-a-OSGi-bundle-tp2918132p4038445.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

10

Thanks Alex,
It works.

I did see a thread config/elasticsearch.yml: add a commented out node.name with docs by avar · Pull Request #1345 · elastic/elasticsearch · GitHub where
it was added in the elasticsearch.yml

I could not find any documentation on what''s the value that should be
provided and why elasticsearch needs to generate so many names?

On Tuesday, July 23, 2013 11:57:18 AM UTC+5:30, Alexander Reelsen wrote:

Hey,

I dont really know anything about OSGI, but you might be able to prevent
the first issue by setting the 'node.name' property (which you should do
anyway in production), as this prevents the file from being searched...

--Alex

On Mon, Jul 22, 2013 at 2:11 PM, pmanvi <praveen...@gmail.com<javascript:>

wrote:

Any updates on this, I see similar questions making elasticsearch OSGI
compliant remain open.

Currently I have following problems & resorted to some hacks to avoid the
same temporarily. Any inputs on correct usage is greatly appreciated.

  1. While instantiating the TransportClient it relies on "Names.txt" & is
    present as part of the jars. It fails to load from classpath in OSGI
    envioronement.
    Work around:
    Currently copying the names.txt in the bundle and specifying the path
    explicitely.
    settingsBuilder.put("path.conf", dirPathHavingNamingDotTextFile);

  2. As elasticsearch also bundles few lucene classes "org.apache.index",
    OSGI
    complains saying it clashes with the classes present in the lucene.
    Work around:
    Remove the classes present in one of the jars and rebuild and deploy in
    the
    OSGI container.

Since elasticsearch configures the compression scheme in TransportClient
CompressorFactory.configure(), there is no way to avoid the execution of
static bloc of CompressionFactory class.
Suggest to make it optional through one more TransportClient constructor

Thanks,
Praveen

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Running-ES-as-a-OSGi-bundle-tp2918132p4038445.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

11

On Tuesday, July 23, 2013 3:15:01 PM UTC+5:30, Costin Leau wrote:

I assume you're talking just about the Java client in OSGi (I don't see
how/why you would use ES with OSGi) and I'm
curious what's your use case?
Also any information about the OSGi framework that you're using?

On 22/07/2013 3:11 PM, pmanvi wrote:

Any updates on this, I see similar questions making elasticsearch OSGI
compliant remain open.

Currently I have following problems & resorted to some hacks to avoid
the
same temporarily. Any inputs on correct usage is greatly appreciated.

  1. While instantiating the TransportClient it relies on "Names.txt" & is
    present as part of the jars. It fails to load from classpath in OSGI
    envioronement.
    Work around:
    Currently copying the names.txt in the bundle and specifying the path
    explicitely.
    settingsBuilder.put("path.conf", dirPathHavingNamingDotTextFile);

What's the exception that you're getting?

Accessing resources in OSGi can be tricky since the classpath is not
always obvious. ES falls back to loading Names.txt
from the Environment class classloader.
My guess would be that since config/ is not an exported package, the
bundle class loader doesn't allow access to it -
try adding config/ to your Export-Package directive and see whether that
helps.

Thanks. It was generic TransportSerializationException. I am not finding
the exception messages not that really useful, It always says stream
corrupted (for invalid schema, fields etc. forcing me to include try {}
catch bloc on each of the statement calls to get the context while throwing
the exception back.

As suggested in the other thread the problem got solved by un-commenting
the line "node.name" in the elasticsearch.yml. But I am not sure what I am
losing by doing this.

  1. As elasticsearch also bundles few lucene classes "org.apache.index",
    OSGI
    complains saying it clashes with the classes present in the lucene.
    Work around:
    Remove the classes present in one of the jars and rebuild and deploy in
    the
    OSGI container.

Is the start-up halted or is it just a warning that you're receiving? Note
that it's perfectly fine for a bundle to have
'internal' packages - simply remove "org.apache.index" from your
Export-Package (inside the Transport client jar) - thus
making the package private - and you should be fine.

It was getting halted. Since we were running in fuse, we don;t have copied
the elastic jar as is without adding OSGI symantics. Will try out defining
OSGI entries for POM and try commenting out the package as suggested.
Thanks.

Since elasticsearch configures the compression scheme in TransportClient
CompressorFactory.configure(), there is no way to avoid the execution of
static bloc of CompressionFactory class.
Suggest to make it optional through one more TransportClient constructor

Why is this a problem?

Since we are using this client library (from fuse container), It
un-necessarly initializes the compressor (JNI call?), it can be done lazily
once the compressor is being called.There is no issue as such.

Thanks,
Praveen

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Running-ES-as-a-OSGi-bundle-tp2918132p4038445.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

12

On Tuesday, July 23, 2013 3:15:01 PM UTC+5:30, Costin Leau wrote:

I assume you're talking just about the Java client in OSGi (I don't see
how/why you would use ES with OSGi) and I'm
curious what's your use case?
Also any information about the OSGi framework that you're using?

Yes elasticsearch is used its client library. Not sure if there is striped
down version exist, something like elasticsearch-client.jar
we are using jboss fuse.

On 22/07/2013 3:11 PM, pmanvi wrote:

Any updates on this, I see similar questions making elasticsearch OSGI
compliant remain open.

Currently I have following problems & resorted to some hacks to avoid
the
same temporarily. Any inputs on correct usage is greatly appreciated.

  1. While instantiating the TransportClient it relies on "Names.txt" & is
    present as part of the jars. It fails to load from classpath in OSGI
    envioronement.
    Work around:
    Currently copying the names.txt in the bundle and specifying the path
    explicitely.
    settingsBuilder.put("path.conf", dirPathHavingNamingDotTextFile);

What's the exception that you're getting?

Accessing resources in OSGi can be tricky since the classpath is not
always obvious. ES falls back to loading Names.txt
from the Environment class classloader.
My guess would be that since config/ is not an exported package, the
bundle class loader doesn't allow access to it -
try adding config/ to your Export-Package directive and see whether that
helps.

  1. As elasticsearch also bundles few lucene classes "org.apache.index",
    OSGI
    complains saying it clashes with the classes present in the lucene.
    Work around:
    Remove the classes present in one of the jars and rebuild and deploy in
    the
    OSGI container.

Is the start-up halted or is it just a warning that you're receiving? Note
that it's perfectly fine for a bundle to have
'internal' packages - simply remove "org.apache.index" from your
Export-Package (inside the Transport client jar) - thus
making the package private - and you should be fine.

Since elasticsearch configures the compression scheme in TransportClient
CompressorFactory.configure(), there is no way to avoid the execution of
static bloc of CompressionFactory class.
Suggest to make it optional through one more TransportClient constructor

Why is this a problem?

Thanks,
Praveen

--
View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/Running-ES-as-a-OSGi-bundle-tp2918132p4038445.html
Sent from the Elasticsearch Users mailing list archive at Nabble.com.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.