Maven shaded/renamed dependencies in es

Elastic Stack Elasticsearch
1

Hello all,

I'm developing some custom facets based on the default ones present in the
elasticsearch.

My problems are the way I manage dependencies for my custom plugin/facet
projects.

Here's my question: what would be the best practice for referencing joda
and guava libraries in the project?

My eclipse workspace setup is:
-elasticsearch sourcecode project
-code for my custom plugins and facets. (gradle build)

As a binary elasticsearch shades/renames the joda and guava libraries to
org.elasticsearch.common.
That means, when using elasticsearch binary jar as a reference for my
custom project, I'm referencing the shaded joda and guava libraries.

BUT! This way I can't debug.

Therefore, at the moment, I'm using 2 sets of imports (literary: 2 sets of
imports in the classes using guava/joda) depending on the task I'm doing:

  • for debugging, I reference the standard guava and joda, this way my
    plugins get deployed and I have access to the internals of es
  • for deploying I reference elasticsearch-0.19.4 binary

It sucks, I know, and I am curious if there is a clean way to do this,
other then maven, preferably with gradle, or any other method.

Tnx,

Alex

2

Regarding having my custom project as a maven project. It still doesnt
solve the issue.
That is because, even if I manage to shade dependencies, I still need to
have hard references to elasticsearch source code in my workspace.
And I still dont have a clear way of dealing with the shaded dependencies.
Damn!:slight_smile:

Or I might be wrong??

On Tue, Jun 12, 2012 at 9:45 AM, Sisu Alexandru sisu.eugen@gmail.comwrote:

Hello all,

I'm developing some custom facets based on the default ones present in the
elasticsearch.

My problems are the way I manage dependencies for my custom plugin/facet
projects.

Here's my question: what would be the best practice for referencing joda
and guava libraries in the project?

My eclipse workspace setup is:
-elasticsearch sourcecode project
-code for my custom plugins and facets. (gradle build)

As a binary elasticsearch shades/renames the joda and guava libraries to
org.elasticsearch.common.
That means, when using elasticsearch binary jar as a reference for my
custom project, I'm referencing the shaded joda and guava libraries.

BUT! This way I can't debug.

Therefore, at the moment, I'm using 2 sets of imports (literary: 2 sets of
imports in the classes using guava/joda) depending on the task I'm doing:

  • for debugging, I reference the standard guava and joda, this way my
    plugins get deployed and I have access to the internals of es
  • for deploying I reference elasticsearch-0.19.4 binary

It sucks, I know, and I am curious if there is a clean way to do this,
other then maven, preferably with gradle, or any other method.

Tnx,

Alex

3

The following compromise had been reached:

  • remove the shading step from elasticsearch building;
  • install the generated elasticsearch jar in your local repository
  • build the project with references to the elasticsearch jar in your local
    repository;
  • shade the resulting project jar;

But its wicked.... Still waiting for suggestions :slight_smile: :-?

On Tue, Jun 12, 2012 at 2:04 PM, Sisu Alexandru sisu.eugen@gmail.comwrote:

Regarding having my custom project as a maven project. It still doesnt
solve the issue.
That is because, even if I manage to shade dependencies, I still need to
have hard references to elasticsearch source code in my workspace.
And I still dont have a clear way of dealing with the shaded dependencies.
Damn!:slight_smile:

Or I might be wrong??

On Tue, Jun 12, 2012 at 9:45 AM, Sisu Alexandru sisu.eugen@gmail.comwrote:

Hello all,

I'm developing some custom facets based on the default ones present in
the elasticsearch.

My problems are the way I manage dependencies for my custom plugin/facet
projects.

Here's my question: what would be the best practice for referencing joda
and guava libraries in the project?

My eclipse workspace setup is:
-elasticsearch sourcecode project
-code for my custom plugins and facets. (gradle build)

As a binary elasticsearch shades/renames the joda and guava libraries to
org.elasticsearch.common.
That means, when using elasticsearch binary jar as a reference for my
custom project, I'm referencing the shaded joda and guava libraries.

BUT! This way I can't debug.

Therefore, at the moment, I'm using 2 sets of imports (literary: 2 sets
of imports in the classes using guava/joda) depending on the task I'm doing:

  • for debugging, I reference the standard guava and joda, this way my
    plugins get deployed and I have access to the internals of es
  • for deploying I reference elasticsearch-0.19.4 binary

It sucks, I know, and I am curious if there is a clean way to do this,
other then maven, preferably with gradle, or any other method.

Tnx,

Alex

4

You could perhaps grab the source code of joda and guava, import it in your IDE
(Eclise for example), and refactor the package name to start with
org.elasticsearch....

Then, add these projects as source when debugging...

It may help.

David.

Le 13 juin 2012 à 13:24, Sisu Alexandru sisu.eugen@gmail.com a écrit :

The following compromise had been reached:

  • remove the shading step from elasticsearch building;
  • install the generated elasticsearch jar in your local repository
  • build the project with references to the elasticsearch jar in your local
    repository;
  • shade the resulting project jar;

But its wicked.... Still waiting for suggestions :slight_smile: :-?

On Tue, Jun 12, 2012 at 2:04 PM, Sisu Alexandru <sisu.eugen@gmail.com
mailto:sisu.eugen@gmail.com > wrote:

Regarding having my custom project as a maven project. It still doesnt
solve the issue.
That is because, even if I manage to shade dependencies, I still need to
have hard references to elasticsearch source code in my workspace.
And I still dont have a clear way of dealing with the shaded
dependencies. Damn!:slight_smile:

Or I might be wrong??

On Tue, Jun 12, 2012 at 9:45 AM, Sisu Alexandru <sisu.eugen@gmail.com
mailto:sisu.eugen@gmail.com > wrote:
> > > Hello all,

 I'm developing some custom facets based on the default ones present

in the elasticsearch.

 My problems are the way I manage dependencies for my custom

plugin/facet projects.

 Here's my question: what would be the best practice for referencing

joda and guava libraries in the project?

 My eclipse workspace setup is:
 -elasticsearch sourcecode project
 -code for my custom plugins and facets. (gradle build)

 As a binary elasticsearch shades/renames the joda and guava libraries

to org.elasticsearch.common.
That means, when using elasticsearch binary jar as a reference for my
custom project, I'm referencing the shaded joda and guava libraries.

 BUT! This way I can't debug.

 Therefore, at the moment, I'm using 2 sets of imports (literary: 2

sets of imports in the classes using guava/joda) depending on the task I'm
doing:
- for debugging, I reference the standard guava and joda, this way my
plugins get deployed and I have access to the internals of es
- for deploying I reference elasticsearch-0.19.4 binary

 It sucks, I know, and I am curious if there is a clean way to do

this, other then maven, preferably with gradle, or any other method.

 Tnx,

 Alex

--
David Pilato
http://dev.david.pilato.fr/
Twitter : @dadoonet

5

Tnx for the suggestion.

Meanwhile what I did, was to create an empty eclipse project. Add the
binaries of the es, and its sources. And start es directly from the jar.
:). (in eclipse >:) ).

And it works. No more shaded dependencies hell.
I can deploy and I can debug.

Cheers,

Alex

On Wed, Jun 13, 2012 at 1:44 PM, David Pilato david@pilato.fr wrote:

**

You could perhaps grab the source code of joda and guava, import it in
your IDE (Eclise for example), and refactor the package name to start with
org.elasticsearch....

Then, add these projects as source when debugging...

It may help.

David.

Le 13 juin 2012 à 13:24, Sisu Alexandru sisu.eugen@gmail.com a écrit :

The following compromise had been reached:

  • remove the shading step from elasticsearch building;
  • install the generated elasticsearch jar in your local repository
  • build the project with references to the elasticsearch jar in your
    local repository;
  • shade the resulting project jar;

But its wicked.... Still waiting for suggestions :slight_smile: :-?

On Tue, Jun 12, 2012 at 2:04 PM, Sisu Alexandru sisu.eugen@gmail.comwrote:

Regarding having my custom project as a maven project. It still doesnt
solve the issue.
That is because, even if I manage to shade dependencies, I still need to
have hard references to elasticsearch source code in my workspace.
And I still dont have a clear way of dealing with the shaded
dependencies. Damn!:slight_smile:

Or I might be wrong??

On Tue, Jun 12, 2012 at 9:45 AM, Sisu Alexandru sisu.eugen@gmail.comwrote:

Hello all,

I'm developing some custom facets based on the default ones present in
the elasticsearch.

My problems are the way I manage dependencies for my custom plugin/facet
projects.

Here's my question: what would be the best practice for referencing joda
and guava libraries in the project?

My eclipse workspace setup is:
-elasticsearch sourcecode project
-code for my custom plugins and facets. (gradle build)

As a binary elasticsearch shades/renames the joda and guava libraries to
org.elasticsearch.common.
That means, when using elasticsearch binary jar as a reference for my
custom project, I'm referencing the shaded joda and guava libraries.

BUT! This way I can't debug.

Therefore, at the moment, I'm using 2 sets of imports (literary: 2 sets
of imports in the classes using guava/joda) depending on the task I'm
doing:

  • for debugging, I reference the standard guava and joda, this way my
    plugins get deployed and I have access to the internals of es
  • for deploying I reference elasticsearch-0.19.4 binary

It sucks, I know, and I am curious if there is a clean way to do this,
other then maven, preferably with gradle, or any other method.

Tnx,

Alex

--
David Pilato
http://dev.david.pilato.fr/
Twitter : @dadoonet