IntelliJ settings for Elasticsearch (JAR shading issue)

Hi there,

This a question a little bit outside ES scope, but not so far...

I'm trying to play with IntelliJ (instead of Eclipse) for Elasticsearch.
Everything is fine when I compile/test 0.20 branch in IntelliJ.

My problem comes when I try to have 2 maven projects in the same workspace (I
should say IntelliJ project):

When I launch tests from spore project, I got the following errors when starting
ES Nodes (in Java):

java.lang.NoClassDefFoundError: com/fasterxml/jackson/core/JsonParser
at
org.elasticsearch.common.xcontent.XContentType$3.xContent(XContentType.java:85)
at
org.elasticsearch.common.xcontent.XContentFactory.xContent(XContentFactory.java:110)
...

I think it's only a configuration problem as Jackson is shaded in elasticsearch
jar and does not have this class in the original package. How do you setup
things in IntelliJ to add breakpoints in Elasticsearch when launching tests from
another project?

Sorry to be a little bit outside the ES scope.

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--

Can't you do that by adding ES via Maven to your project, and attach
sources? IIRC this worked for me before

On Fri, Dec 28, 2012 at 1:18 PM, David Pilato david@pilato.fr wrote:

**
Hi there,

This a question a little bit outside ES scope, but not so far...

I'm trying to play with IntelliJ (instead of Eclipse) for Elasticsearch.
Everything is fine when I compile/test 0.20 branch in IntelliJ.

My problem comes when I try to have 2 maven projects in the same
workspace (I should say IntelliJ project):

When I launch tests from spore project, I got the following errors when
starting ES Nodes (in Java):

java.lang.NoClassDefFoundError: com/fasterxml/jackson/core/JsonParser
at
org.elasticsearch.common.xcontent.XContentType$3.xContent(XContentType.java:85)
at
org.elasticsearch.common.xcontent.XContentFactory.xContent(XContentFactory.java:110)
...

I think it's only a configuration problem as Jackson is shaded in
elasticsearch jar and does not have this class in the original package. How
do you setup things in IntelliJ to add breakpoints in Elasticsearch when
launching tests from another project?

Sorry to be a little bit outside the ES scope.

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--

--

Thanks Itamar, It works fine without importing ES project.

The fact is I can not modify ES source code directly this way. But, it's
probably a bad practice to launch tests from another project than from ES
project itself!

Thanks again.

Le 28 décembre 2012 à 12:29, Itamar Syn-Hershko itamar@code972.com a écrit :

Can't you do that by adding ES via Maven to your project, and attach sources?
IIRC this worked for me before

On Fri, Dec 28, 2012 at 1:18 PM, David Pilato <david@pilato.fr
mailto:david@pilato.fr > wrote:

Hi there,

This a question a little bit outside ES scope, but not so far...

I'm trying to play with IntelliJ (instead of Eclipse) for Elasticsearch.
Everything is fine when I compile/test 0.20 branch in IntelliJ.

My problem comes when I try to have 2 maven projects in the same
workspace (I should say IntelliJ project):

When I launch tests from spore project, I got the following errors when
starting ES Nodes (in Java):

java.lang.NoClassDefFoundError: com/fasterxml/jackson/core/JsonParser
at
org.elasticsearch.common.xcontent.XContentType$3.xContent(XContentType.java:85)
at
org.elasticsearch.common.xcontent.XContentFactory.xContent(XContentFactory.java:110)
...

I think it's only a configuration problem as Jackson is shaded in
elasticsearch jar and does not have this class in the original package. How
do you setup things in IntelliJ to add breakpoints in Elasticsearch when
launching tests from another project?

Sorry to be a little bit outside the ES scope.

--
David Pilato
http://www.scrutmydocs.org/ http://www.scrutmydocs.org/
http://dev.david.pilato.fr/ http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--

--

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--

Yeah, I know, tried that too with a few hacks but it kept confusing
IntelliJ.

My workflow is to have ES open in another IntelliJ instance and whenever I
need to change ES I change it in that other instance, do mvn install and
have the other IntelliJ instance import it again.

Visual Studio allows you to do that, I just couldn't figure out how this
can be done with IntelliJ (yet?).

On Fri, Dec 28, 2012 at 3:32 PM, David Pilato david@pilato.fr wrote:

**
Thanks Itamar, It works fine without importing ES project.

The fact is I can not modify ES source code directly this way. But, it's
probably a bad practice to launch tests from another project than from ES
project itself!

Thanks again.

Le 28 décembre 2012 à 12:29, Itamar Syn-Hershko itamar@code972.com a
écrit :

Can't you do that by adding ES via Maven to your project, and attach
sources? IIRC this worked for me before

On Fri, Dec 28, 2012 at 1:18 PM, David Pilato david@pilato.fr wrote:

Hi there,

This a question a little bit outside ES scope, but not so far...

I'm trying to play with IntelliJ (instead of Eclipse) for Elasticsearch.
Everything is fine when I compile/test 0.20 branch in IntelliJ.

My problem comes when I try to have 2 maven projects in the same
workspace (I should say IntelliJ project):

When I launch tests from spore project, I got the following errors when
starting ES Nodes (in Java):

java.lang.NoClassDefFoundError: com/fasterxml/jackson/core/JsonParser
at
org.elasticsearch.common.xcontent.XContentType$3.xContent(XContentType.java:85)
at
org.elasticsearch.common.xcontent.XContentFactory.xContent(XContentFactory.java:110)
...

I think it's only a configuration problem as Jackson is shaded in
elasticsearch jar and does not have this class in the original package. How
do you setup things in IntelliJ to add breakpoints in Elasticsearch when
launching tests from another project?

Sorry to be a little bit outside the ES scope.

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--

--

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--

--

I was wondering if it's a goog idea to shade libraries when building the
artifact instead of doing this in the generate-resources phase?
What can be done (I will check that), is to shade all libraries in the
generate-resources phase into the target dir. So it will be available when
compiling and that way, on the ES project, imports will be done on the target
package not on the original one.

I don't know yet if it can be done, either if it's a best practice or not. I
will try it.

Le 28 décembre 2012 à 14:50, Itamar Syn-Hershko itamar@code972.com a écrit :

Yeah, I know, tried that too with a few hacks but it kept confusing IntelliJ.

My workflow is to have ES open in another IntelliJ instance and whenever I
need to change ES I change it in that other instance, do mvn install and have
the other IntelliJ instance import it again.

Visual Studio allows you to do that, I just couldn't figure out how this can
be done with IntelliJ (yet?).

On Fri, Dec 28, 2012 at 3:32 PM, David Pilato <david@pilato.fr
mailto:david@pilato.fr > wrote:

Thanks Itamar, It works fine without importing ES project.

The fact is I can not modify ES source code directly this way. But, it's
probably a bad practice to launch tests from another project than from ES
project itself!

Thanks again.

Le 28 décembre 2012 tel:2012 à 12:29, Itamar Syn-Hershko <
itamar@code972.com mailto:itamar@code972.com > a écrit :

> > >     Can't you do that by adding ES via Maven to your project, and
> > > attach sources? IIRC this worked for me before
On Fri, Dec 28, 2012 <tel:2012> at 1:18 PM, David Pilato

<david@pilato.fr mailto:david@pilato.fr > wrote:
> > > > Hi there,

  This a question a little bit outside ES scope, but not so far...

  I'm trying to play with IntelliJ (instead of Eclipse) for

Elasticsearch.
Everything is fine when I compile/test 0.20 branch in IntelliJ.

  My problem comes when I try to have 2 maven projects in the same

workspace (I should say IntelliJ project):
- Elasticsearch (branch 0.20)
- Spore Elasticsearch
https://github.com/dadoonet/spore-elasticsearch project

  When I launch tests from spore project, I got the following errors

when starting ES Nodes (in Java):

  java.lang.NoClassDefFoundError:

com/fasterxml/jackson/core/JsonParser
at
org.elasticsearch.common.xcontent.XContentType$3.xContent(XContentType.java:85)
at
org.elasticsearch.common.xcontent.XContentFactory.xContent(XContentFactory.java:110)
...

  I think it's only a configuration problem as Jackson is shaded in

elasticsearch jar and does not have this class in the original package.
How do you setup things in IntelliJ to add breakpoints in Elasticsearch
when launching tests from another project?

  Sorry to be a little bit outside the ES scope.



  --
  David Pilato
  http://www.scrutmydocs.org/ <http://www.scrutmydocs.org/>
  http://dev.david.pilato.fr/ <http://dev.david.pilato.fr/>
  Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs



  --


> > > 
--

--
David Pilato
http://www.scrutmydocs.org/ http://www.scrutmydocs.org/
http://dev.david.pilato.fr/ http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--

--

--
David Pilato
http://www.scrutmydocs.org/
http://dev.david.pilato.fr/
Twitter : @dadoonet / @elasticsearchfr / @scrutmydocs

--

Hey David,

strange, I haven't seen a "java.lang.NoClassDefFoundError" yet.

I guess the IDE test classloader is fooled by the Maven-based shading for
the class jar artifact, this conflicts with the Jackson sources in the IDE
classpath. As long as it does not rely on the shaded class jar artifact,
there is no chance to trace the real class name of shaded libs.

During my work on a modularized ES codebase, I had to decide where to shade
dependencies. I added the shading on a global level just as an add-on jar,
to deploy a special ES node artifact. If it were on submodule levels, IDEs
got confused.

Because the shaded ES artifact is minimized, IDEs may not get alarmed when
unused classes were silently dropped from the jar artifact in the Maven
repo. It's a headache to reuse Guava, Netty, Jackson etc. when developing
apps that build upon ES.

I am also having a hard time to understand Eclipse and IntelliJ with their
non-Maven strategies. Netbeans has the fewest flaws in ES testing IMHO, it
just uses Maven surefire for testing.

Just my 2p.

Jörg

--