ElasticSearch multi-threading and the Java EE specifications


(Tasha) #1

Hi all,

As you know, ElasticSearch relies a lot on multi-threading.

I'm currently using ES in a Java EE 6 application (using the node client as
pure client node) and the Java EE specifications state that classic thread
creation (à la "new Thread() { @Override public void run() {}}") is
actually forbidden.

I know that it is working, at least on the application servers I know, but
I'm wondering what the ES teams says to this dilemma.

After all, doing something "illegal" against the specs might become
blocked/impossible in the future (not likely in this case) and it might
also cause strange side-effects on application servers.

Your opinions?

Best,

Tasha

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/aa786741-30fa-4ee8-a3ad-7e3b007b74de%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Nik Everett) #2

I've heard of this rule but never seen anyone follow it. They'd mostly
make threads/thread pools and just make sure they were shut down when the
application was shut down. So my advice is to follow the intent of the law
(shutdown your threads) rather than follow the letter of the law.

Nik

On Mon, Sep 8, 2014 at 3:54 PM, Tasha tasha.cts@gmail.com wrote:

Hi all,

As you know, ElasticSearch relies a lot on multi-threading.

I'm currently using ES in a Java EE 6 application (using the node client
as pure client node) and the Java EE specifications state that classic
thread creation (à la "new Thread() { @Override public void run() {}}") is
actually forbidden.

I know that it is working, at least on the application servers I know, but
I'm wondering what the ES teams says to this dilemma.

After all, doing something "illegal" against the specs might become
blocked/impossible in the future (not likely in this case) and it might
also cause strange side-effects on application servers.

Your opinions?

Best,

Tasha

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/aa786741-30fa-4ee8-a3ad-7e3b007b74de%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/aa786741-30fa-4ee8-a3ad-7e3b007b74de%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAPmjWd0spoH2rskq5dBP4h8qE_c1xjyiY-s2p7aC%2Begrhs2zNg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #3

Your question is not very clear, but maybe you refer to JSR 236 for Java
EE.

Elasticsearch was never designed to run as a container managed Java EE
component in a Java EE container, so the question about thread creation is
unrelated.

If you want to use an ES client from within a Java EE container, start a
node client or transport client as a singleton, and stop it when the
service stops.

Can you be more specific about "strange side effects on application
servers"?

Jörg

On Mon, Sep 8, 2014 at 9:54 PM, Tasha tasha.cts@gmail.com wrote:

Hi all,

As you know, ElasticSearch relies a lot on multi-threading.

I'm currently using ES in a Java EE 6 application (using the node client
as pure client node) and the Java EE specifications state that classic
thread creation (à la "new Thread() { @Override public void run() {}}") is
actually forbidden.

I know that it is working, at least on the application servers I know, but
I'm wondering what the ES teams says to this dilemma.

After all, doing something "illegal" against the specs might become
blocked/impossible in the future (not likely in this case) and it might
also cause strange side-effects on application servers.

Your opinions?

Best,

Tasha

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/aa786741-30fa-4ee8-a3ad-7e3b007b74de%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/aa786741-30fa-4ee8-a3ad-7e3b007b74de%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFwxuEEZ0h_WPr5S4JSFKbQjzNEW1osdi6edMVC32_nKg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Tasha) #4

Elasticsearch was never designed to run as a container managed Java EE

component in a Java EE container, so the question about thread creation

is unrelated.

That's exactly my point and that's why I ask this question.

If you want to use an ES client from within a Java EE container,

start a node client or transport client as a singleton,

and stop it when the service stops.

That's the obvious way to handle this and that's exactly what I do. Still,
you include you ES JAR in your application and this JAR creates threads out
of the control of the Java EE container. Practically, they are part of the
application and the ES client runs inside the Java EE container.

Can you be more specific about "strange side effects on application
servers"?

I cannot be more specific since I did not yet encounter such effects. I
imagine that for example under heavy load, the ES client (JAR) could create
too many threads of which the container is not aware of that together with
the threads created by the container itself, the OS/VM limits/quota are
hit.

Tasha

On 9 September 2014 09:39, joergprante@gmail.com joergprante@gmail.com
wrote:

Your question is not very clear, but maybe you refer to JSR 236 for Java
EE.

Elasticsearch was never designed to run as a container managed Java EE
component in a Java EE container, so the question about thread creation is
unrelated.

If you want to use an ES client from within a Java EE container, start a
node client or transport client as a singleton, and stop it when the
service stops.

Can you be more specific about "strange side effects on application
servers"?

Jörg

On Mon, Sep 8, 2014 at 9:54 PM, Tasha tasha.cts@gmail.com wrote:

Hi all,

As you know, ElasticSearch relies a lot on multi-threading.

I'm currently using ES in a Java EE 6 application (using the node client
as pure client node) and the Java EE specifications state that classic
thread creation (à la "new Thread() { @Override public void run() {}}") is
actually forbidden.

I know that it is working, at least on the application servers I know,
but I'm wondering what the ES teams says to this dilemma.

After all, doing something "illegal" against the specs might become
blocked/impossible in the future (not likely in this case) and it might
also cause strange side-effects on application servers.

Your opinions?

Best,

Tasha

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/aa786741-30fa-4ee8-a3ad-7e3b007b74de%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/aa786741-30fa-4ee8-a3ad-7e3b007b74de%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/VqJMG87S9jk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFwxuEEZ0h_WPr5S4JSFKbQjzNEW1osdi6edMVC32_nKg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFwxuEEZ0h_WPr5S4JSFKbQjzNEW1osdi6edMVC32_nKg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CABeBvCeXV-Fg-%3DjHS434_oKi7i8Dr839dAsB0PA6EmobrMeqsg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #5

I think you confuse two things:

  • the Java platform (Java Language, Java Virtual Machine)

  • the Java EE specification

These two things are different.

As the Java EE specification is community-driven and defined, it does not
mean to put any burden or restrictions on the Java platform, or on any
applications running on the JVM.

Note that ES nodes solve problems for distributed applications that Java EE
is not even aware of.

It is possible to give up certain ES features and write another ES node to
use the Java EE specification regarding the managed components and thread
pools very exactly but there is not much sense in it. Why should you expect
a Java EE container like Tomcat/Wildfly to operate like an ES cluster? They
simply can not do so and they will never do like this. You will challenge
many fields: resource allocation, reduced performance, less throughput,
less fault tolerance, distributed JVM orchestration etc.

Jörg

On Tue, Sep 9, 2014 at 9:57 AM, Tasha CARL tasha.cts@gmail.com wrote:

Elasticsearch was never designed to run as a container managed Java EE

component in a Java EE container, so the question about thread creation

is unrelated.

That's exactly my point and that's why I ask this question.

If you want to use an ES client from within a Java EE container,

start a node client or transport client as a singleton,

and stop it when the service stops.

That's the obvious way to handle this and that's exactly what I do. Still,
you include you ES JAR in your application and this JAR creates threads out
of the control of the Java EE container. Practically, they are part of the
application and the ES client runs inside the Java EE container.

Can you be more specific about "strange side effects on application
servers"?

I cannot be more specific since I did not yet encounter such effects. I
imagine that for example under heavy load, the ES client (JAR) could create
too many threads of which the container is not aware of that together with
the threads created by the container itself, the OS/VM limits/quota are
hit.

Tasha

On 9 September 2014 09:39, joergprante@gmail.com joergprante@gmail.com
wrote:

Your question is not very clear, but maybe you refer to JSR 236 for Java
EE.

Elasticsearch was never designed to run as a container managed Java EE
component in a Java EE container, so the question about thread creation is
unrelated.

If you want to use an ES client from within a Java EE container, start a
node client or transport client as a singleton, and stop it when the
service stops.

Can you be more specific about "strange side effects on application
servers"?

Jörg

On Mon, Sep 8, 2014 at 9:54 PM, Tasha tasha.cts@gmail.com wrote:

Hi all,

As you know, ElasticSearch relies a lot on multi-threading.

I'm currently using ES in a Java EE 6 application (using the node client
as pure client node) and the Java EE specifications state that classic
thread creation (à la "new Thread() { @Override public void run() {}}") is
actually forbidden.

I know that it is working, at least on the application servers I know,
but I'm wondering what the ES teams says to this dilemma.

After all, doing something "illegal" against the specs might become
blocked/impossible in the future (not likely in this case) and it might
also cause strange side-effects on application servers.

Your opinions?

Best,

Tasha

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/aa786741-30fa-4ee8-a3ad-7e3b007b74de%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/aa786741-30fa-4ee8-a3ad-7e3b007b74de%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/VqJMG87S9jk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFwxuEEZ0h_WPr5S4JSFKbQjzNEW1osdi6edMVC32_nKg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFwxuEEZ0h_WPr5S4JSFKbQjzNEW1osdi6edMVC32_nKg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CABeBvCeXV-Fg-%3DjHS434_oKi7i8Dr839dAsB0PA6EmobrMeqsg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CABeBvCeXV-Fg-%3DjHS434_oKi7i8Dr839dAsB0PA6EmobrMeqsg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoEXnqUk9Kn3kd53yiWYG8vi2uas5NrkOLGHGeoOiNGE3w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Tasha) #6

Let me ask a different question then:

Do you agree when I say that if you include ES in your Java EE application
(JAR file) and you create a single tone to manage the ES connection to the
cluster. That in this case, the ES engine runs inside of your Java EE
application container?

Tasha

On 9 September 2014 10:15, joergprante@gmail.com joergprante@gmail.com
wrote:

I think you confuse two things:

  • the Java platform (Java Language, Java Virtual Machine)

  • the Java EE specification

These two things are different.

As the Java EE specification is community-driven and defined, it does not
mean to put any burden or restrictions on the Java platform, or on any
applications running on the JVM.

Note that ES nodes solve problems for distributed applications that Java
EE is not even aware of.

It is possible to give up certain ES features and write another ES node to
use the Java EE specification regarding the managed components and thread
pools very exactly but there is not much sense in it. Why should you expect
a Java EE container like Tomcat/Wildfly to operate like an ES cluster? They
simply can not do so and they will never do like this. You will challenge
many fields: resource allocation, reduced performance, less throughput,
less fault tolerance, distributed JVM orchestration etc.

Jörg

On Tue, Sep 9, 2014 at 9:57 AM, Tasha CARL tasha.cts@gmail.com wrote:

Elasticsearch was never designed to run as a container managed Java EE

component in a Java EE container, so the question about thread creation

is unrelated.

That's exactly my point and that's why I ask this question.

If you want to use an ES client from within a Java EE container,

start a node client or transport client as a singleton,

and stop it when the service stops.

That's the obvious way to handle this and that's exactly what I do.
Still, you include you ES JAR in your application and this JAR creates
threads out of the control of the Java EE container. Practically, they are
part of the application and the ES client runs inside the Java EE container.

Can you be more specific about "strange side effects on application
servers"?

I cannot be more specific since I did not yet encounter such effects. I
imagine that for example under heavy load, the ES client (JAR) could create
too many threads of which the container is not aware of that together with
the threads created by the container itself, the OS/VM limits/quota are
hit.

Tasha

On 9 September 2014 09:39, joergprante@gmail.com joergprante@gmail.com
wrote:

Your question is not very clear, but maybe you refer to JSR 236 for Java
EE.

Elasticsearch was never designed to run as a container managed Java EE
component in a Java EE container, so the question about thread creation is
unrelated.

If you want to use an ES client from within a Java EE container, start a
node client or transport client as a singleton, and stop it when the
service stops.

Can you be more specific about "strange side effects on application
servers"?

Jörg

On Mon, Sep 8, 2014 at 9:54 PM, Tasha tasha.cts@gmail.com wrote:

Hi all,

As you know, ElasticSearch relies a lot on multi-threading.

I'm currently using ES in a Java EE 6 application (using the node
client as pure client node) and the Java EE specifications state that
classic thread creation (à la "new Thread() { @Override public void run()
{}}") is actually forbidden.

I know that it is working, at least on the application servers I know,
but I'm wondering what the ES teams says to this dilemma.

After all, doing something "illegal" against the specs might become
blocked/impossible in the future (not likely in this case) and it might
also cause strange side-effects on application servers.

Your opinions?

Best,

Tasha

--

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CABeBvCfqwPz%3DnkWi4Tgx8i3RZWafreJy%2BtXr1SBWZPzsMWLsvA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #7

It depends.

With ES engine you mean the machinery to search and index I presume.

If you use embedded single node in a JEE container: yes.

If you use NodeClient in JEE container with data, but being part of an ES
cluster: yes and no.

If you use TransportClient: no, you just connect a client to an existing ES
cluster which runs outside JEE.

Jörg

On Tue, Sep 9, 2014 at 10:32 AM, Tasha CARL tasha.cts@gmail.com wrote:

Let me ask a different question then:

Do you agree when I say that if you include ES in your Java EE application
(JAR file) and you create a single tone to manage the ES connection to the
cluster. That in this case, the ES engine runs inside of your Java EE
application container?

Tasha

On 9 September 2014 10:15, joergprante@gmail.com joergprante@gmail.com
wrote:

I think you confuse two things:

  • the Java platform (Java Language, Java Virtual Machine)

  • the Java EE specification

These two things are different.

As the Java EE specification is community-driven and defined, it does not
mean to put any burden or restrictions on the Java platform, or on any
applications running on the JVM.

Note that ES nodes solve problems for distributed applications that Java
EE is not even aware of.

It is possible to give up certain ES features and write another ES node
to use the Java EE specification regarding the managed components and
thread pools very exactly but there is not much sense in it. Why should you
expect a Java EE container like Tomcat/Wildfly to operate like an ES
cluster? They simply can not do so and they will never do like this. You
will challenge many fields: resource allocation, reduced performance, less
throughput, less fault tolerance, distributed JVM orchestration etc.

Jörg

On Tue, Sep 9, 2014 at 9:57 AM, Tasha CARL tasha.cts@gmail.com wrote:

Elasticsearch was never designed to run as a container managed Java EE

component in a Java EE container, so the question about thread creation

is unrelated.

That's exactly my point and that's why I ask this question.

If you want to use an ES client from within a Java EE container,

start a node client or transport client as a singleton,

and stop it when the service stops.

That's the obvious way to handle this and that's exactly what I do.
Still, you include you ES JAR in your application and this JAR creates
threads out of the control of the Java EE container. Practically, they are
part of the application and the ES client runs inside the Java EE container.

Can you be more specific about "strange side effects on application
servers"?

I cannot be more specific since I did not yet encounter such effects. I
imagine that for example under heavy load, the ES client (JAR) could create
too many threads of which the container is not aware of that together with
the threads created by the container itself, the OS/VM limits/quota are
hit.

Tasha

On 9 September 2014 09:39, joergprante@gmail.com joergprante@gmail.com
wrote:

Your question is not very clear, but maybe you refer to JSR 236 for
Java EE.

Elasticsearch was never designed to run as a container managed Java EE
component in a Java EE container, so the question about thread creation is
unrelated.

If you want to use an ES client from within a Java EE container, start
a node client or transport client as a singleton, and stop it when the
service stops.

Can you be more specific about "strange side effects on application
servers"?

Jörg

On Mon, Sep 8, 2014 at 9:54 PM, Tasha tasha.cts@gmail.com wrote:

Hi all,

As you know, ElasticSearch relies a lot on multi-threading.

I'm currently using ES in a Java EE 6 application (using the node
client as pure client node) and the Java EE specifications state that
classic thread creation (à la "new Thread() { @Override public void run()
{}}") is actually forbidden.

I know that it is working, at least on the application servers I know,
but I'm wondering what the ES teams says to this dilemma.

After all, doing something "illegal" against the specs might become
blocked/impossible in the future (not likely in this case) and it might
also cause strange side-effects on application servers.

Your opinions?

Best,

Tasha

--

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CABeBvCfqwPz%3DnkWi4Tgx8i3RZWafreJy%2BtXr1SBWZPzsMWLsvA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CABeBvCfqwPz%3DnkWi4Tgx8i3RZWafreJy%2BtXr1SBWZPzsMWLsvA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHan%3DtfN3s2n%2B4X2p5TkLRk9p5LrF8hJ-rwoa%2Be8oAe%3DQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Tasha) #8

If you use NodeClient in JEE container with data, but being part of an ES
cluster: yes and no.

(think you mean without data)

That's what I do. Why yes and no? IMHO it's yes and if this is the case, it
should respect the Java EE specs :slight_smile:

On 9 September 2014 16:14, joergprante@gmail.com joergprante@gmail.com
wrote:

It depends.

With ES engine you mean the machinery to search and index I presume.

If you use embedded single node in a JEE container: yes.

If you use NodeClient in JEE container with data, but being part of an ES
cluster: yes and no.

If you use TransportClient: no, you just connect a client to an existing
ES cluster which runs outside JEE.

Jörg

On Tue, Sep 9, 2014 at 10:32 AM, Tasha CARL tasha.cts@gmail.com wrote:

Let me ask a different question then:

Do you agree when I say that if you include ES in your Java EE
application (JAR file) and you create a single tone to manage the ES
connection to the cluster. That in this case, the ES engine runs inside of
your Java EE application container?

Tasha

On 9 September 2014 10:15, joergprante@gmail.com joergprante@gmail.com
wrote:

I think you confuse two things:

  • the Java platform (Java Language, Java Virtual Machine)

  • the Java EE specification

These two things are different.

As the Java EE specification is community-driven and defined, it does
not mean to put any burden or restrictions on the Java platform, or on any
applications running on the JVM.

Note that ES nodes solve problems for distributed applications that Java
EE is not even aware of.

It is possible to give up certain ES features and write another ES node
to use the Java EE specification regarding the managed components and
thread pools very exactly but there is not much sense in it. Why should you
expect a Java EE container like Tomcat/Wildfly to operate like an ES
cluster? They simply can not do so and they will never do like this. You
will challenge many fields: resource allocation, reduced performance, less
throughput, less fault tolerance, distributed JVM orchestration etc.

Jörg

On Tue, Sep 9, 2014 at 9:57 AM, Tasha CARL tasha.cts@gmail.com wrote:

Elasticsearch was never designed to run as a container managed Java EE

component in a Java EE container, so the question about thread
creation

is unrelated.

That's exactly my point and that's why I ask this question.

If you want to use an ES client from within a Java EE container,

start a node client or transport client as a singleton,

and stop it when the service stops.

That's the obvious way to handle this and that's exactly what I do.
Still, you include you ES JAR in your application and this JAR creates
threads out of the control of the Java EE container. Practically, they are
part of the application and the ES client runs inside the Java EE container.

Can you be more specific about "strange side effects on application
servers"?

I cannot be more specific since I did not yet encounter such effects. I
imagine that for example under heavy load, the ES client (JAR) could create
too many threads of which the container is not aware of that together with
the threads created by the container itself, the OS/VM limits/quota are
hit.

Tasha

On 9 September 2014 09:39, joergprante@gmail.com <joergprante@gmail.com

wrote:

Your question is not very clear, but maybe you refer to JSR 236 for
Java EE.

Elasticsearch was never designed to run as a container managed Java EE
component in a Java EE container, so the question about thread creation is
unrelated.

If you want to use an ES client from within a Java EE container, start
a node client or transport client as a singleton, and stop it when the
service stops.

Can you be more specific about "strange side effects on application
servers"?

Jörg

On Mon, Sep 8, 2014 at 9:54 PM, Tasha tasha.cts@gmail.com wrote:

Hi all,

As you know, ElasticSearch relies a lot on multi-threading.

I'm currently using ES in a Java EE 6 application (using the node
client as pure client node) and the Java EE specifications state that
classic thread creation (à la "new Thread() { @Override public void run()
{}}") is actually forbidden.

I know that it is working, at least on the application servers I
know, but I'm wondering what the ES teams says to this dilemma.

After all, doing something "illegal" against the specs might become
blocked/impossible in the future (not likely in this case) and it might
also cause strange side-effects on application servers.

Your opinions?

Best,

Tasha

--

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CABeBvCfqwPz%3DnkWi4Tgx8i3RZWafreJy%2BtXr1SBWZPzsMWLsvA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CABeBvCfqwPz%3DnkWi4Tgx8i3RZWafreJy%2BtXr1SBWZPzsMWLsvA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/VqJMG87S9jk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHan%3DtfN3s2n%2B4X2p5TkLRk9p5LrF8hJ-rwoa%2Be8oAe%3DQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHan%3DtfN3s2n%2B4X2p5TkLRk9p5LrF8hJ-rwoa%2Be8oAe%3DQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CABeBvCfkbws8fKjEbVcv8XndL8pE_Rbz8hoUCbu%3Dwj7P_qsetg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #9

Again: why do you want ES respect the JEE specs? It is not a JEE component
at all and will never be.

You can run anything in a JEE container, also software which is not a JEE
component.

Running a NodeClient (with or without data) means the node is becoming a
part of the ES cluster, i.e. the JVM of the JEE server is becoming part of
the ES cluster. I'm not sure if this is what you want to achieve, regarding
cluster state management etc. A singleton of a TransportClient in a JEE
container is better for separation of concerns.

Jörg

On Tue, Sep 9, 2014 at 4:25 PM, Tasha CARL tasha.cts@gmail.com wrote:

If you use NodeClient in JEE container with data, but being part of an ES
cluster: yes and no.

(think you mean without data)

That's what I do. Why yes and no? IMHO it's yes and if this is the case,
it should respect the Java EE specs :slight_smile:

On 9 September 2014 16:14, joergprante@gmail.com joergprante@gmail.com
wrote:

It depends.

With ES engine you mean the machinery to search and index I presume.

If you use embedded single node in a JEE container: yes.

If you use NodeClient in JEE container with data, but being part of an ES
cluster: yes and no.

If you use TransportClient: no, you just connect a client to an existing
ES cluster which runs outside JEE.

Jörg

On Tue, Sep 9, 2014 at 10:32 AM, Tasha CARL tasha.cts@gmail.com wrote:

Let me ask a different question then:

Do you agree when I say that if you include ES in your Java EE
application (JAR file) and you create a single tone to manage the ES
connection to the cluster. That in this case, the ES engine runs inside of
your Java EE application container?

Tasha

On 9 September 2014 10:15, joergprante@gmail.com joergprante@gmail.com
wrote:

I think you confuse two things:

  • the Java platform (Java Language, Java Virtual Machine)

  • the Java EE specification

These two things are different.

As the Java EE specification is community-driven and defined, it does
not mean to put any burden or restrictions on the Java platform, or on any
applications running on the JVM.

Note that ES nodes solve problems for distributed applications that
Java EE is not even aware of.

It is possible to give up certain ES features and write another ES node
to use the Java EE specification regarding the managed components and
thread pools very exactly but there is not much sense in it. Why should you
expect a Java EE container like Tomcat/Wildfly to operate like an ES
cluster? They simply can not do so and they will never do like this. You
will challenge many fields: resource allocation, reduced performance, less
throughput, less fault tolerance, distributed JVM orchestration etc.

Jörg

On Tue, Sep 9, 2014 at 9:57 AM, Tasha CARL tasha.cts@gmail.com wrote:

Elasticsearch was never designed to run as a container managed Java
EE

component in a Java EE container, so the question about thread
creation

is unrelated.

That's exactly my point and that's why I ask this question.

If you want to use an ES client from within a Java EE container,

start a node client or transport client as a singleton,

and stop it when the service stops.

That's the obvious way to handle this and that's exactly what I do.
Still, you include you ES JAR in your application and this JAR creates
threads out of the control of the Java EE container. Practically, they are
part of the application and the ES client runs inside the Java EE container.

Can you be more specific about "strange side effects on application
servers"?

I cannot be more specific since I did not yet encounter such effects.
I imagine that for example under heavy load, the ES client (JAR) could
create too many threads of which the container is not aware of that
together with the threads created by the container itself, the OS/VM
limits/quota are hit.

Tasha

On 9 September 2014 09:39, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Your question is not very clear, but maybe you refer to JSR 236 for
Java EE.

Elasticsearch was never designed to run as a container managed Java
EE component in a Java EE container, so the question about thread creation
is unrelated.

If you want to use an ES client from within a Java EE container,
start a node client or transport client as a singleton, and stop it when
the service stops.

Can you be more specific about "strange side effects on application
servers"?

Jörg

On Mon, Sep 8, 2014 at 9:54 PM, Tasha tasha.cts@gmail.com wrote:

Hi all,

As you know, ElasticSearch relies a lot on multi-threading.

I'm currently using ES in a Java EE 6 application (using the node
client as pure client node) and the Java EE specifications state that
classic thread creation (à la "new Thread() { @Override public void run()
{}}") is actually forbidden.

I know that it is working, at least on the application servers I
know, but I'm wondering what the ES teams says to this dilemma.

After all, doing something "illegal" against the specs might become
blocked/impossible in the future (not likely in this case) and it might
also cause strange side-effects on application servers.

Your opinions?

Best,

Tasha

--

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CABeBvCfqwPz%3DnkWi4Tgx8i3RZWafreJy%2BtXr1SBWZPzsMWLsvA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CABeBvCfqwPz%3DnkWi4Tgx8i3RZWafreJy%2BtXr1SBWZPzsMWLsvA%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/elasticsearch/VqJMG87S9jk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHan%3DtfN3s2n%2B4X2p5TkLRk9p5LrF8hJ-rwoa%2Be8oAe%3DQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHan%3DtfN3s2n%2B4X2p5TkLRk9p5LrF8hJ-rwoa%2Be8oAe%3DQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CABeBvCfkbws8fKjEbVcv8XndL8pE_Rbz8hoUCbu%3Dwj7P_qsetg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CABeBvCfkbws8fKjEbVcv8XndL8pE_Rbz8hoUCbu%3Dwj7P_qsetg%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGd9Afea1jeccCza8nN87Fnch3D9Nvqr795FRm9j8RSPA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Tasha) #10

You can run anything in a JEE container, also software which is not a JEE component.

That's exactly the point of the question :slight_smile:

De : elasticsearch@googlegroups.com [mailto:elasticsearch@googlegroups.com] De la part de joergprante@gmail.com
Envoyé : mardi 9 septembre 2014 16:51
À : elasticsearch@googlegroups.com
Objet : Re: ElasticSearch multi-threading and the Java EE specifications

Again: why do you want ES respect the JEE specs? It is not a JEE component at all and will never be.

You can run anything in a JEE container, also software which is not a JEE component.

Running a NodeClient (with or without data) means the node is becoming a part of the ES cluster, i.e. the JVM of the JEE server is becoming part of the ES cluster. I'm not sure if this is what you want to achieve, regarding cluster state management etc. A singleton of a TransportClient in a JEE container is better for separation of concerns.

Jörg

On Tue, Sep 9, 2014 at 4:25 PM, Tasha CARL tasha.cts@gmail.com wrote:

If you use NodeClient in JEE container with data, but being part of an ES cluster: yes and no.

(think you mean without data)

That's what I do. Why yes and no? IMHO it's yes and if this is the case, it should respect the Java EE specs :slight_smile:

On 9 September 2014 16:14, joergprante@gmail.com joergprante@gmail.com wrote:

It depends.

With ES engine you mean the machinery to search and index I presume.

If you use embedded single node in a JEE container: yes.

If you use NodeClient in JEE container with data, but being part of an ES cluster: yes and no.

If you use TransportClient: no, you just connect a client to an existing ES cluster which runs outside JEE.

Jörg

On Tue, Sep 9, 2014 at 10:32 AM, Tasha CARL tasha.cts@gmail.com wrote:

Let me ask a different question then:

Do you agree when I say that if you include ES in your Java EE application (JAR file) and you create a single tone to manage the ES connection to the cluster. That in this case, the ES engine runs inside of your Java EE application container?

Tasha

On 9 September 2014 10:15, joergprante@gmail.com joergprante@gmail.com wrote:

I think you confuse two things:

  • the Java platform (Java Language, Java Virtual Machine)

  • the Java EE specification

These two things are different.

As the Java EE specification is community-driven and defined, it does not mean to put any burden or restrictions on the Java platform, or on any applications running on the JVM.

Note that ES nodes solve problems for distributed applications that Java EE is not even aware of.

It is possible to give up certain ES features and write another ES node to use the Java EE specification regarding the managed components and thread pools very exactly but there is not much sense in it. Why should you expect a Java EE container like Tomcat/Wildfly to operate like an ES cluster? They simply can not do so and they will never do like this. You will challenge many fields: resource allocation, reduced performance, less throughput, less fault tolerance, distributed JVM orchestration etc.

Jörg

On Tue, Sep 9, 2014 at 9:57 AM, Tasha CARL tasha.cts@gmail.com wrote:

Elasticsearch was never designed to run as a container managed Java EE

component in a Java EE container, so the question about thread creation

is unrelated.

That's exactly my point and that's why I ask this question.

If you want to use an ES client from within a Java EE container,

start a node client or transport client as a singleton,

and stop it when the service stops.

That's the obvious way to handle this and that's exactly what I do. Still, you include you ES JAR in your application and this JAR creates threads out of the control of the Java EE container. Practically, they are part of the application and the ES client runs inside the Java EE container.

Can you be more specific about "strange side effects on application servers"?

I cannot be more specific since I did not yet encounter such effects. I imagine that for example under heavy load, the ES client (JAR) could create too many threads of which the container is not aware of that together with the threads created by the container itself, the OS/VM limits/quota are hit.

Tasha

On 9 September 2014 09:39, joergprante@gmail.com joergprante@gmail.com wrote:

Your question is not very clear, but maybe you refer to JSR 236 for Java EE.

Elasticsearch was never designed to run as a container managed Java EE component in a Java EE container, so the question about thread creation is unrelated.

If you want to use an ES client from within a Java EE container, start a node client or transport client as a singleton, and stop it when the service stops.

Can you be more specific about "strange side effects on application servers"?

Jörg

On Mon, Sep 8, 2014 at 9:54 PM, Tasha tasha.cts@gmail.com wrote:

Hi all,

As you know, ElasticSearch relies a lot on multi-threading.

I'm currently using ES in a Java EE 6 application (using the node client as pure client node) and the Java EE specifications state that classic thread creation (à la "new Thread() { @Override public void run() {}}") is actually forbidden.

I know that it is working, at least on the application servers I know, but I'm wondering what the ES teams says to this dilemma.

After all, doing something "illegal" against the specs might become blocked/impossible in the future (not likely in this case) and it might also cause strange side-effects on application servers.

Your opinions?

Best,

Tasha

--

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CABeBvCfqwPz%3DnkWi4Tgx8i3RZWafreJy%2BtXr1SBWZPzsMWLsvA%40mail.gmail.com https://groups.google.com/d/msgid/elasticsearch/CABeBvCfqwPz%3DnkWi4Tgx8i3RZWafreJy%2BtXr1SBWZPzsMWLsvA%40mail.gmail.com?utm_medium=email&utm_source=footer .

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/VqJMG87S9jk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHan%3DtfN3s2n%2B4X2p5TkLRk9p5LrF8hJ-rwoa%2Be8oAe%3DQ%40mail.gmail.com https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHan%3DtfN3s2n%2B4X2p5TkLRk9p5LrF8hJ-rwoa%2Be8oAe%3DQ%40mail.gmail.com?utm_medium=email&utm_source=footer .

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CABeBvCfkbws8fKjEbVcv8XndL8pE_Rbz8hoUCbu%3Dwj7P_qsetg%40mail.gmail.com https://groups.google.com/d/msgid/elasticsearch/CABeBvCfkbws8fKjEbVcv8XndL8pE_Rbz8hoUCbu%3Dwj7P_qsetg%40mail.gmail.com?utm_medium=email&utm_source=footer .

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elasticsearch/VqJMG87S9jk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGd9Afea1jeccCza8nN87Fnch3D9Nvqr795FRm9j8RSPA%40mail.gmail.com https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGd9Afea1jeccCza8nN87Fnch3D9Nvqr795FRm9j8RSPA%40mail.gmail.com?utm_medium=email&utm_source=footer .
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/540f1e74.8850c20a.03f6.1f08%40mx.google.com.
For more options, visit https://groups.google.com/d/optout.


(system) #11