Correct way to use TransportClient connection object


(bagui) #1

Hi,

I'm using the below code to get a singleton object for TransportClient
object. I'm using the getInstance() to get the client object which is
already alive in webapplication.

public static Client getInstance()
{
if (instance == null)
{
logger.debug("the client instance is null, creating a new instance");
ImmutableSettings.Builder settings = ImmutableSettings.settingsBuilder();
settings.put("node.client", true);
settings.put("node.data", false);
settings.put("node.name", "node-client");
settings.put("cluster.name", "elasticsearch");
settings.build();
instance = new TransportClient(settings)
.addTransportAddress(new
InetSocketTransportAddress("10.203.238.139", 9300));
logger.debug("returning the new created client instance...");
return instance;
}
return instance;
}

Calling the client as below from search api.
Client client = ESClientFactory.getInstance();

Now the issue is if I don't close client like client.close() I'm getting
memory leak warning from webserver tomcat side. If I do close the
connection using client.close() after search api call then I'm getting NoNodeAvailableException
exception.

Please suggest what is the correct way to call the connection object.

Thanks,
Subhadip

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/e2aaf77f-cd18-4e52-98fc-c25ed03601fd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #2

Closing the transport client may not be enough.

Try this:

  • wait for all outstanding actions (all actions send responses
    asynchronously)
  • then shut down client.threadpool() (perhaps with shutdownNow() or
    shutdown()), this effectively disables new actions form being started
  • then close the transport client with client.close()
  • then set the client instance to null to allow GC to clean up

Also you should put the static transport client object in a
ServetContextListener so the web app container is able to manage start and
stop in contextInitialized() and contextDestroyed()

Beside this, memory leak warnings from tomcat are warnings. They may be
right or wrong.

Jörg

On Fri, Jun 6, 2014 at 7:43 AM, Subhadip Bagui i.bagui@gmail.com wrote:

Hi,

I'm using the below code to get a singleton object for TransportClient
object. I'm using the getInstance() to get the client object which is
already alive in webapplication.

public static Client getInstance()
{
if (instance == null)
{
logger.debug("the client instance is null, creating a new instance");
ImmutableSettings.Builder settings = ImmutableSettings.settingsBuilder();
settings.put("node.client", true);
settings.put("node.data", false);
settings.put("node.name", "node-client");
settings.put("cluster.name", "elasticsearch");
settings.build();
instance = new TransportClient(settings)
.addTransportAddress(new
InetSocketTransportAddress("10.203.238.139", 9300));
logger.debug("returning the new created client instance...");
return instance;
}
return instance;
}

Calling the client as below from search api.
Client client = ESClientFactory.getInstance();

Now the issue is if I don't close client like client.close() I'm getting
memory leak warning from webserver tomcat side. If I do close the
connection using client.close() after search api call then I'm getting NoNodeAvailableException
exception.

Please suggest what is the correct way to call the connection object.

Thanks,
Subhadip

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/e2aaf77f-cd18-4e52-98fc-c25ed03601fd%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/e2aaf77f-cd18-4e52-98fc-c25ed03601fd%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHL3rAeBsHLrWYkm4-MqO7oU_9CJmL7Ge6SPS15FoyJmg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(bagui) #3

Hi Jorg,

Sorry to open the thread again. But the issue is I'm getting OOM error
currently in tomcat and the webapplication is crushing. As u suggested I'm
calling the static TransportClient instance from contextInitialized() and
shutting down with contextDestroyed(). But the methods are not getting
called it seems. Trying like below.
Can you please check.

public class ESClientFactory implements ServletContextListener {

/** The logger. */
private static Logger logger = Logger.getLogger(ESClientFactory.class);

/** The instance. */
public static TransportClient instance;

/**
 * Instantiates a new eS client factory.
 */
private ESClientFactory() {
}

/**
 * Gets the single instance of ESClientFactory.
 *
 * @return single instance of ESClientFactory
 */
public static Client getInstance() {
    String ipAddress = MessageTranslator.getMessage("es.cluster.ip");
    int transportClientPort = 0;
    String clusterName = 

MessageTranslator.getMessage("es.cluster.name");

    try {
        transportClientPort =
            Integer.parseInt(MessageTranslator
                .getMessage("es.transportclient.port"));
    }
    catch (Exception e) {
        transportClientPort = 9300;
        LogImpl.setWarning(ESClientFactory.class, e);
    }

    logger.debug("got the client ip as :" + ipAddress + " and port :"
        + transportClientPort);
    if (instance == null) {
        logger
            .debug("the client instance is null, creating a new 

instance");
ImmutableSettings.Builder settings =
ImmutableSettings.settingsBuilder();
settings.put("cluster.name", clusterName);
settings.build();
instance =
new TransportClient(settings)
.addTransportAddress(new InetSocketTransportAddress(
ipAddress, transportClientPort));

        logger.debug("returning the new created client instance...");
        return instance;
    }
    logger
        .debug("returning the existing transport client object 

connection.");
return instance;
}

@Override
public void contextInitialized(ServletContextEvent sce) {
logger.debug("initializing the servletContextListener... TransportClient");
getInstance();
}

@Override
public void contextDestroyed(ServletContextEvent sce) {
logger.debug("closing the servlet context");
shutdown();
logger.debug("successfully shutdown threadpool");
}

public synchronized void shutdown() {
if (instance != null) {
logger.debug("shutdown started");
instance.close();
instance.threadPool().shutdown();
instance = null;
logger.debug("shutdown complete");
}
}

}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/814fdb41-461b-4b5c-8bad-835039d76c24%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Ed Kim) #4

Did you declare the listener in your web.xml?

On Tuesday, January 13, 2015 at 3:27:15 AM UTC-8, Subhadip Bagui wrote:

Hi Jorg,

Sorry to open the thread again. But the issue is I'm getting OOM error
currently in tomcat and the webapplication is crushing. As u suggested I'm
calling the static TransportClient instance from contextInitialized() and
shutting down with contextDestroyed(). But the methods are not getting
called it seems. Trying like below.
Can you please check.

public class ESClientFactory implements ServletContextListener {

/** The logger. */
private static Logger logger = Logger.getLogger(ESClientFactory.class);

/** The instance. */
public static TransportClient instance;

/**
 * Instantiates a new eS client factory.
 */
private ESClientFactory() {
}

/**
 * Gets the single instance of ESClientFactory.
 *
 * @return single instance of ESClientFactory
 */
public static Client getInstance() {
    String ipAddress = MessageTranslator.getMessage("es.cluster.ip");
    int transportClientPort = 0;
    String clusterName = MessageTranslator.getMessage("es.cluster.name

");

    try {
        transportClientPort =
            Integer.parseInt(MessageTranslator
                .getMessage("es.transportclient.port"));
    }
    catch (Exception e) {
        transportClientPort = 9300;
        LogImpl.setWarning(ESClientFactory.class, e);
    }

    logger.debug("got the client ip as :" + ipAddress + " and port :"
        + transportClientPort);
    if (instance == null) {
        logger
            .debug("the client instance is null, creating a new 

instance");
ImmutableSettings.Builder settings =
ImmutableSettings.settingsBuilder();
settings.put("cluster.name", clusterName);
settings.build();
instance =
new TransportClient(settings)
.addTransportAddress(new InetSocketTransportAddress(
ipAddress, transportClientPort));

        logger.debug("returning the new created client instance...");
        return instance;
    }
    logger
        .debug("returning the existing transport client object 

connection.");
return instance;
}

@Override
public void contextInitialized(ServletContextEvent sce) {
logger.debug("initializing the servletContextListener... TransportClient");
getInstance();
}

@Override
public void contextDestroyed(ServletContextEvent sce) {
logger.debug("closing the servlet context");
shutdown();
logger.debug("successfully shutdown threadpool");
}

public synchronized void shutdown() {
if (instance != null) {
logger.debug("shutdown started");
instance.close();
instance.threadPool().shutdown();
instance = null;
logger.debug("shutdown complete");
}
}

}

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/fae4cd13-9f9f-4187-9e23-93d8c73c3132%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(bagui) #5

Yeah. I'm using listener, but it was giving some error. Fixed it and now
it's working. My mistake :blush:

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/26d40ffe-4d0b-4c49-a81c-e86e5f091b9a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #6

In Servlet 3.0 you can also add @WebServletContextListener as annotation to
the class, without having to use web.xml

Example:

https://blogs.oracle.com/swchan/entry/servlet_3_0_annotations

Jörg

On Wed, Jan 14, 2015 at 7:50 AM, Subhadip Bagui i.bagui@gmail.com wrote:

Yeah. I'm using listener, but it was giving some error. Fixed it and now
it's working. My mistake :blush:

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/26d40ffe-4d0b-4c49-a81c-e86e5f091b9a%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/26d40ffe-4d0b-4c49-a81c-e86e5f091b9a%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoGhcP72aYs9arcM_VCqu2HZDp3e%2B2w5sf5NhXPE8DMqbg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(bagui) #7

Hi,

In the same context... some times when I'm shutting down tomcat getting the
below exception. And other times it works. Any idea why ?

Jan 19, 2015 8:59:30 AM org.apache.catalina.core.StandardContext
listenerStop
SEVERE: Exception sending context destroyed event to listener instance of
class com.aricent.aricloud.es.service.ESClientFactory
java.lang.NoClassDefFoundError:
org/elasticsearch/transport/netty/NettyTransport$4
at
org.elasticsearch.transport.netty.NettyTransport.doStop(NettyTransport.java:403)
at
org.elasticsearch.common.component.AbstractLifecycleComponent.stop(AbstractLifecycleComponent.java:105)
at
org.elasticsearch.transport.TransportService.doStop(TransportService.java:100)
at
org.elasticsearch.common.component.AbstractLifecycleComponent.stop(AbstractLifecycleComponent.java:105)
at
org.elasticsearch.common.component.AbstractLifecycleComponent.close(AbstractLifecycleComponent.java:117)
at
org.elasticsearch.client.transport.TransportClient.close(TransportClient.java:268)
at
com.aricent.aricloud.es.service.ESClientFactory.shutdown(ESClientFactory.java:118)
at
com.aricent.aricloud.es.service.ESClientFactory.contextDestroyed(ESClientFactory.java:111)

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4b948428-c260-4aef-ad82-93346e7488cb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #8

This is an exception because TransportClient still has open requests and
was not closed in time, i.e. before Tomcat closed the web app and removed
the class loader.

Jörg

On Mon, Jan 19, 2015 at 4:37 AM, Subhadip Bagui i.bagui@gmail.com wrote:

Hi,

In the same context... some times when I'm shutting down tomcat getting
the below exception. And other times it works. Any idea why ?

Jan 19, 2015 8:59:30 AM org.apache.catalina.core.StandardContext
listenerStop
SEVERE: Exception sending context destroyed event to listener instance of
class com.aricent.aricloud.es.service.ESClientFactory
java.lang.NoClassDefFoundError:
org/elasticsearch/transport/netty/NettyTransport$4
at
org.elasticsearch.transport.netty.NettyTransport.doStop(NettyTransport.java:403)
at
org.elasticsearch.common.component.AbstractLifecycleComponent.stop(AbstractLifecycleComponent.java:105)
at
org.elasticsearch.transport.TransportService.doStop(TransportService.java:100)
at
org.elasticsearch.common.component.AbstractLifecycleComponent.stop(AbstractLifecycleComponent.java:105)
at
org.elasticsearch.common.component.AbstractLifecycleComponent.close(AbstractLifecycleComponent.java:117)
at
org.elasticsearch.client.transport.TransportClient.close(TransportClient.java:268)
at
com.aricent.aricloud.es.service.ESClientFactory.shutdown(ESClientFactory.java:118)
at
com.aricent.aricloud.es.service.ESClientFactory.contextDestroyed(ESClientFactory.java:111)

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/4b948428-c260-4aef-ad82-93346e7488cb%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/4b948428-c260-4aef-ad82-93346e7488cb%40googlegroups.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFv4vXGUHK%3DGR11UDb6KZ1N8jT7WMGeKkpj1UCcSwV45w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Arivazhagan Jeganathan) #9

I am facing org.elasticsearch.common.util.concurrent.EsRejectedExecutionException: during shutdown of client.

I am doing the following operations:

	client.threadPool().scheduler().shutdownNow();
	while (!client.threadPool().scheduler().isShutdown()) {
		client.threadPool().scheduler().shutdown();
	}
	client.threadPool().awaitTermination(60000, TimeUnit.MILLISECONDS);
	client.close();
	client = null;

I am getting error while invoking client.close(), Please help.
Due to this class loader is not getting unloaded.


(system) #10