Yet another "WARNING: Message not fully read" thread


(tufi) #1

Hi,

also I am getting the warning "Message not fully read" from one of my ES
Nodes.
I certainly googled a lot and found out that, different versions of JVM as
well as different versions of ES server and clients can cause this.

Well, I'm pretty sure, I have the same versions everywhere.

My use case:
Programming a multi-thread java-app to import about 30G data out of approx.
1000 csv files to Elasticsearch. I use transport client for that, to send
bulkImport requests with <10.000 bulk objects for each request.

ES Cluster:
*1 Master: *4 Cores + 16 GB RAM
Overwritten properties in elasticsearch.yml

cluster.name: $CLUSTERNAME
node.name: $NODENAME
node.master: true
#node.data: false
node.max_local_storage_nodes: 1
index.number_of_shards: 8 # 4 x $numberOfNodes since I have 2 data nodes 8
index.number_of_replicas: 1

Versions:

$bin/elasticsearch -v
Version: 1.0.0, Build: a46900e/2014-02-12T16:18:34Z, JVM: 1.7.0_25

1st Slave: 4 Cores + 16 GB RAM
Overwritten properties in elasticsearch.yml

cluster.name: $CLUSTERNAME
node.name: $NODENAME
node.master: false

node.max_local_storage_nodes: 1

thread pool

threadpool.bulk.queue_size: -1

path.data: /opt/elasticsearch/data
path.plugins: /opt/elasticsearch/plugins
path.conf: /opt/elasticsearch/config

Versions:
$bin/elasticsearch -v
Version: 1.0.0, Build: a46900e/2014-02-12T16:18:34Z, JVM: 1.7.0_25

2nd Slave: 4 Cores + 16 GB RAM
Overwritten properties in elasticsearch.yml

cluster.name: $CLUSTERNAME
node.name: $NODENAME
node.master: false

node.max_local_storage_nodes: 1

thread pool

threadpool.bulk.queue_size: -1

path.data: /opt/elasticsearch/data
path.plugins: /opt/elasticsearch/plugins
path.conf: /opt/elasticsearch/config

Versions:
$bin/elasticsearch -v
Version: 1.0.0, Build: a46900e/2014-02-12T16:18:34Z, JVM: 1.7.0_25

*My Client: *4 Cores + 16 GB RAM
pom.xml

<es.version>1.0.0</es.version>



org.elasticsearch
elasticsearch
${es.version}

Maven: org.elasticsearch:elasticsearch:1.0.0
is indeed under libs/ :

elasticsearch-1.0.0.jar

Java Code to initialize the transport client:

    XMLConfiguration config = ConfigLoader.getInstance().getConfig();
    config.setExpressionEngine(new XPathExpressionEngine());
    HierarchicalConfiguration hConf = config.configurationAt(

"/database/es/node[@master='true']");

    String clusterName = hConf.getString("clusterName");
    String url = hConf.getString("url");
    int port = hConf.getInt("port/transport");

    System.out.println("clusterName = " + clusterName);
    Settings settings = ImmutableSettings.settingsBuilder()
            .put("cluster.name", clusterName).build();
    client = new TransportClient(settings).addTransportAddress(
            new InetSocketTransportAddress(url, port));

Last but not least; java version on App-Runner:

System.out.println(System.getProperty("java.version"));

1.7.0_25

Is there anything else, which I can check for getting rid of these
warnings?
What exactly does this warning mean? Do I lose data?

Since I'm new to ES; do I have to set the ES config values node.max_local_storage_nodes,
index.number_of_shards, index.number_of_replicas
also in the slave
configs?

Would you need additional information?

Thank you very much for your help.

Tufi.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/c64630fb-768f-47c8-9225-4d1290580703%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #2

Thanks for the very commendable description.

Do you have also a full stack trace of the "Message not fully read"
exception? Is it happening in the client?

Do you use any plugins?

Jörg

On Thu, Mar 20, 2014 at 3:37 PM, tufi tufan.oezduman@yoc.com wrote:

Hi,

also I am getting the warning "Message not fully read" from one of my ES
Nodes.
I certainly googled a lot and found out that, different versions of JVM as
well as different versions of ES server and clients can cause this.

Well, I'm pretty sure, I have the same versions everywhere.

My use case:
Programming a multi-thread java-app to import about 30G data out of
approx. 1000 csv files to Elasticsearch. I use transport client for that,
to send bulkImport requests with <10.000 bulk objects for each request.

ES Cluster:
*1 Master: *4 Cores + 16 GB RAM
Overwritten properties in elasticsearch.yml

cluster.name: $CLUSTERNAME
node.name: $NODENAME
node.master: true
#node.data: false
node.max_local_storage_nodes: 1
index.number_of_shards: 8 # 4 x $numberOfNodes since I have 2 data nodes 8
index.number_of_replicas: 1

Versions:

$bin/elasticsearch -v
Version: 1.0.0, Build: a46900e/2014-02-12T16:18:34Z, JVM: 1.7.0_25

1st Slave: 4 Cores + 16 GB RAM
Overwritten properties in elasticsearch.yml

cluster.name: $CLUSTERNAME
node.name: $NODENAME
node.master: false

node.max_local_storage_nodes: 1

thread pool

threadpool.bulk.queue_size: -1

path.data: /opt/elasticsearch/data
path.plugins: /opt/elasticsearch/plugins
path.conf: /opt/elasticsearch/config

Versions:
$bin/elasticsearch -v
Version: 1.0.0, Build: a46900e/2014-02-12T16:18:34Z, JVM: 1.7.0_25

2nd Slave: 4 Cores + 16 GB RAM
Overwritten properties in elasticsearch.yml

cluster.name: $CLUSTERNAME
node.name: $NODENAME
node.master: false

node.max_local_storage_nodes: 1

thread pool

threadpool.bulk.queue_size: -1

path.data: /opt/elasticsearch/data
path.plugins: /opt/elasticsearch/plugins
path.conf: /opt/elasticsearch/config

Versions:
$bin/elasticsearch -v
Version: 1.0.0, Build: a46900e/2014-02-12T16:18:34Z, JVM: 1.7.0_25

*My Client: *4 Cores + 16 GB RAM
pom.xml

<es.version>1.0.0</es.version>



org.elasticsearch
elasticsearch
${es.version}

Maven: org.elasticsearch:elasticsearch:1.0.0
is indeed under libs/ :

elasticsearch-1.0.0.jar

Java Code to initialize the transport client:

    XMLConfiguration config = ConfigLoader.getInstance().getConfig();
    config.setExpressionEngine(new XPathExpressionEngine());
    HierarchicalConfiguration hConf = config.configurationAt(

"/database/es/node[@master='true']");

    String clusterName = hConf.getString("clusterName");
    String url = hConf.getString("url");
    int port = hConf.getInt("port/transport");

    System.out.println("clusterName = " + clusterName);
    Settings settings = ImmutableSettings.settingsBuilder()
            .put("cluster.name", clusterName).build();
    client = new TransportClient(settings).addTransportAddress(
            new InetSocketTransportAddress(url, port));

Last but not least; java version on App-Runner:

System.out.println(System.getProperty("java.version"));

1.7.0_25

Is there anything else, which I can check for getting rid of these
warnings?
What exactly does this warning mean? Do I lose data?

Since I'm new to ES; do I have to set the ES config values node.max_local_storage_nodes,
index.number_of_shards, index.number_of_replicas
also in the slave
configs?

Would you need additional information?

Thank you very much for your help.

Tufi.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c64630fb-768f-47c8-9225-4d1290580703%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/c64630fb-768f-47c8-9225-4d1290580703%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoH_cQvgf3SWM%2BJEPLzEdJoAw8Lfp2c-OEG0mtQKTn5Z-g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(tufi) #3

Hi Jörg,

yes, more or less... The full warning line is:

Mrz 20, 2014 11:55:11 AM org.elasticsearch.transport.netty
WARNING: [Stick] Message not fully read (response) for [5507] handler future
(org.elasticsearch.client.transport.
TransportClientNodesService$SimpleNodeSampler$1@4893ccd0), error [false],resetting

This is happening in the client.
I do use elasticsearch-head and HQ plug-ins installed on the master.

Thanks.
Tufi.

Am Donnerstag, 20. März 2014 20:28:55 UTC+1 schrieb Jörg Prante:

Thanks for the very commendable description.

Do you have also a full stack trace of the "Message not fully read"
exception? Is it happening in the client?

Do you use any plugins?

Jörg

On Thu, Mar 20, 2014 at 3:37 PM, tufi <tufan.o...@yoc.com <javascript:>>wrote:

Hi,

also I am getting the warning "Message not fully read" from one of my ES
Nodes.
I certainly googled a lot and found out that, different versions of JVM
as well as different versions of ES server and clients can cause this.

Well, I'm pretty sure, I have the same versions everywhere.

My use case:
Programming a multi-thread java-app to import about 30G data out of
approx. 1000 csv files to Elasticsearch. I use transport client for that,
to send bulkImport requests with <10.000 bulk objects for each request.

ES Cluster:
*1 Master: *4 Cores + 16 GB RAM
Overwritten properties in elasticsearch.yml

cluster.name: $CLUSTERNAME
node.name: $NODENAME
node.master: true
#node.data: false
node.max_local_storage_nodes: 1
index.number_of_shards: 8 # 4 x $numberOfNodes since I have 2 data nodes
8
index.number_of_replicas: 1

Versions:

$bin/elasticsearch -v
Version: 1.0.0, Build: a46900e/2014-02-12T16:18:34Z, JVM: 1.7.0_25

1st Slave: 4 Cores + 16 GB RAM
Overwritten properties in elasticsearch.yml

cluster.name: $CLUSTERNAME
node.name: $NODENAME
node.master: false

node.max_local_storage_nodes: 1

thread pool

threadpool.bulk.queue_size: -1

path.data: /opt/elasticsearch/data
path.plugins: /opt/elasticsearch/plugins
path.conf: /opt/elasticsearch/config

Versions:
$bin/elasticsearch -v
Version: 1.0.0, Build: a46900e/2014-02-12T16:18:34Z, JVM: 1.7.0_25

2nd Slave: 4 Cores + 16 GB RAM
Overwritten properties in elasticsearch.yml

cluster.name: $CLUSTERNAME
node.name: $NODENAME
node.master: false

node.max_local_storage_nodes: 1

thread pool

threadpool.bulk.queue_size: -1

path.data: /opt/elasticsearch/data
path.plugins: /opt/elasticsearch/plugins
path.conf: /opt/elasticsearch/config

Versions:
$bin/elasticsearch -v
Version: 1.0.0, Build: a46900e/2014-02-12T16:18:34Z, JVM: 1.7.0_25

*My Client: *4 Cores + 16 GB RAM
pom.xml

<es.version>1.0.0</es.version>



org.elasticsearch
elasticsearch
${es.version}

Maven: org.elasticsearch:elasticsearch:1.0.0
is indeed under libs/ :

elasticsearch-1.0.0.jar

Java Code to initialize the transport client:

    XMLConfiguration config = ConfigLoader.getInstance().getConfig();
    config.setExpressionEngine(new XPathExpressionEngine());
    HierarchicalConfiguration hConf = config.configurationAt(

"/database/es/node[@master='true']");

    String clusterName = hConf.getString("clusterName");
    String url = hConf.getString("url");
    int port = hConf.getInt("port/transport");

    System.out.println("clusterName = " + clusterName);
    Settings settings = ImmutableSettings.settingsBuilder()
            .put("cluster.name", clusterName).build();
    client = new TransportClient(settings).addTransportAddress(
            new InetSocketTransportAddress(url, port));

Last but not least; java version on App-Runner:

System.out.println(System.getProperty("java.version"));

1.7.0_25

Is there anything else, which I can check for getting rid of these
warnings?
What exactly does this warning mean? Do I lose data?

Since I'm new to ES; do I have to set the ES config values node.max_local_storage_nodes,
index.number_of_shards, index.number_of_replicas
also in the slave
configs?

Would you need additional information?

Thank you very much for your help.

Tufi.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearc...@googlegroups.com <javascript:>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/c64630fb-768f-47c8-9225-4d1290580703%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/c64630fb-768f-47c8-9225-4d1290580703%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/254d604f-e6eb-4cb6-b592-ccb27575de47%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(tufi) #4

...I noticed that, also an update request from ES HQ browser frontend
causes this warning. So this seems to be an issue of right configuration
parameters?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/7baaec6f-3549-4d71-a625-bf60b2a228d2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #5