_status won't show indices after 0.6.0 to 0.7.1 upgrade


(John Chang) #1

I was using 0.6.0 successfully. I wanted to upgrade to 0.7.1.

I unzipped 0.7.1 (different location on machine) and copied over elasticsearch.yml from the 0.6.0 installation.

I stopped the 0.6.0 instance and started the 0.7.1 instance.

I can add indices as expected (as described here: http://www.elasticsearch.com/docs/elasticsearch/rest_api/admin/indices/create_index/).

When I do a _status call, though, it shows no indices. I go back and try to add the index again, and it complains that the index already exists. Yet, _status always shows no indices.

When I try to query against the index (as described here: http://www.elasticsearch.com/docs/elasticsearch/rest_api/search/), it just hangs. When I try to query against an index I know to be non-existent, it complains as expected that the index is not there.


(Shay Banon) #2

Can you post the configuration and the log? Do you start a single instance?

On Sat, May 22, 2010 at 1:18 AM, John Chang jchangkihtest2@gmail.comwrote:

I was using 0.6.0 successfully. I wanted to upgrade to 0.7.1.

I unzipped 0.7.1 (different location on machine) and copied over
elasticsearch.yml from the 0.6.0 installation.

I stopped the 0.6.0 instance and started the 0.7.1 instance.

I can add indices as expected (as described here:

http://www.elasticsearch.com/docs/elasticsearch/rest_api/admin/indices/create_index/
).

When I do a _status call, though, it shows no indices. I go back and try
to
add the index again, and it complains that the index already exists. Yet,
_status always shows no indices.

When I try to query against the index (as described here:
http://www.elasticsearch.com/docs/elasticsearch/rest_api/search/), it just
hangs. When I try to query against an index I know to be non-existent, it
complains as expected that the index is not there.

View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/status-won-t-show-indices-after-0-6-0-to-0-7-1-upgrade-tp835467p835467.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.


(John Chang) #3

It is a cluster of 5, with one being a no-data node.

The logs are attached. log1-4.txt for the regular nodes, log_nodata.txt for the node that does not hold data.
<nabble_a href="log1.txt">log1.txt</nabble_a>
<nabble_a href="log2.txt">log2.txt</nabble_a>
<nabble_a href="log3.txt">log3.txt</nabble_a>
<nabble_a href="log4.txt">log4.txt</nabble_a>
<nabble_a href="elasticsearch_no_data.yml">elasticsearch_no_data.yml</nabble_a>

The configs are attached. elasticsearch_no_data.yml for the node that does not hold data (although when I used it it is simply clalled elasticsearch.yml). For the other 4 data nodes, see the attached elasticsearch.yml.
<nabble_a href="elasticsearch_no_data.yml">elasticsearch_no_data.yml</nabble_a>
<nabble_a href="elasticsearch.yml">elasticsearch.yml</nabble_a>

Note for the non-data node, I do the following before starting it:
export ES_JAVA_OPTS="-Djgroups.bind_port=9700 -Djgroups.tcpping.initial_hosts=10.215.221.123[9700],10.241.111.48[9700],10.209.185.130[9700],10.241.57.188[9700],10.254.110.127[9700]"

The ip's here do correspond to the machine names found in rest_commands.txt
<nabble_a href="rest_commands.txt">rest_commands.txt</nabble_a>

This file shows you what I do after startup. I define the index, I set the mappings, and then try to add a document. When I try to add the document, it hangs for a minute and then I get: {"error":"PrimaryNotStartedActionException[[testindex0][2] Timeout waiting for [1m]]"}. Also, when do a _status against the all the machines, machines (http://server:port/_status), it shows it all as empty, even though trying to re-add the same index will cause an error that the index already exists:
{
* ok: true
*
-
_shards: {
o total: 0
o successful: 0
o failed: 0
}
* indices: { }
}

Thanks for your help.


(Shay Banon) #4

I see that you use jgroups to configure it unicast discovery. JGroups is not
longer the default discovery, there is a new discovery called zen, and there
is a different way to configured unicast discovery explained here:
http://www.elasticsearch.com/docs/elasticsearch/modules/discovery/zen/.
Basically, you should configure in the configuration file:
discovery.zen.ping.unicast.hosts and set it to a list of hosts.

Also, in your configuration file, if you want to enable mmap, you need to
set index.store.type to mmapfs.

Could not open all the files you attached, but it seems like this is the
problem. It is very easy to in the logs by seeing that nodes do not
discovery each other.

cheers,
shay.banon

On Sat, May 22, 2010 at 2:53 AM, John Chang jchangkihtest2@gmail.comwrote:

It is a cluster of 5, with one being a no-data node.

The logs are attached. log1-4.txt for the regular nodes, log_nodata.txt
for
the node that does not hold data.
http://n3.nabble.com/file/n835646/log1.txt log1.txt
http://n3.nabble.com/file/n835646/log2.txt log2.txt
http://n3.nabble.com/file/n835646/log3.txt log3.txt
http://n3.nabble.com/file/n835646/log4.txt log4.txt
http://n3.nabble.com/file/n835646/elasticsearch_no_data.yml
elasticsearch_no_data.yml

The configs are attached. elasticsearch_no_data.yml for the node that does
not hold data (although when I used it it is simply clalled
elasticsearch.yml). For the other 4 data nodes, see the attached
elasticsearch.yml.
http://n3.nabble.com/file/n835646/elasticsearch_no_data.yml
elasticsearch_no_data.yml
http://n3.nabble.com/file/n835646/elasticsearch.yml elasticsearch.yml

Note for the non-data node, I do the following before starting it:
export ES_JAVA_OPTS="-Djgroups.bind_port=9700

-Djgroups.tcpping.initial_hosts=10.215.221.123[9700],10.241.111.48[9700],10.209.185.130[9700],10.241.57.188[9700],10.254.110.127[9700]"

The ip's here do correspond to the machine names found in rest_commands.txt
http://n3.nabble.com/file/n835646/rest_commands.txt rest_commands.txt

This file shows you what I do after startup. I define the index, I set the
mappings, and then try to add a document. When I try to add the document,
it hangs for a minute and then I get:
{"error":"PrimaryNotStartedActionException[[testindex0][2] Timeout waiting
for [1m]]"}. Also, when do a _status against the all the machines,
machines
(http://server:port/_status), it shows it all as empty, even though trying
to re-add the same index will cause an error that the index already exists:
{

  • ok: true
-
 _shards: {
     o total: 0
     o successful: 0
     o failed: 0
 }
  • indices: { }
    }

Thanks for your help.

View this message in context:
http://elasticsearch-users.115913.n3.nabble.com/status-won-t-show-indices-after-0-6-0-to-0-7-1-upgrade-tp835467p835646.html
Sent from the ElasticSearch Users mailing list archive at Nabble.com.


(John Chang) #5

Ok, I'm trying w/ zen, and I am still having trouble getting the nodes to connect. Below is my config. Note that I added index.store.type: mmapfs. Below that is log output. All the nodes say they are a "New Master" if I understand the log below correctly. The _status call still shows an empty cluster w/no other nodes and no indices.

gateway:
type: fs
fs:
location: /mnt/elasticsearch/gateway
index:
store:
type: mmapfs
fs:
mmapfs:
enabled: true
merge:
policy:
merge_factor: 20
path:
work: /mnt/elasticsearch/work
logs: /mnt/elasticsearch/logs
discovery:
zen:
ping:
unicast:
hosts: 10.215.221.123:9700,10.241.111.48:9700,10.209.185.130:9700,10.241.57.188:9700,10.254.110.127:9700

[22:25:04,438][INFO ][node ] [Rankin, Calvin] {ElasticSearch/0.7.1}[22808]: Initializing ...
[22:25:04,464][INFO ][plugins ] [Rankin, Calvin] Loaded []
[22:25:05,667][INFO ][node ] [Rankin, Calvin] {ElasticSearch/0.7.1}[22808]: Initialized
[22:25:05,668][INFO ][node ] [Rankin, Calvin] {ElasticSearch/0.7.1}[22808]: Starting ...
[22:25:05,809][INFO ][transport ] [Rankin, Calvin] bound_address[inet[/0:0:0:0:0:0:0:0:9300]], publish_address[inet[domU-12-31-39-00-69-71/10.254.110.127:9300]]
[22:25:10,452][INFO ][cluster.service ] [Rankin, Calvin] New Master [Rankin, Calvin][8c581aea-9f3e-4f54-b31e-3d9203476c9d][inet[domU-12-31-39-00-69-71/10.254.110.127:9300]]
[22:25:10,469][INFO ][discovery ] [Rankin, Calvin] elasticsearch/8c581aea-9f3e-4f54-b31e-3d9203476c9d
[22:25:10,473][INFO ][http ] [Rankin, Calvin] bound_address[inet[/0:0:0:0:0:0:0:0:9200]], publish_address[inet[domU-12-31-39-00-69-71/10.254.110.127:9200]]
[22:25:10,779][INFO ][jmx ] [Rankin, Calvin] bound_address[service:jmx:rmi:///jndi/rmi://:9400/jmxrmi], publish_address[service:jmx:rmi:///jndi/rmi://10.254.110.127:9400/jmxrmi]
[22:25:10,779][INFO ][node ] [Rankin, Calvin] {ElasticSearch/0.7.1}[22808]: Started


(John Chang) #6

I figured it out. I needed to change the ports from 9700 to 9300 when I changed from the transport client to zen. Thanks.


(system) #7