I ran a "single-node" ES setup, with the gateway config on my ES
server (eventually it may be replaced by a cluster of boxes with a
shared file system) as:
and it works, I start it up, set up a few indices, (corresponding data
and metadata appeared in the file location specified) and everything
is available.
Then on another machine - I'll call it the ES Client - I use the Java
API (setting the node on the application to be a non-data client node
without any gateway setting).
I noticed that, subsequently messing with the machines:
If the ES Server node starts up first, and ES Client joins its cluster
every thing is fine, indices are available....
If the ES Client node starts up first, and the ES Server node
subsequently starts and join its cluster - indices are not available -
as if they were not created, guessing the metadata isn't read from the
gateway.
I cannot guarantee order of startup if there's a network/power outage.
I want the index data/metadata to live just on the ES Server, so the
ES Client shouldn't need access to gateway store at /home/escluster
(its just a client and not a data node).
Does anyone has an idea how to get around this or where my
configuration is going wrong?
Can you try and use 0.14.2, see if it helps? Also, note that you get long
term persistency with the default local gateway as well. So make sure you
want to use the fs gateway.
I ran a "single-node" ES setup, with the gateway config on my ES
server (eventually it may be replaced by a cluster of boxes with a
shared file system) as:
and it works, I start it up, set up a few indices, (corresponding data
and metadata appeared in the file location specified) and everything
is available.
Then on another machine - I'll call it the ES Client - I use the Java
API (setting the node on the application to be a non-data client node
without any gateway setting).
I noticed that, subsequently messing with the machines:
If the ES Server node starts up first, and ES Client joins its cluster
every thing is fine, indices are available....
If the ES Client node starts up first, and the ES Server node
subsequently starts and join its cluster - indices are not available -
as if they were not created, guessing the metadata isn't read from the
gateway.
I cannot guarantee order of startup if there's a network/power outage.
I want the index data/metadata to live just on the ES Server, so the
ES Client shouldn't need access to gateway store at /home/escluster
(its just a client and not a data node).
Does anyone has an idea how to get around this or where my
configuration is going wrong?
In 0.14.2 the metadata is now being saved on the client non-data node
too. However ClusterHealthRequest for a specific index seems to be not
working as expected now....
I run with an index named "test".
When my data node (ES Client) starts up - I perform a
ClusterHealthRequest("test") regularly (background thread) to verify
if it is connected to a cluster (i.e. ES Server) that can access that
index.
in 0.13.1, if its not connected - check returns RED or index not found
exception - which was great :).
in 0.14.2, if its not connected - it returns GREEN initially
consistently even if the index is not there yet - breaking the check.
Subsequently I tried indexing something. It then created the index
(metadata appears in data directory on the client node), then threw an
exception ( failed to index - expected because there is no data node)
Can you try and use 0.14.2, see if it helps? Also, note that you get long
term persistency with the default local gateway as well. So make sure you
want to use the fs gateway.
I ran a "single-node" ES setup, with the gateway config on my ES
server (eventually it may be replaced by a cluster of boxes with a
shared file system) as:
and it works, I start it up, set up a few indices, (corresponding data
and metadata appeared in the file location specified) and everything
is available.
Then on another machine - I'll call it the ES Client - I use the Java
API (setting the node on the application to be a non-data client node
without any gateway setting).
I noticed that, subsequently messing with the machines:
If the ES Server node starts up first, and ES Client joins its cluster
every thing is fine, indices are available....
If the ES Client node starts up first, and the ES Server node
subsequently starts and join its cluster - indices are not available -
as if they were not created, guessing the metadata isn't read from the
gateway.
I cannot guarantee order of startup if there's a network/power outage.
I want the index data/metadata to live just on the ES Server, so the
ES Client shouldn't need access to gateway store at /home/escluster
(its just a client and not a data node).
Does anyone has an idea how to get around this or where my
configuration is going wrong?
When my data node (ES Client) starts up - I perform a
ClusterHealthRequest("test") regularly (background thread) to verify
if it is connected to a cluster (i.e. ES Server) that can access that
index.
I meant:
When my client node (ES Client) starts up - I perform a
ClusterHealthRequest("test") regularly (background thread) to verify
if it is connected to a cluster (i.e. ES Server) that can access that
index.
In 0.14.2 the metadata is now being saved on the client non-data node
too. However ClusterHealthRequest for a specific index seems to be not
working as expected now....
I run with an index named "test".
When my data node (ES Client) starts up - I perform a
ClusterHealthRequest("test") regularly (background thread) to verify
if it is connected to a cluster (i.e. ES Server) that can access that
index.
in 0.13.1, if its not connected - check returns RED or index not found
exception - which was great :).
in 0.14.2, if its not connected - it returns GREEN initially
consistently even if the index is not there yet - breaking the check.
Subsequently I tried indexing something. It then created the index
(metadata appears in data directory on the client node), then threw an
exception ( failed to index - expected because there is no data node)
Can you try and use 0.14.2, see if it helps? Also, note that you get long
term persistency with the default local gateway as well. So make sure you
want to use the fs gateway.
I ran a "single-node" ES setup, with the gateway config on my ES
server (eventually it may be replaced by a cluster of boxes with a
shared file system) as:
and it works, I start it up, set up a few indices, (corresponding data
and metadata appeared in the file location specified) and everything
is available.
Then on another machine - I'll call it the ES Client - I use the Java
API (setting the node on the application to be a non-data client node
without any gateway setting).
I noticed that, subsequently messing with the machines:
If the ES Server node starts up first, and ES Client joins its cluster
every thing is fine, indices are available....
If the ES Client node starts up first, and the ES Server node
subsequently starts and join its cluster - indices are not available -
as if they were not created, guessing the metadata isn't read from the
gateway.
I cannot guarantee order of startup if there's a network/power outage.
I want the index data/metadata to live just on the ES Server, so the
ES Client shouldn't need access to gateway store at /home/escluster
(its just a client and not a data node).
Does anyone has an idea how to get around this or where my
configuration is going wrong?
When my data node (ES Client) starts up - I perform a
ClusterHealthRequest("test") regularly (background thread) to verify
if it is connected to a cluster (i.e. ES Server) that can access that
index.
I meant, when my client node (ES Client) starts up. sorry for the
typo.
In 0.14.2 the metadata is now being saved on the client non-data node
too. However ClusterHealthRequest for a specific index seems to be not
working as expected now....
I run with an index named "test".
When my data node (ES Client) starts up - I perform a
ClusterHealthRequest("test") regularly (background thread) to verify
if it is connected to a cluster (i.e. ES Server) that can access that
index.
in 0.13.1, if its not connected - check returns RED or index not found
exception - which was great :).
in 0.14.2, if its not connected - it returns GREEN initially
consistently even if the index is not there yet - breaking the check.
Subsequently I tried indexing something. It then created the index
(metadata appears in data directory on the client node), then threw an
exception ( failed to index - expected because there is no data node)
Can you try and use 0.14.2, see if it helps? Also, note that you get long
term persistency with the default local gateway as well. So make sure you
want to use the fs gateway.
I ran a "single-node" ES setup, with the gateway config on my ES
server (eventually it may be replaced by a cluster of boxes with a
shared file system) as:
and it works, I start it up, set up a few indices, (corresponding data
and metadata appeared in the file location specified) and everything
is available.
Then on another machine - I'll call it the ES Client - I use the Java
API (setting the node on the application to be a non-data client node
without any gateway setting).
I noticed that, subsequently messing with the machines:
If the ES Server node starts up first, and ES Client joins its cluster
every thing is fine, indices are available....
If the ES Client node starts up first, and the ES Server node
subsequently starts and join its cluster - indices are not available -
as if they were not created, guessing the metadata isn't read from the
gateway.
I cannot guarantee order of startup if there's a network/power outage.
I want the index data/metadata to live just on the ES Server, so the
ES Client shouldn't need access to gateway store at /home/escluster
(its just a client and not a data node).
Does anyone has an idea how to get around this or where my
configuration is going wrong?
Here is a gist, running with 0.14.2, that creates first a client node, and
then a data node. The client node "data_client" location is never created.
Calling health on the client node before any other node has joined will
cause a MasterNotDiscoveredException (as expected). When adding the data
node, then its data location is created, and you can create an index. At no
point is there data created on the file system for the client node created.
How is it different than your test?
When my data node (ES Client) starts up - I perform a
ClusterHealthRequest("test") regularly (background thread) to verify
if it is connected to a cluster (i.e. ES Server) that can access that
index.
I meant, when my client node (ES Client) starts up. sorry for the
typo.
In 0.14.2 the metadata is now being saved on the client non-data node
too. However ClusterHealthRequest for a specific index seems to be not
working as expected now....
I run with an index named "test".
When my data node (ES Client) starts up - I perform a
ClusterHealthRequest("test") regularly (background thread) to verify
if it is connected to a cluster (i.e. ES Server) that can access that
index.
in 0.13.1, if its not connected - check returns RED or index not found
exception - which was great :).
in 0.14.2, if its not connected - it returns GREEN initially
consistently even if the index is not there yet - breaking the check.
Subsequently I tried indexing something. It then created the index
(metadata appears in data directory on the client node), then threw an
exception ( failed to index - expected because there is no data node)
Can you try and use 0.14.2, see if it helps? Also, note that you get
long
term persistency with the default local gateway as well. So make sure
you
want to use the fs gateway.
I ran a "single-node" ES setup, with the gateway config on my ES
server (eventually it may be replaced by a cluster of boxes with a
shared file system) as:
and it works, I start it up, set up a few indices, (corresponding
data
and metadata appeared in the file location specified) and everything
is available.
Then on another machine - I'll call it the ES Client - I use the Java
API (setting the node on the application to be a non-data client node
without any gateway setting).
I noticed that, subsequently messing with the machines:
If the ES Server node starts up first, and ES Client joins its
cluster
every thing is fine, indices are available....
If the ES Client node starts up first, and the ES Server node
subsequently starts and join its cluster - indices are not available
as if they were not created, guessing the metadata isn't read from
the
gateway.
I cannot guarantee order of startup if there's a network/power
outage.
I want the index data/metadata to live just on the ES Server, so the
ES Client shouldn't need access to gateway store at /home/escluster
(its just a client and not a data node).
Does anyone has an idea how to get around this or where my
configuration is going wrong?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
