Understanding local gateway


(Ahmet Arslan) #1

Hello,

I am using version 0.17.1.

I activated local gateway in elasticsearch.yml file as follows:

Gateway Settings

gateway:
recover_after_nodes: 1
recover_after_time: 5m
expected_nodes: 2

I read somewhere that file system location (used by local gateway) is the
work location by default.
So I was expecting work directory is created under $ES_HOME. But it didn't.
I thought that some files (used to recover cluster state etc ) will be
there.

Am I missing something here?

Where I can find more information about local gateway?

How do I make sure ES gateway is working correctly?

Thanks,
Ahmet


(Shay Banon) #2

Each node stores the data under the data directory. By default, the local
gateway is used, even when you don't enable the below settings.

On Sat, Jul 30, 2011 at 12:50 AM, iorixxx@yahoo.com iorixxx@yahoo.comwrote:

Hello,

I am using version 0.17.1.

I activated local gateway in elasticsearch.yml file as follows:

Gateway Settings

gateway:
recover_after_nodes: 1
recover_after_time: 5m
expected_nodes: 2

I read somewhere that file system location (used by local gateway) is the
work location by default.
So I was expecting work directory is created under $ES_HOME. But it
didn't.
I thought that some files (used to recover cluster state etc ) will be
there.

Am I missing something here?

Where I can find more information about local gateway?

How do I make sure ES gateway is working correctly?

Thanks,
Ahmet


(Otis Gospodnetić) #3

Hi,

I think Ahmet is referring to the "work" directory mentioned on
http://www.elasticsearch.org/guide/reference/setup/dir-layout.html and
wondering when that dir is created and what exactly is in it.

Otis

Sematext is hiring search engineers -- http://sematext.com/about/jobs.html

On Jul 30, 7:45 am, Shay Banon kim...@gmail.com wrote:

Each node stores the data under the data directory. By default, the local
gateway is used, even when you don't enable the below settings.

On Sat, Jul 30, 2011 at 12:50 AM, iori...@yahoo.com iori...@yahoo.comwrote:

Hello,

I am using version 0.17.1.

I activated local gateway in elasticsearch.yml file as follows:

Gateway Settings

gateway:
recover_after_nodes: 1
recover_after_time: 5m
expected_nodes: 2

I read somewhere that file system location (used by local gateway) is the
work location by default.
So I was expecting work directory is created under $ES_HOME. But it
didn't.
I thought that some files (used to recover cluster state etc ) will be
there.

Am I missing something here?

Where I can find more information about local gateway?

How do I make sure ES gateway is working correctly?

Thanks,
Ahmet


(Shay Banon) #4

That page explains that data is where data is stored. work is used for what
is explained there, its the "tmp" of elasticsearch. Its not used much.

Though, a lot of versions ago, the data used to be stored under work, but
that has changed to data, maybe thats where the confusion is coming from.

On Sun, Jul 31, 2011 at 1:30 PM, Otis Gospodnetic <
otis.gospodnetic@gmail.com> wrote:

Hi,

I think Ahmet is referring to the "work" directory mentioned on
http://www.elasticsearch.org/guide/reference/setup/dir-layout.html and
wondering when that dir is created and what exactly is in it.

Otis

Sematext is hiring search engineers -- http://sematext.com/about/jobs.html

On Jul 30, 7:45 am, Shay Banon kim...@gmail.com wrote:

Each node stores the data under the data directory. By default, the
local
gateway is used, even when you don't enable the below settings.

On Sat, Jul 30, 2011 at 12:50 AM, iori...@yahoo.com <iori...@yahoo.com
wrote:

Hello,

I am using version 0.17.1.

I activated local gateway in elasticsearch.yml file as follows:

Gateway Settings

gateway:
recover_after_nodes: 1
recover_after_time: 5m
expected_nodes: 2

I read somewhere that file system location (used by local gateway) is
the

work location by default.
So I was expecting work directory is created under $ES_HOME. But it
didn't.
I thought that some files (used to recover cluster state etc ) will be
there.

Am I missing something here?

Where I can find more information about local gateway?

How do I make sure ES gateway is working correctly?

Thanks,
Ahmet


(Ahmet Arslan) #5

Hi Shay,

I was trying to figure out what is stored by local gateway. I read one of your messages saying that work is used but it turned out to be this is the case in earlier versions.

Confusion about work directory is resolved. But I still don't understand how local gateway works.

Isn't data directory considered as 'transient'? How local disk (data directory) can be used for long term persistence storage?

Thanks,
Ahmet

--- On Sun, 7/31/11, Shay Banon kimchy@gmail.com wrote:

From: Shay Banon kimchy@gmail.com
Subject: Re: understanding local gateway
To: elasticsearch@googlegroups.com
Date: Sunday, July 31, 2011, 1:43 PM

That page explains that data is where data is stored. work is used for what is explained there, its the "tmp" of elasticsearch. Its not used much.
Though, a lot of versions ago, the data used to be stored under work, but that has changed to data, maybe thats where the confusion is coming from.

On Sun, Jul 31, 2011 at 1:30 PM, Otis Gospodnetic otis.gospodnetic@gmail.com wrote:

Hi,

I think Ahmet is referring to the "work" directory mentioned on

http://www.elasticsearch.org/guide/reference/setup/dir-layout.html and

wondering when that dir is created and what exactly is in it.

Otis

--

Sematext is hiring search engineers -- http://sematext.com/about/jobs.html

On Jul 30, 7:45 am, Shay Banon kim...@gmail.com wrote:

Each node stores the data under the data directory. By default, the local

gateway is used, even when you don't enable the below settings.

On Sat, Jul 30, 2011 at 12:50 AM, iori...@yahoo.com iori...@yahoo.comwrote:

Hello,

I am using version 0.17.1.

I activated local gateway in elasticsearch.yml file as follows:

Gateway Settings

gateway:

recover_after_nodes: 1
recover_after_time: 5m
expected_nodes: 2

I read somewhere that file system location (used by local gateway) is the

work location by default.

So I was expecting work directory is created under $ES_HOME. But it

didn't.

I thought that some files (used to recover cluster state etc ) will be

there.

Am I missing something here?

Where I can find more information about local gateway?

How do I make sure ES gateway is working correctly?

Thanks,

Ahmet


(Shay Banon) #6

local data directory is not considered transient when using the local
gateway. What exactly are you missing from this explanation:
http://www.elasticsearch.org/guide/reference/modules/gateway/local.html? The
low level details on how it works?

On Mon, Aug 1, 2011 at 11:14 AM, Ahmet Arslan iorixxx@yahoo.com wrote:

Hi Shay,

I was trying to figure out what is stored by local gateway. I read one of
your messages saying that work is used but it turned out to be this is the
case in earlier versions.

Confusion about work directory is resolved. But I still don't understand
how local gateway works.

Isn't data directory considered as 'transient'? How local disk (data
directory) can be used for long term persistence storage?

Thanks,
Ahmet

--- On Sun, 7/31/11, Shay Banon kimchy@gmail.com wrote:

From: Shay Banon kimchy@gmail.com
Subject: Re: understanding local gateway
To: elasticsearch@googlegroups.com
Date: Sunday, July 31, 2011, 1:43 PM

That page explains that data is where data is stored. work is used for what
is explained there, its the "tmp" of elasticsearch. Its not used much.

Though, a lot of versions ago, the data used to be stored under work, but
that has changed to data, maybe thats where the confusion is coming from.

On Sun, Jul 31, 2011 at 1:30 PM, Otis Gospodnetic <
otis.gospodnetic@gmail.comhttp://mc/compose?to=otis.gospodnetic@gmail.com

wrote:

Hi,

I think Ahmet is referring to the "work" directory mentioned on
http://www.elasticsearch.org/guide/reference/setup/dir-layout.html and
wondering when that dir is created and what exactly is in it.

Otis

Sematext is hiring search engineers -- http://sematext.com/about/jobs.html

On Jul 30, 7:45 am, Shay Banon <kim...@gmail.comhttp://mc/compose?to=kim...@gmail.com>
wrote:

Each node stores the data under the data directory. By default, the
local
gateway is used, even when you don't enable the below settings.

On Sat, Jul 30, 2011 at 12:50 AM, iori...@yahoo.comhttp://mc/compose?to=iori...@yahoo.com<
iori...@yahoo.com http://mc/compose?to=iori...@yahoo.com>wrote:

Hello,

I am using version 0.17.1.

I activated local gateway in elasticsearch.yml file as follows:

Gateway Settings

gateway:
recover_after_nodes: 1
recover_after_time: 5m
expected_nodes: 2

I read somewhere that file system location (used by local gateway) is
the

work location by default.
So I was expecting work directory is created under $ES_HOME. But it
didn't.
I thought that some files (used to recover cluster state etc ) will be
there.

Am I missing something here?

Where I can find more information about local gateway?

How do I make sure ES gateway is working correctly?

Thanks,
Ahmet


(system) #7