I noticed a strange behavior while setting up my ES 1.4.4 cluster: the
transactions logs seem to be stored at the wrong place.
I start ES using the option --default.path.data=/var/lib/elasticsearch
(default setting when using deb packages).
Then, in /etc/elasticsearch/elasticsearch.yml, I change the data path for 4
SSD :
When I create an index and start pushing data, the shards are correctly
created on my SSD
(/data/self//clustername/nodes/0/indices/indexname//index/) but the
transaction logs are created in
/var/lib/elasticsearch/clustername/nodes/0/indices/indexname//translog/translog-*.
It seems like the config file did correctly override the data path setting
for the shards but not for the transactions logs.
I noticed a strange behavior while setting up my ES 1.4.4 cluster: the
transactions logs seem to be stored at the wrong place.
I start ES using the option --default.path.data=/var/lib/elasticsearch
(default setting when using deb packages).
Then, in /etc/elasticsearch/elasticsearch.yml, I change the data path for
4 SSD :
When I create an index and start pushing data, the shards are correctly
created on my SSD
(/data/self//clustername/nodes/0/indices/indexname//index/) but the
transaction logs are created in
/var/lib/elasticsearch/clustername/nodes/0/indices/indexname//translog/translog-*.
It seems like the config file did correctly override the data path setting
for the shards but not for the transactions logs.
Well, my /var/lib partition is small while I have 4 SSD with plenty of
space that I use to store indices.
So I have ran several times into a "no more space on device" situation when
a developer decided to increase index.translog.flush_threshold_size to
speed up indexing. And then I could not even flush the index because there
was not enough space to create new empty translogs.
I managed to move the translogs to the path.data configured in
elasticsearch.yml by removing the --default.path.data option from the init
script. However this is not a good solution as it involves altering a
packaged file.
It would be nice to either store translogs correctly with the corresponding
indices or have a setting to change the storage path.
Regards,
Hervé BRY
System Administrator
Le dimanche 1 mars 2015 09:15:08 UTC+1, Mark Walkom a écrit :
Well the translog isn't really data like an index is, however you raise a
valid point if this is the behaviour.
Is it a problem for you to have them there or just a point of interest?
On 27 February 2015 at 01:58, Hervé Bry <herv...@geneanet.org
<javascript:>> wrote:
Hello Community,
I noticed a strange behavior while setting up my ES 1.4.4 cluster: the
transactions logs seem to be stored at the wrong place.
I start ES using the option --default.path.data=/var/lib/elasticsearch
(default setting when using deb packages).
Then, in /etc/elasticsearch/elasticsearch.yml, I change the data path for
4 SSD :
When I create an index and start pushing data, the shards are correctly
created on my SSD
(/data/self//clustername/nodes/0/indices/indexname//index/) but the
transaction logs are created in
/var/lib/elasticsearch/clustername/nodes/0/indices/indexname//translog/translog-*.
It seems like the config file did correctly override the data path
setting for the shards but not for the transactions logs.
Well, my /var/lib partition is small while I have 4 SSD with plenty of
space that I use to store indices.
So I have ran several times into a "no more space on device" situation
when a developer decided to increase index.translog.flush_threshold_size to
speed up indexing. And then I could not even flush the index because there
was not enough space to create new empty translogs.
I managed to move the translogs to the path.data configured in
elasticsearch.yml by removing the --default.path.data option from the init
script. However this is not a good solution as it involves altering a
packaged file.
It would be nice to either store translogs correctly with the
corresponding indices or have a setting to change the storage path.
I noticed a strange behavior while setting up my ES 1.4.4 cluster: the
transactions logs seem to be stored at the wrong place.
I start ES using the option --default.path.data=/var/lib/elasticsearch
(default setting when using deb packages).
Then, in /etc/elasticsearch/elasticsearch.yml, I change the data path
for 4 SSD :
When I create an index and start pushing data, the shards are correctly
created on my SSD (/data/self//clustername/nodes/0/indices/indexname//index/)
but the transaction logs are created in /var/lib/elasticsearch/
clustername/nodes/0/indices/indexname//translog/translog-*.
It seems like the config file did correctly override the data path
setting for the shards but not for the transactions logs.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.