How can i move index files to another path without stop the es service


(meilian zhu) #1

my local disk is very small,just can store one day index files.So i wanted
to move my index file to another fs path.
This is my sh.It excute everyday.
#!/bin/bash
#copy the index files form elasticsearch to clusterFS now
ddate=date -d "1 days ago" +%Y.%m.%d
clustername="mams_online_log"
indexdir="/data/soft/logmonitor/elasticsearch-1.1.0/data"
ipip=$(hostname)
dstdir=/mnt/lego-log/$ipip
echo "Start time: $(date "+%Y.%m.%d %H:%M:%S")"
echo "$dstdir"

copy the old index file;

mkdir $dstdir
cp -r $indexdir/$clustername/nodes/0/indices/logstash-$ddate $dstdir/
mv $indexdir/$clustername/nodes/0/indices/logstash-$ddate
$indexdir/$clustername/nodes/0/indices/tmp
ln -s $dstdir/logstash-$ddate
$indexdir/$clustername/nodes/0/indices/logstash-$ddate
rm -rf $indexdir/$clustername/nodes/0/indices/tmp
echo "ok"
echo "End time: $(date "+%Y.%m.%d %H:%M:%S")"

But when i "lsof" this is always so many files used by the es pid.
So how i can do for this ?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ca99a71c-3524-4d8d-9c68-a21337aba62c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Michael Salmon) #2

Basically you can't do with no stop at all. ES holds indexes open so that
it can access the data quickly. You could close the index, copy the data
and then open the index again but it is probably best to stop es, move the
data, change the data path and then start es again. You can have multiple
data paths so that you can skip the copying but you need to restart es.

/Michael

On Monday, 28 April 2014 11:44:27 UTC+2, meilian zhu wrote:

my local disk is very small,just can store one day index files.So i wanted
to move my index file to another fs path.
This is my sh.It excute everyday.
#!/bin/bash
#copy the index files form elasticsearch to clusterFS now
ddate=date -d "1 days ago" +%Y.%m.%d
clustername="mams_online_log"
indexdir="/data/soft/logmonitor/elasticsearch-1.1.0/data"
ipip=$(hostname)
dstdir=/mnt/lego-log/$ipip
echo "Start time: $(date "+%Y.%m.%d %H:%M:%S")"
echo "$dstdir"

copy the old index file;

mkdir $dstdir
cp -r $indexdir/$clustername/nodes/0/indices/logstash-$ddate $dstdir/
mv $indexdir/$clustername/nodes/0/indices/logstash-$ddate
$indexdir/$clustername/nodes/0/indices/tmp
ln -s $dstdir/logstash-$ddate
$indexdir/$clustername/nodes/0/indices/logstash-$ddate
rm -rf $indexdir/$clustername/nodes/0/indices/tmp
echo "ok"
echo "End time: $(date "+%Y.%m.%d %H:%M:%S")"

But when i "lsof" this is always so many files used by the es pid.
So how i can do for this ?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/0a0464a5-0fad-4522-8c86-8d96f2311339%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(Kyle Lin) #3

How about closing index before moving files? Does it fit your need?

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-open-close.html

meilian zhu於 2014年4月28日星期一UTC+8下午5時44分27秒寫道:

my local disk is very small,just can store one day index files.So i wanted
to move my index file to another fs path.
This is my sh.It excute everyday.
#!/bin/bash
#copy the index files form elasticsearch to clusterFS now
ddate=date -d "1 days ago" +%Y.%m.%d
clustername="mams_online_log"
indexdir="/data/soft/logmonitor/elasticsearch-1.1.0/data"
ipip=$(hostname)
dstdir=/mnt/lego-log/$ipip
echo "Start time: $(date "+%Y.%m.%d %H:%M:%S")"
echo "$dstdir"

copy the old index file;

mkdir $dstdir
cp -r $indexdir/$clustername/nodes/0/indices/logstash-$ddate $dstdir/
mv $indexdir/$clustername/nodes/0/indices/logstash-$ddate
$indexdir/$clustername/nodes/0/indices/tmp
ln -s $dstdir/logstash-$ddate
$indexdir/$clustername/nodes/0/indices/logstash-$ddate
rm -rf $indexdir/$clustername/nodes/0/indices/tmp
echo "ok"
echo "End time: $(date "+%Y.%m.%d %H:%M:%S")"

But when i "lsof" this is always so many files used by the es pid.
So how i can do for this ?

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/2590b704-6e43-4166-bbbc-6ec36b29adca%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


(system) #4