Saving Elasticsearch indices to drive spare drive so root drive willl not be full


(Ravi) #1

Hi,
I have recently setup a ElasticSearch server which is running on Centos 7.2
which is dell 720 with 2 root drive configured in raid 1 mirrored
4tb x 12 configured in Raid with 45Tb which is /dev/sdb1 44T 31M 42T 1% /data/0
[root@elkhost]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/centos-root 50G 17G 34G 34% /
devtmpfs 95G 4.0K 95G 1% /dev
tmpfs 95G 0 95G 0% /dev/shm
tmpfs 95G 17M 95G 1% /run
tmpfs 95G 0 95G 0% /sys/fs/cgroup
/dev/sda1 497M 165M 332M 34% /boot
/dev/mapper/centos-home 82G 33M 82G 1% /home
/dev/sdb1 44T 31M 42T 1% /data/0
tmpfs 19G 0 19G 0% /run/user/0

elasticsearch.yml

----------------------------------- Paths ------------------------------------

Path to directory where to store the data (separate multiple locations by comma):

path:
logs: /path/data/0/log/
data: /path/data/0

I want to save data in /data/0 which is 44 TB so root drive will not get full with data, Any input will be helpful

[root@elkhost indices]# pwd
/var/lib/elasticsearch/elknode/nodes/0/indices
[root@elkhost indices]#

Thanks in Advance


(Mark Walkom) #2

Is the path.data change not working?


(Ravi) #3

Yes if

[root@elasticsearch 0]# service elasticsearch status
● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Fri 2016-09-23 17:14:20 EDT; 2min 44s ago
Docs: http://www.elastic.co
Process: 17205 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -Des.pidfile=${PID_DIR}/elasticsearch.pid -Des.default.path.home=${ES_HOME} -Des.default.path.logs=${LOG_DIR} -Des.default.path.data=${DATA_DIR} -Des.default.path.conf=${CONF_DIR} (code=exited, status=1/FAILURE)
Process: 17202 ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec (code=exited, status=0/SUCCESS)
Main PID: 17205 (code=exited, status=1/FAILURE)

Sep 23 17:14:20 elasticsearch[17205]: at org.elasticsearch.bootstrap.Security.createPermissions(Security.java:212)
Sep 23 17:14:20 elasticsearch[17205]: at org.elasticsearch.bootstrap.Security.configure(Security.java:118)
Sep 23 17:14:20 elasticsearch[17205]: at org.elasticsearch.bootstrap.Bootstrap.setupSecurity(Bootstrap.java:212)
Sep 23 17:14:20 elasticsearch[17205]: at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:183)
Sep 23 17:14:20 elasticsearch[17205]: at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:286)
Sep 23 17:14:20 elasticsearch[17205]: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
Sep 23 17:14:20 elasticsearch[17205]: Refer to the log for complete error details.
Sep 23 17:14:20 systemd[1]: elasticsearch.service: main process exited, code=exited, status=1/FAILURE
Sep 23 17:14:20 systemd[1]: Unit elasticsearch.service entered failed state.
Sep 23 17:14:20 systemd[1]: elasticsearch.service failed.


(Ravi) #4

Currently i changed the path of path.data in elasticsearch.yml
still logs been saving in root drive which is
[root@elkhost]# du -sh
5.1G .
[root@elkhost]# pwd
/var/lib/elasticsearch/qdnelklab/nodes
[root@elkhost nodes]#
path:
logs: /path/data/0/log/
data: /path/data/0

[root@elkhost ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/centos-root 50G 12G 39G 24% /
devtmpfs 95G 4.0K 95G 1% /dev
tmpfs 95G 0 95G 0% /dev/shm
tmpfs 95G 17M 95G 1% /run
tmpfs 95G 0 95G 0% /sys/fs/cgroup
/dev/sda1 497M 165M 332M 34% /boot
/dev/mapper/centos-home 82G 33M 82G 1% /home
/dev/sdb1 44T 34M 42T 1% /data/0
tmpfs 19G 0 19G 0% /run/user/0
[root@elkhost ~]# cd /var/log/elasticsearch/q
qdnelklab_deprecation.log qdnelklab_index_search_slowlog.log qndelklab_deprecation.log qndelklab_index_search_slowlog.log
qdnelklab_index_indexing_slowlog.log qdnelklab.log qndelklab_index_indexing_slowlog.log qndelklab.log
[root@elkhost ~]# cd /var/lib/
alternatives/ dbus/ docker/ initramfs/ logstash/ NetworkManager/ polkit-1/ rpm-state/ sfcb/ tpm/
authconfig/ dhclient/ elasticsearch/ kibana/ machines/ os-prober/ postfix/ rsyslog/ stateless/ yum/
dav/ dnsmasq/ games/ logrotate.status misc/ plymouth/ rpm/ selinux/ systemd/
[root@elkhost ~]# cd /var/lib/elasticsearch/
elasticsearch/ elklab/ qdnelklab/ qndelklab/ qndlklab/
[root@~]# cd /var/lib/elasticsearch/qdnelklab/
[root@]# ls
nodes
[root@]# cd nodes/
[root@]# du -sh
5.1G .
[root@]# pwd
/var/lib/elasticsearch/qdnelklab/nodes
[root@]#


(Ravi) #5

[root@hostelasticsearch]# pwd
/var/lib/elasticsearch

followed steps in the link but still no luck logs still saving in above mentioned path


(Mark Walkom) #6

Looks like ES doesn't have permissions to the mount?


(system) #7