I commented some lines but elasticsearch can't work


(Jason) #1

Hello.
I added some lines in "elasticsearch.yml" and commented them but elasticsearch can't run:

# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
# path.data: /path/to/data
# path.repo: ["/var/log/back/repo","/var/log/back-long/repo"]
#
# Path to log files:
path.logs: /var/log/elastic
# path.logs: /var/log/elasticsearch
#

When I did below command:

[root@localhost ~]# service elasticsearch status
● elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Mon 2017-10-02 17:36:52 +0330; 1min 14s ago
     Docs: http://www.elastic.co
  Process: 1343 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet -Edefault.path.logs=${LOG_DIR} -Edefault.path.data=${DATA_DIR} -Edefault.path.conf=${CONF_DIR} (code=exited, status=1/FAILURE)
  Process: 1338 ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec (code=exited, status=0/SUCCESS)
 Main PID: 1343 (code=exited, status=1/FAILURE)

Oct 02 17:36:52 localhost.localdomain elasticsearch[1343]: 2017-10-02 17:36:52,285 main ERROR Null object returned for Rol...ders.
Oct 02 17:36:52 localhost.localdomain elasticsearch[1343]: 2017-10-02 17:36:52,285 main ERROR Null object returned for Rol...ders.
Oct 02 17:36:52 localhost.localdomain elasticsearch[1343]: 2017-10-02 17:36:52,286 main ERROR Null object returned for Rol...ders.
Oct 02 17:36:52 localhost.localdomain elasticsearch[1343]: 2017-10-02 17:36:52,286 main ERROR Unable to locate appender "r...root"
Oct 02 17:36:52 localhost.localdomain elasticsearch[1343]: 2017-10-02 17:36:52,286 main ERROR Unable to locate appender "i...ndex"
Oct 02 17:36:52 localhost.localdomain elasticsearch[1343]: 2017-10-02 17:36:52,287 main ERROR Unable to locate appender "i...wlog"
Oct 02 17:36:52 localhost.localdomain elasticsearch[1343]: 2017-10-02 17:36:52,287 main ERROR Unable to locate appender "d...tion"
Oct 02 17:36:52 localhost.localdomain systemd[1]: elasticsearch.service: main process exited, code=exited, status=1/FAILURE
Oct 02 17:36:52 localhost.localdomain systemd[1]: Unit elasticsearch.service entered failed state.
Oct 02 17:36:52 localhost.localdomain systemd[1]: elasticsearch.service failed.
Hint: Some lines were ellipsized, use -l to show in full.

Log is:

# cat /var/log/elasticsearch/elasticsearch.log 
Caused by: java.nio.file.AccessDeniedException: /var/log/back/repo
	at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) ~[?:1.8.0_131]
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:1.8.0_131]
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:1.8.0_131]
	at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384) ~[?:1.8.0_131]
	at java.nio.file.Files.createDirectory(Files.java:674) ~[?:1.8.0_131]
	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) ~[?:1.8.0_131]
	at java.nio.file.Files.createDirectories(Files.java:767) ~[?:1.8.0_131]
	at org.elasticsearch.bootstrap.Security.ensureDirectoryExists(Security.java:455) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.bootstrap.Security.addPath(Security.java:411) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.bootstrap.Security.addFilePermissions(Security.java:296) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.bootstrap.Security.createPermissions(Security.java:224) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.bootstrap.Security.configure(Security.java:114) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:227) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:350) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) ~[elasticsearch-5.4.1.jar:5.4.1]
	... 6 more

As you see I commented /var/log/back/repo but not worked!!!

Any idea?

Thank you.


(David Pilato) #2

Are you sure you edited the right file?
May be you added somewhere some settings like -Epath.repo=/var/log/back/repo?


(Jason) #3

I edited "elasticsearch.yml" with nano editor and my config file is:

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
# path.data: /path/to/data
# path.repo: ["/var/log/back/repo","/var/log/back-long/repo"]
#
# Path to log files:
path.logs: /var/log/elastic
# path.logs: /var/log/elasticsearch
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
#discovery.zen.ping.unicast.hosts: ["host1", "host2"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
#
#discovery.zen.minimum_master_nodes: 3
#
# For more information, consult the zen discovery module documentation.
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ---------------------------------- Various -----------------------------------
#
# Require explicit names when deleting indices:
#

Any wrong?


(Snobear) #4

Even though you commented it out of elasticsearch.yml, the repository still persists in the cluster settings if you've created a repository at some point with that path. So if you want to change the storage path, you'll need to get ES running again, delete existing repos with the ES API, and restart with your new path.repo settings. Basically:

  1. Create /var/log/back/repo and set permissions so ES can read/write to it.
  2. Uncomment path.repo: and start Elasticsearch.
  3. Check if repos are configured with
curl -s localhost:9200/_snapshot/_all?pretty
  1. Delete the repos you no longer need (warning: data will be destroyed) with:
curl -XDELETE localhost:9200/_snapshot/myrepo

Then you can change the settings, restart ES, and add the new repo. I haven't tested the above commands, but thats the basic idea. Perhaps there is a way to update/change the path instead of deleting/recreating, but I'm not sure how to do that.


(Jason) #5

I did:

[root@localhost ~]# cd /var/log/back
[root@localhost back]# ls
repo
[root@localhost back]# ls -l
total 0
drwxr-xr-x. 3 elastic elastic 17 Oct  2 12:45 repo

My config is:

# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
# path.data: /path/to/data
path.repo: ["/var/log/back/repo"]
#
# Path to log files:
path.logs: /var/log/elastic
# path.logs: /var/log/elasticsearch
#

and

[root@localhost back]# curl -s localhost:9200/_snapshot/_all?pretty
[root@localhost back]#

and

[root@localhost back]# curl -XDELETE localhost:9200/_snapshot/myrepo
curl: (7) Failed connect to localhost:9200; Connection refused

I guess above error is because of elasticsearch is not running.
I restart the service and got same error:

Caused by: java.nio.file.AccessDeniedException: /var/log/back/repo
	at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) ~[?:1.8.0_131]
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:1.8.0_131]
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:1.8.0_131]
	at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384) ~[?:1.8.0_131]
	at java.nio.file.Files.createDirectory(Files.java:674) ~[?:1.8.0_131]
	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) ~[?:1.8.0_131]
	at java.nio.file.Files.createDirectories(Files.java:767) ~[?:1.8.0_131]
	at org.elasticsearch.bootstrap.Security.ensureDirectoryExists(Security.java:455) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.bootstrap.Security.addPath(Security.java:411) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.bootstrap.Security.addFilePermissions(Security.java:296) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.bootstrap.Security.createPermissions(Security.java:224) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.bootstrap.Security.configure(Security.java:114) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:227) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:350) ~[elasticsearch-5.4.1.jar:5.4.1]
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123) ~[elasticsearch-5.4.1.jar:5.4.1]
	... 6 more

Any idea?


(Snobear) #6

I think you also need to check that that directory exists on all nodes in the cluster (and with correct perms).


(Jason) #7

What does it mean? I just have one server.


(Bernt Rostad) #8

You may want to make sure you have no tabs or white spaces after any of the parameter values in elasticsearch.yml, I had that problem once and it tripped up my Elasticsearch installation.

A simple test would be to diff your elasticsearch.yml with the original from the download / intall to see if there are any unexpected differences, such as white space.

I always make a copy of the elasticsearch.yml before making any changes to it, then I run diff to make sure only the expected values were changed and that no extra white space got saved. It has kept me out of trouble since.


(Jason) #9

I created a .yml file with just two lines:

path.logs: /var/log/elastic
network.host: 0.0.0.0

But, got same error!!!!


(David Pilato) #10

May be you created the /var/log/back/repo as root but the user running elasticsearch is probably elasticsearch.


(Jason) #11

But, I changed the permission:

drwxr-xr-x. 3 elastic       elastic           17 Oct  2 12:38 back
drwxr-xr-x. 3 elastic       elastic           17 Oct  2 12:30 back-long

And user is "elastic" not "elasticsearch":

# su - elasticsearch
Last login: Tue Sep 26 11:14:51 +0330 2017 on pts/0
su: warning: cannot change directory to /home/elasticsearch: No such file or directory
This account is currently not available.
[root@localhost log]# su - elastic
Last login: Mon Oct  2 12:30:13 +0330 2017 on pts/1
[elastic@localhost ~]$ 

(Jason) #12

Problem solved. It is because of "/var/log/elastic" permission. I changed the owner of directory to "elastic" and problem solved.


(system) #13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.