I am unable to start elasticsearch 5.1.1 on Rhel 6. I am getting the following error :
2017-01-16 12:20:10,344 main ERROR RollingFileManager (/var/log/elasticsearch/clarity.log) java.io.FileNotFoundException: /var/log/elasticsearch/clarity.log (Permission denied) java.io.FileNotFoundException: /var/log/elasticsearch/clarity.log (Permission denied)
at java.io.FileOutputStream.open0(Native Method)
I think that you installed Elasticsearch via the RPM distribution, but are attempting to start Elasticsearch using the script /usr/share/elasticsearch/bin/elasticsearch. However, the RPM is designed to be used as a service so you should use the service manager (I think RHEL 6 is Sys V init) on your system to start and stop Elasticsearch. If you do not want to use a service manager, you should download an archive distribution of Elasticsearch (the tar.gz or zip distribution).
The stacktrace indicates we are trying to open a log file.
You haven't indicated what the name of your cluster is. Is it clarity? If so, I suppose this is during startup. It would be easier to tell if you provided the entire stacktrace. If it's not clarity, what does clarity mean to you?
Either way, the error message that you are seeing indicates that you do not have permissions to access the log file.
When the service is installed, /var/log/elasticsearch is created with user elasticsearch as the owner and all log files created under there will have the same ownership.
If you start as the service, Elasticsearch is running as the elasticsearch user. That is why I say that it appears you are not running as the service (because then you would be running as a user other than the elasticsearch user and that user probably does not have permissions to access this directory). That is what these situations amount to the vast majority of the time.
You're telling me that this not the case here, yet you are still seeing the permissions problem.
If what you say is correct (that you are indeed running as the installed service), please check the permissions of the entire tree starting at /var/log/elasticsearch.
I clearly understand what you are saying. I, however, clearly do not understand where I'm going wrong.
sudo yum install elasticsearch - this is the command I ran to install
sudo -i service elasticsearch start - this is the command I ran to start
drw-rw-rw-. 2 elasticsearch elasticsearch 4096 Jan 11 15:21 elasticsearch -- these are the permissions to the directory
Starting elasticsearch: 2017-01-17 12:12:41,671 main ERROR Unable to create file /var/log/elasticsearch/elasticsearch.log java.io.IOException: Permission denied -- this is the first line of the error log.
Moreso, this is the permissions to the /var/log directory
drw-rw-rw-. 18 root root 4096 Jan 17 11:18 log
And the worst part of all of it is that it was working. I then have rebooted the machine when it came back up logstash continued to respawning non-stop and elasticsearch would not start.
Why does no one have execute permissions on the /var/log (or /var/log/elasticsearch for that matter)? If you do not have execute permissions on a directory, you can not access any files inside the directory.
@jasontedor yes the cluster is clarity, i have changed the permissions and now i am not getting the permissions issue but now when i am trying to start elasticsearch i am getting the following error :
"elasticsearch dead but subsys locked "
i checked the logs and i got this :
[2017-01-18T06:56:18,222][ERROR][o.e.b.Bootstrap ] [clar-1] node validation exception
bootstrap checks failed
max number of threads  for user [elasticsearch] is too low, increase to at least 
I had changed the limit to 2048, but still i am getting the same error.
Please check in /etc/security/limits.d/ for the existence of a file that is overriding your limit (grep nproc /etc/security/limits.d/*); I bet there is a file matching *-nproc.conf overriding your limit.