Elasticsearch doesn't start and there is no log


(Safwan Erooth) #1

Hello I just Installed Elasticsearch following the official documentation on Ubuntu 16.04. Installation completed without any error. After installation, I turned on the service using the command, sudo -i service elasticsearch start. However when checking status am getting the following error:

● elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendo
   Active: failed (Result: exit-code) since Fri 2017-06-02 15:09:41 IST; 6s ago
     Docs: http://www.elastic.co
  Process: 11092 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_D
  Process: 11090 ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd
 Main PID: 11092 (code=exited, status=1/FAILURE)
Jun 02 15:09:41 safwan-laptop systemd[1]: Starting Elasticsearch...
Jun 02 15:09:41 safwan-laptop systemd[1]: Started Elasticsearch.
Jun 02 15:09:41 safwan-laptop systemd[1]: elasticsearch.service: Main process ex
Jun 02 15:09:41 safwan-laptop systemd[1]: elasticsearch.service: Unit entered fa
Jun 02 15:09:41 safwan-laptop systemd[1]: elasticsearch.service: Failed with res

There are no logs in /var/log/elasticsearch.

My configuration file is as below:

> # ======================== Elasticsearch Configuration =========================
> #
> # NOTE: Elasticsearch comes with reasonable defaults for most settings.
> #       Before you set out to tweak and tune the configuration, make sure you
> #       understand what are you trying to accomplish and the consequences.
> #
> # The primary way of configuring a node is via this file. This template lists
> # the most important settings you may want to configure for a production cluster.
> #
> # Please consult the documentation for further information on configuration options:
> # https://www.elastic.co/guide/en/elasticsearch/reference/index.html
> #
> # ---------------------------------- Cluster -----------------------------------
> #
> # Use a descriptive name for your cluster:
> #
> cluster.name: hf-app
> #
> # ------------------------------------ Node ------------------------------------
> #
> # Use a descriptive name for the node:
> #
> node.name: hf-node-1
> #
> # Add custom attributes to the node:
> #
> #node.attr.rack: r1
> #
> # ----------------------------------- Paths ------------------------------------
> #
> # Path to directory where to store the data (separate multiple locations by comma):
> #
> #path.data: /path/to/data
> #
> # Path to log files:
> #
> #path.logs: /var/log/elasticsearch
> #
> # ----------------------------------- Memory -----------------------------------
> #
> # Lock the memory on startup:
> #
> #bootstrap.memory_lock: true
> #
> # Make sure that the heap size is set to about half the memory available
> # on the system and that the owner of the process is allowed to use this
> # limit.
> #
> # Elasticsearch performs poorly when the system is swapping the memory.
> #
> # ---------------------------------- Network -----------------------------------
> #
> # Set the bind address to a specific IP (IPv4 or IPv6):
> #
> #network.host: 192.168.0.1
> #
> # Set a custom port for HTTP:
> #
> #http.port: 9200
> #
> # For more information, consult the network module documentation.
> #
> # --------------------------------- Discovery ----------------------------------
> #
> # Pass an initial list of hosts to perform discovery when new node is started:
> # The default list of hosts is ["127.0.0.1", "[::1]"]
> #
> #discovery.zen.ping.unicast.hosts: ["host1", "host2"]
> #
> # Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1):
> #
> #discovery.zen.minimum_master_nodes: 3
> #
> # For more information, consult the zen discovery module documentation.
> #
> # ---------------------------------- Gateway -----------------------------------
> #
> # Block initial recovery after a full cluster restart until N nodes are started:
> #
> #gateway.recover_after_nodes: 3
> #
> # For more information, consult the gateway module documentation.
> #
> # ---------------------------------- Various -----------------------------------
> #
> # Require explicit names when deleting indices:
> #
> #action.destructive_requires_name: true

My system memory is 4 GB.

I am as of now, out of options. Tried many solutions found online with nothing actually working.


ElasticSearch Does Not Start on Clean Install
(Mark Walkom) #2

What version?


(Safwan Erooth) #3

The version is 5.4.


(Eddie Turizo) #4

@esafwan Does running the elasticsearch script manually provide more information than what starting the service produces?

/usr/share/elasticsearch/bin/elasticsearch

(Eddie Turizo) #5

You may also want to consider reading a solution posted @fcoelho in the thread linked below to see if it is applicable to your setup.

https://discuss.elastic.co/t/cant-start-elasticsearch-with-ubuntu-16-04/48730/9?u=eddieturizo


(Safwan Erooth) #6

@EddieTurizo

Thanks for this suggestion. It finally printed out the issue. ElasticSearch was not finding Java in the path. Once that was fixed, I got further issues. I was able to find solution for those and start.

For others who might have same issue, am adding the details of how I reached the solution below and the output I got when ElasticSearch started.

Error 1, I got after fixing adding Java to path:

2017-06-03 12:50:37,044 main ERROR Could not register mbeans java.security.AccessControlException: access denied ("javax.management.MBeanTrustPermission" "register")
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:585)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.checkMBeanTrustPermission(DefaultMBeanServerInterceptor.java:1848)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:322)
at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
at org.apache.logging.log4j.core.jmx.Server.register(Server.java:389)
at org.apache.logging.log4j.core.jmx.Server.reregisterMBeansAfterReconfigure(Server.java:167)
at org.apache.logging.log4j.core.jmx.Server.reregisterMBeansAfterReconfigure(Server.java:140)
at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:556)
at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:261)
at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:206)
at org.apache.logging.log4j.core.config.Configurator.initialize(Configurator.java:221)
at org.apache.logging.log4j.core.config.Configurator.initialize(Configurator.java:197)
at org.elasticsearch.common.logging.LogConfigurator.configureStatusLogger(LogConfigurator.java:126)
at org.elasticsearch.common.logging.LogConfigurator.configureWithoutConfig(LogConfigurator.java:68)
at org.elasticsearch.cli.Command.main(Command.java:85)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84)

Error 2, I got after fixing adding Java to path:

Exception in thread "main" org.elasticsearch.bootstrap.BootstrapException: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/config
Likely root cause: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/config
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
at java.nio.file.Files.readAttributes(Files.java:1737)
at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:225)
at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
at java.nio.file.Files.walkFileTree(Files.java:2662)
at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:101)
at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:84)
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:316)
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:123)
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:114)
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:67)
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122)
at org.elasticsearch.cli.Command.main(Command.java:88)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84)
Refer to the log for complete error details.

Solution for Error 1:
Got solved when I changed permission of the folder /etc/elasticsearch. I got this suggestion while googling the error message.

Solution for Error2:
The issue is that, ElasticSearch is reading wrong path for configuration. While my config is at /etc/elasticsearch' its trying to read it at/usr/share/elasticsearch/config' as you can see in the error.

I was able to get over this error by starting elastic search with any of the below commands:

  1. sudo systemctl start elasticsearch
  2. service elasticsearch start

Instead of the command /usr/share/elasticsearch/bin/elasticsearch, which helped find the root issues initially.

There are still some errors, but now finally ElasticSearch starts. That's a first . The output, when elasticsearch start is added below:

● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
Active: active (running) since Sat 2017-06-03 12:46:00 IST; 20min ago
Docs: http://www.elastic.co
Process: 1761 ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec (code=exited, status=0/SUCCESS)
Main PID: 1767 (java)
CGroup: /system.slice/elasticsearch.service
└─1767 /usr/bin/java -Xms2g -Xmx2g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnl

Jun 03 12:46:02 safwan-laptop elasticsearch[1767]: 2017-06-03 12:46:02,337 main ERROR Null object returned for RollingFile in Appenders.
Jun 03 12:46:02 safwan-laptop elasticsearch[1767]: 2017-06-03 12:46:02,337 main ERROR Null object returned for RollingFile in Appenders.
Jun 03 12:46:02 safwan-laptop elasticsearch[1767]: 2017-06-03 12:46:02,337 main ERROR Null object returned for RollingFile in Appenders.
Jun 03 12:46:02 safwan-laptop elasticsearch[1767]: 2017-06-03 12:46:02,337 main ERROR Null object returned for RollingFile in Appenders.
Jun 03 12:46:02 safwan-laptop elasticsearch[1767]: 2017-06-03 12:46:02,338 main ERROR Unable to locate appender "rolling" for logger config "
Jun 03 12:46:02 safwan-laptop elasticsearch[1767]: 2017-06-03 12:46:02,338 main ERROR Unable to locate appender "index_indexing_slowlog_rolli
Jun 03 12:46:02 safwan-laptop elasticsearch[1767]: 2017-06-03 12:46:02,338 main ERROR Unable to locate appender "index_search_slowlog_rolling
Jun 03 12:46:02 safwan-laptop elasticsearch[1767]: 2017-06-03 12:46:02,338 main ERROR Unable to locate appender "deprecation_rolling" for log
Jun 03 13:03:32 safwan-laptop systemd[1]: Started Elasticsearch.
Jun 03 13:05:58 safwan-laptop systemd[1]: Started Elasticsearch.


(system) #7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.