Elasticsearch cannot start when change the default path of log and data

Elastic Stack Elasticsearch
1

hi all,
I install elasticsearch using RPM version. first i used default path of log and data as following ,

path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch

when i used default path, elasticsearch can start successfully by following command:
systemctl start elasticsearch.service

but when I change the path of log and data as following, elasticseach cannot start and no logs be created in the defined path
path.data: /home/srahimi/logelk
path.logs: /home/srahimi/logelk

could you please advise me?

2

Hi @sahere37,

Please check the permissions of that folder. if all ok then please share the logs of elasticsearch. so that i can identify the problem.

Regards,
Harsh Bajaj

3

the defined folder needs no permission. its is the home directory.
it is noted that the path (/var/log/elasticsearch) needs sudo permission but the defined path has no one. maybe, the path which we define for storing logs and data in RPM mode should have permission??

4

Could you please share the elasticsearch logs so that i can see the actual problem.

5

thanks for replying.
as i said, when i use default path (var/log/elasticsearch), everything is ok and elasticsearch.log is as following:

[2018-11-28T02:27:50,488][INFO ][o.e.n.Node ] [node-1] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.GLvLpTRo, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/elasticsearch, -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:/var/log/elasticsearch/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/etc/elasticsearch, -Des.distribution.flavor=default, -Des.distribution.type=rpm]
[2018-11-28T02:27:53,431][INFO ][o.e.p.PluginsService ] [node-1] loaded module [aggs-matrix-stats]
[2018-11-28T02:27:53,431][INFO ][o.e.p.PluginsService ] [node-1] loaded module [analysis-common]
[2018-11-28T02:27:53,431][INFO ][o.e.p.PluginsService ] [node-1] loaded module [ingest-common]
[2018-11-28T02:27:53,431][INFO ][o.e.p.PluginsService ] [node-1] loaded module [lang-expression]
[2018-11-28T02:27:53,431][INFO ][o.e.p.PluginsService ] [node-1] loaded module [lang-mustache]
[2018-11-28T02:27:53,431][INFO ][o.e.p.PluginsService ] [node-1] loaded module [lang-painless]
[2018-11-28T02:27:53,431][INFO ][o.e.p.PluginsService ] [node-1] loaded module [mapper-extras]
[2018-11-28T02:27:53,431][INFO ][o.e.p.PluginsService ] [node-1] loaded module [parent-join]
[2018-11-28T02:27:53,431][INFO ][o.e.p.PluginsService ] [node-1] loaded module [percolator]
[2018-11-28T02:27:53,431][INFO ][o.e.p.PluginsService ] [node-1] loaded module [rank-eval]
[2018-11-28T02:27:53,431][INFO ][o.e.p.PluginsService ] [node-1] loaded module [reindex]
[2018-11-28T02:27:53,431][INFO ][o.e.p.PluginsService ] [node-1] loaded module [repository-url]
[2018-11-28T02:27:53,432][INFO ][o.e.p.PluginsService ] [node-1] loaded module [transport-netty4]
[2018-11-28T02:27:53,432][INFO ][o.e.p.PluginsService ] [node-1] loaded module [tribe]
[2018-11-28T02:27:53,432][INFO ][o.e.p.PluginsService ] [node-1] loaded module [x-pack-ccr]
[2018-11-28T02:27:53,432][INFO ][o.e.p.PluginsService ] [node-1] loaded module [x-pack-core]
[2018-11-28T02:27:53,432][INFO ][o.e.p.PluginsService ] [node-1] loaded module [x-pack-deprecation]
[2018-11-28T02:27:53,432][INFO ][o.e.p.PluginsService ] [node-1] loaded module [x-pack-graph]
[2018-11-28T02:27:53,432][INFO ][o.e.p.PluginsService ] [node-1] loaded module [x-pack-logstash]
[2018-11-28T02:27:53,432][INFO ][o.e.p.PluginsService ] [node-1] loaded module [x-pack-ml]
[2018-11-28T02:27:53,432][INFO ][o.e.p.PluginsService ] [node-1] loaded module [x-pack-monitoring]
[2018-11-28T02:27:53,432][INFO ][o.e.p.PluginsService ] [node-1] loaded module [x-pack-rollup]
[2018-11-28T02:27:53,432][INFO ][o.e.p.PluginsService ] [node-1] loaded module [x-pack-security]
[2018-11-28T02:27:53,432][INFO ][o.e.p.PluginsService ] [node-1] loaded module [x-pack-sql]
[2018-11-28T02:27:53,432][INFO ][o.e.p.PluginsService ] [node-1] loaded module [x-pack-upgrade]
[2018-11-28T02:27:53,432][INFO ][o.e.p.PluginsService ] [node-1] loaded module [x-pack-watcher]
[2018-11-28T02:27:53,432][INFO ][o.e.p.PluginsService ] [node-1] no plugins loaded
[2018-11-28T02:28:00,145][INFO ][o.e.x.s.a.s.FileRolesStore] [node-1] parsed [0] roles from file [/etc/elasticsearch/roles.yml]
[2018-11-28T02:28:00,952][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [node-1] [controller/3996] [Main.cc@109] controller (64 bit): Version 6.5.1 (Build 1c5fe241dd9aea) Copyright (c) 2018 Elasticsearch BV
[2018-11-28T02:28:01,548][DEBUG][o.e.a.ActionModule ] [node-1] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2018-11-28T02:28:01,860][INFO ][o.e.d.DiscoveryModule ] [node-1] using discovery type [zen] and host providers [settings]
[2018-11-28T02:28:03,111][INFO ][o.e.n.Node ] [node-1] initialized
[2018-11-28T02:28:03,111][INFO ][o.e.n.Node ] [node-1] starting ...
[2018-11-28T02:28:03,333][INFO ][o.e.t.TransportService ] [node-1] publish_address {192.168.170.137:9300}, bound_addresses {192.168.170.137:9300}
[2018-11-28T02:28:03,370][INFO ][o.e.b.BootstrapChecks ] [node-1] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2018-11-28T02:28:06,540][INFO ][o.e.c.s.MasterService ] [node-1] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {node-1}{0nnC4wUtRjmaA40wl-LyhQ}{_w9pZt5KSCWWiAf7mQDHrA}{192.168.170.137}{192.168.170.137:9300}{ml.machine_memory=3873095680, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
[2018-11-28T02:28:06,545][INFO ][o.e.c.s.ClusterApplierService] [node-1] new_master {node-1}{0nnC4wUtRjmaA40wl-LyhQ}{_w9pZt5KSCWWiAf7mQDHrA}{192.168.170.137}{192.168.170.137:9300}{ml.machine_memory=3873095680, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {node-1}{0nnC4wUtRjmaA40wl-LyhQ}{_w9pZt5KSCWWiAf7mQDHrA}{192.168.170.137}{192.168.170.137:9300}{ml.machine_memory=3873095680, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]])
[2018-11-28T02:28:06,669][INFO ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [node-1] publish_address {192.168.170.137:9200}, bound_addresses {192.168.170.137:9200}
[2018-11-28T02:28:06,669][INFO ][o.e.n.Node ] [node-1] started
[2018-11-28T02:28:07,024][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [node-1] Failed to clear cache for realms []
[2018-11-28T02:28:07,083][INFO ][o.e.l.LicenseService ] [node-1] license [2b3ea0e4-fbaa-42e0-961e-a4ec36ac1793] mode [basic] - valid
[2018-11-28T02:28:07,103][INFO ][o.e.g.GatewayService ] [node-1] recovered [0] indices into cluster_state
[2018-11-28T02:29:19,595][INFO ][o.e.n.Node ] [node-1] stopping ...
[2018-11-28T02:29:19,680][INFO ][o.e.x.w.WatcherService ] [node-1] stopping watch service, reason [shutdown initiated]
[2018-11-28T02:29:20,053][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [node-1] [controller/3996] [Main.cc@148] Ml controller exiting
[2018-11-28T02:29:20,057][INFO ][o.e.x.m.j.p.NativeController] [node-1] Native controller process has stopped - no new native processes can be started

6

but, when i use my defined path (/home/srahimi/logelk) elasticsearch cannot start, and more importantly, there is no log in (/home/srahimi/logelk)

7

could you please check "var/log/messages" and share while starting elasticsearch with defined path.

8

Nov 28 03:22:11 localhost elasticsearch: 2018-11-28 03:22:11,387 main ERROR Could not create plugin of type class org.apache.logging.log4j.core.appender.RollingFileAppender for element RollingFile: java.lang.IllegalStateException: ManagerFactory [org.apache.logging.log4j.core.appender.rolling.RollingFileManager$RollingFileManagerFactory@5c671d7f] unable to create manager for [/home/srahimi/log/elasticsearch_deprecation.log] with data [org.apache.logging.log4j.core.appender.rolling.RollingFileManager$FactoryData@313b2ea6[pattern=/home/srahimi/log/elasticsearch_deprecation-%i.log.gz, append=true, bufferedIO=true, bufferSize=8192, policy=CompositeTriggeringPolicy(policies=[SizeBasedTriggeringPolicy(size=1073741824)]), strategy=DefaultRolloverStrategy(min=1, max=4, useMax=true), advertiseURI=null, layout=[%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %.-10000m%n, filePermissions=null, fileOwner=null]] java.lang.IllegalStateException: ManagerFactory [org.apache.logging.log4j.core.appender.rolling.RollingFileManager$RollingFileManagerFactory@5c671d7f] unable to create manager for [/home/srahimi/log/elasticsearch_deprecation.log] with data [org.apache.logging.log4j.core.appender.rolling.RollingFileManager$FactoryData@313b2ea6[pattern=/home/srahimi/log/elasticsearch_deprecation-%i.log.gz, append=true, bufferedIO=true, bufferSize=8192, policy=CompositeTriggeringPolicy(policies=[SizeBasedTriggeringPolicy(size=1073741824)]), strategy=DefaultRolloverStrategy(min=1, max=4, useMax=true), advertiseURI=null, layout=[%d{ISO8601}][%-5p][%-25c{1.}] [%node_name]%marker %.-10000m%n, filePermissions=null, fileOwner=null]]
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.appender.AbstractManager.getManager(AbstractManager.java:115)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.appender.OutputStreamManager.getManager(OutputStreamManager.java:114)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.appender.rolling.RollingFileManager.getFileManager(RollingFileManager.java:188)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.appender.RollingFileAppender$Builder.build(RollingFileAppender.java:145)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.appender.RollingFileAppender$Builder.build(RollingFileAppender.java:61)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:123)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:959)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:899)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:891)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.config.AbstractConfiguration.doConfigure(AbstractConfiguration.java:514)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.config.AbstractConfiguration.initialize(AbstractConfiguration.java:238)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:250)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:547)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:263)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:234)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:127)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:302)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:127)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.cli.Command.main(Command.java:90)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:86)
Nov 28 03:22:11 localhost elasticsearch: 2018-11-28 03:22:11,388 main ERROR Unable to invoke factory method in class org.apache.logging.log4j.core.appender.RollingFileAppender for element RollingFile: java.lang.IllegalStateException: No factory method found for class org.apache.logging.log4j.core.appender.RollingFileAppender java.lang.IllegalStateException: No factory method found for class org.apache.logging.log4j.core.appender.RollingFileAppender
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.findFactoryMethod(PluginBuilder.java:235)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:135)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:959)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:899)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:891)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.config.AbstractConfiguration.doConfigure(AbstractConfiguration.java:514)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.config.AbstractConfiguration.initialize(AbstractConfiguration.java:238)

9

Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:250)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:547)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:263)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:234)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:127)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:302)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:127)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.cli.Command.main(Command.java:90)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:86)
Nov 28 03:22:11 localhost elasticsearch: 2018-11-28 03:22:11,389 main ERROR Unable to create file /home/srahimi/log/elasticsearch_index_search_slowlog.log java.io.IOException: Could not create directory /home/srahimi/log
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.util.FileUtils.mkdir(FileUtils.java:127)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.util.FileUtils.makeParentDirs(FileUtils.java:144)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.appender.rolling.RollingFileManager$RollingFileManagerFactory.createManager(RollingFileManager.java:627)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.appender.rolling.RollingFileManager$RollingFileManagerFactory.createManager(RollingFileManager.java:608)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.appender.AbstractManager.getManager(AbstractManager.java:113)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.appender.OutputStreamManager.getManager(OutputStreamManager.java:114)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.appender.rolling.RollingFileManager.getFileManager(RollingFileManager.java:188)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.appender.RollingFileAppender$Builder.build(RollingFileAppender.java:145)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.appender.RollingFileAppender$Builder.build(RollingFileAppender.java:61)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.config.plugins.util.PluginBuilder.build(PluginBuilder.java:123)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.config.AbstractConfiguration.createPluginObject(AbstractConfiguration.java:959)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:899)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.config.AbstractConfiguration.createConfiguration(AbstractConfiguration.java:891)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.config.AbstractConfiguration.doConfigure(AbstractConfiguration.java:514)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.config.AbstractConfiguration.initialize(AbstractConfiguration.java:238)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.config.AbstractConfiguration.start(AbstractConfiguration.java:250)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.LoggerContext.setConfiguration(LoggerContext.java:547)
Nov 28 03:22:11 localhost elasticsearch: at org.apache.logging.log4j.core.LoggerContext.start(LoggerContext.java:263)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:234)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:127)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:302)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:127)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.cli.Command.main(Command.java:90)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93)
Nov 28 03:22:11 localhost elasticsearch: at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:86)
Nov 28 03:22:11 localhost elasticsearch: 2018-11-28 03:22:11,390 main ERROR Could not create plugin of type class

10

Hi,
According to your logs, im able to see that there is some permission problem to create the defined file.

Request you to provide the proper permissions and check,

/home/srahimi/log/elasticsearch_index_search_slowlog.log java.io.IOException: Could not create directory /home/srahimi/log

Regards,
Harsh

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.