# Elasticsearch doesn’t start: status=1/FAILURE (TLS/x-pack) **URL:** \ **Category:** Elasticsearch\ **Created:** [March 12, 2018, 8:00am UTC](https://discuss.elastic.co/t/elasticsearch-doesn-t-start-status-1-failure-tls-x-pack/123498 "2018-03-12T08:00:21Z")\ **Posts on this page:** 18\ **Page:** 1
**Author:** ![manst](https://avatars.discourse-cdn.com/v4/letter/m/ed655f/32.png) [@manst](https://discuss.elastic.co/u/manst)\ **Post date:** [March 12, 2018, 8:00am UTC](https://discuss.elastic.co/t/elasticsearch-doesn-t-start-status-1-failure-tls-x-pack/123498/1 "2018-03-12T08:00:21Z")
Hello! I've tried to follow this list: > [@Unable to run set-password in 6.x](https://discuss.elastic.co/t/unable-to-run-set-password-in-6-x/117382/4): > > Support is definitely the best option here, but in order to make as much information available online here's a few pointers: Generally I would recommend running setup-passwords before configuring TLS. You don't have to do that, but when you're starting out for the very first time, I think that the most straight forward path is the order below. It does require starting and stopping you node(s) multiple times, and changing configuration as you go, so it's not suitable for every environment (whi… Install a single ES node. Install X-Pack on that node. Run setup-passwords on that node. Add additional nodes (ES + X-Pack). Enable TLS for node transport. Enable TLS for HTTP. I'm using this manual [https://www.elastic.co/guide/en/elasticsearch/reference/current/installing-xpack-es.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/installing-xpack-es.html) I've setuped 1node in test purposes , than installed and setuped x-pack. After I've installed additional 2 node. And if I uncomment strings in first node, for example: > `#discovery.zen.ping.unicast.hosts: ["192.168.222.232, 192.168.222.233"]` or ``` #xpack.security.transport.ssl.enabled: true #xpack.security.transport.ssl.verification_mode: certificate #xpack.security.transport.ssl.keystore.path: /etc/elasticsearch/certs/master.p12 #xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/certs/master.p12 ``` my elastic goes down ( : ``` Mar 11 14:46:33 master systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE Mar 11 14:46:33 master systemd[1]: elasticsearch.service: Unit entered failed state. Mar 11 14:46:33 master systemd[1]: elasticsearch.service: Failed with result 'exit-code'. [2018-03-11T15:19:20,930][ERROR][o.e.b.Bootstrap] [master] node validation exception ``` And I don't understand, what I should do.....( ---
**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\ **Post date:** [March 12, 2018, 8:07am UTC](https://discuss.elastic.co/t/elasticsearch-doesn-t-start-status-1-failure-tls-x-pack/123498/2 "2018-03-12T08:07:37Z")
- Did you [generate certificates for all your nodes](https://www.elastic.co/guide/en/elasticsearch/reference/current/configuring-tls.html#node-certificates) ? - - Did you enable TLS in all nodes ? Can you paste in a larger part of your log so that we can see the error? ---
**Author:** ![manst](https://avatars.discourse-cdn.com/v4/letter/m/ed655f/32.png) [@manst](https://discuss.elastic.co/u/manst)\ **Post date:** [March 12, 2018, 8:24am UTC](https://discuss.elastic.co/t/elasticsearch-doesn-t-start-status-1-failure-tls-x-pack/123498/3 "2018-03-12T08:24:47Z")
Yes, I did . I'll paste log later ---
**Author:** ![ikakavas](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ikakavas/32/34430_2.png) [@ikakavas](https://discuss.elastic.co/u/ikakavas)\ **Post date:** [March 12, 2018, 8:28am UTC](https://discuss.elastic.co/t/elasticsearch-doesn-t-start-status-1-failure-tls-x-pack/123498/4 "2018-03-12T08:28:02Z")
> [@manst](#): > > I'll paste log later Right below: `[2018-03-11T15:19:20,930][ERROR][o.e.b.Bootstrap] [master] node validation exception` you should see that bootstrap checks that have failed. This will give us a proper indication of what goes wrong ---
**Author:** ![manst](https://avatars.discourse-cdn.com/v4/letter/m/ed655f/32.png) [@manst](https://discuss.elastic.co/u/manst)\ **Post date:** [March 12, 2018, 8:05pm UTC](https://discuss.elastic.co/t/elasticsearch-doesn-t-start-status-1-failure-tls-x-pack/123498/5 "2018-03-12T20:05:17Z")
``` [2018-03-12T22:23:17,470][INFO][o.e.n.Node] [master] initializing ... [2018-03-12T22:23:18,270][INFO][o.e.e.NodeEnvironment] [master] using [1] data paths, mounts [[/ (/dev/sda1)]], net usable_space [8.6gb], net total_space [11.9gb], types [btrfs] [2018-03-12T22:23:18,270][INFO][o.e.e.NodeEnvironment] [master] heap size [247.5mb], compressed ordinary object pointers [true] [2018-03-12T22:23:18,412][INFO][o.e.n.Node] [master] node name [master], node ID [KC0RJjSzQhiagzzhHcDjag] [2018-03-12T22:23:18,412][INFO][o.e.n.Node] [master] version[6.2.2], pid[384], build[10b1edd/2018-02-16T19:01:30.685723Z], OS[Linux/4.9.0-6-amd64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_151/25.151-b12] [2018-03-12T22:23:18,412][INFO][o.e.n.Node] [master] JVM arguments [-Xms256m, -Xmx256m, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.2ED2eX9r, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/elasticsearch, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:/var/log/elasticsearch/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/etc/elasticsearch] [2018-03-12T22:23:24,190][INFO][o.e.p.PluginsService] [master] loaded module [aggs-matrix-stats] [2018-03-12T22:23:24,190][INFO][o.e.p.PluginsService] [master] loaded module [analysis-common] [2018-03-12T22:23:24,190][INFO][o.e.p.PluginsService] [master] loaded module [ingest-common] [2018-03-12T22:23:24,190][INFO][o.e.p.PluginsService] [master] loaded module [lang-expression] [2018-03-12T22:23:24,190][INFO][o.e.p.PluginsService] [master] loaded module [lang-mustache] [2018-03-12T22:23:24,190][INFO][o.e.p.PluginsService] [master] loaded module [lang-painless] [2018-03-12T22:23:24,190][INFO][o.e.p.PluginsService] [master] loaded module [mapper-extras] [2018-03-12T22:23:24,190][INFO][o.e.p.PluginsService] [master] loaded module [parent-join] [2018-03-12T22:23:24,190][INFO][o.e.p.PluginsService] [master] loaded module [percolator] [2018-03-12T22:23:24,190][INFO][o.e.p.PluginsService] [master] loaded module [rank-eval] [2018-03-12T22:23:24,190][INFO][o.e.p.PluginsService] [master] loaded module [reindex] [2018-03-12T22:23:24,191][INFO][o.e.p.PluginsService] [master] loaded module [repository-url] [2018-03-12T22:23:24,191][INFO][o.e.p.PluginsService] [master] loaded module [transport-netty4] [2018-03-12T22:23:24,191][INFO][o.e.p.PluginsService] [master] loaded module [tribe] [2018-03-12T22:23:24,191][INFO][o.e.p.PluginsService] [master] loaded plugin [x-pack-core] [2018-03-12T22:23:24,191][INFO][o.e.p.PluginsService] [master] loaded plugin [x-pack-deprecation] [2018-03-12T22:23:24,191][INFO][o.e.p.PluginsService] [master] loaded plugin [x-pack-graph] [2018-03-12T22:23:24,191][INFO][o.e.p.PluginsService] [master] loaded plugin [x-pack-logstash] [2018-03-12T22:23:24,192][INFO][o.e.p.PluginsService] [master] loaded plugin [x-pack-ml] [2018-03-12T22:23:24,192][INFO][o.e.p.PluginsService] [master] loaded plugin [x-pack-monitoring] [2018-03-12T22:23:24,192][INFO][o.e.p.PluginsService] [master] loaded plugin [x-pack-security] [2018-03-12T22:23:24,192][INFO][o.e.p.PluginsService] [master] loaded plugin [x-pack-upgrade] [2018-03-12T22:23:24,192][INFO][o.e.p.PluginsService] [master] loaded plugin [x-pack-watcher] [2018-03-12T22:23:27,885][INFO][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/816] [Main.cc@128] controller (64 bit): Version 6.2.2 (Build 61c43548e5e0f2) Copyright (c) 2018 Elasticsearch BV [2018-03-12T22:23:29,882][DEBUG][o.e.a.ActionModule] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security [2018-03-12T22:23:30,434][INFO][o.e.d.DiscoveryModule] [master] using discovery type [zen] [2018-03-12T22:23:31,352][INFO][o.e.n.Node] [master] initialized [2018-03-12T22:23:31,352][INFO][o.e.n.Node] [master] starting ... [2018-03-12T22:23:31,590][INFO][o.e.t.TransportService] [master] publish_address {192.168.222.232:9300}, bound_addresses {192.168.222.232:9300} [2018-03-12T22:23:31,629][INFO][o.e.b.BootstrapChecks] [master] bound or publishing to a non-loopback address, enforcing bootstrap checks [2018-03-12T22:23:34,724][INFO][o.e.c.s.MasterService] [master] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {master}{KC0RJjSzQhiagzzhHcDjag}{6njvab3JSzuhSdFSM6Cg4Q}{192.168.222.232}{192.168.222.232:9300}{ml.machine_memory=2101534720, ml.max_open_jobs=20, ml.enabled=true} [2018-03-12T22:23:34,729][INFO][o.e.c.s.ClusterApplierService] [master] new_master {master}{KC0RJjSzQhiagzzhHcDjag}{6njvab3JSzuhSdFSM6Cg4Q}{192.168.222.232}{192.168.222.232:9300}{ml.machine_memory=2101534720, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {master}{KC0RJjSzQhiagzzhHcDjag}{6njvab3JSzuhSdFSM6Cg4Q}{192.168.222.232}{192.168.222.232:9300}{ml.machine_memory=2101534720, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]]) [2018-03-12T22:23:34,797][INFO][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [master] publish_address {192.168.222.232:9200}, bound_addresses {[::]:9200} [2018-03-12T22:23:34,798][INFO][o.e.n.Node] [master] started [2018-03-12T22:23:37,601][INFO][o.e.l.LicenseService] [master] license [5068e9b2-4634-413c-8dfe-3bfe8d47409f] mode [trial] - valid [2018-03-12T22:23:37,608][WARN][o.e.l.LicenseService] [master] # # License [will expire] on [Tuesday, April 03, 2018]. If you have a new license, please update it. # Otherwise, please reach out to your support contact. # ``` ---
**Author:** ![manst](https://avatars.discourse-cdn.com/v4/letter/m/ed655f/32.png) [@manst](https://discuss.elastic.co/u/manst)\ **Post date:** [March 12, 2018, 8:06pm UTC](https://discuss.elastic.co/t/elasticsearch-doesn-t-start-status-1-failure-tls-x-pack/123498/6 "2018-03-12T20:06:41Z")
``` # Commercial plugins operate with reduced functionality on license expiration: # - security # - Cluster health, cluster stats and indices stats operations are blocked # - All data operations (read and write) continue to work # - watcher # - PUT / GET watch APIs are disabled, DELETE watch API continues to work # - Watches execute and write to the history # - The actions of the watches don't execute # - monitoring # - The agent will stop collecting cluster and indices metrics # - The agent will stop automatically cleaning indices older than [xpack.monitoring.history.duration] # - graph # - Graph explore APIs are disabled # - ml # - Machine learning APIs are disabled # - logstash # - Logstash will continue to poll centrally-managed pipelines # - deprecation # - Deprecation APIs are disabled # - upgrade # - Upgrade API is disabled [2018-03-12T22:23:37,619][INFO][o.e.g.GatewayService] [master] recovered [14] indices into cluster_state [2018-03-12T22:23:41,490][INFO][o.e.x.s.a.s.m.NativeRoleMappingStore] [master] The security index is not yet available - no role mappings can be loaded [2018-03-12T22:23:42,497][INFO][o.e.c.m.MetaDataCreateIndexService] [master] [.monitoring-es-6-2018.03.12] creating index, cause [auto(bulk api)], templates [.monitoring-es], shards [1]/[0], mappings [doc] [2018-03-12T22:23:54,074][INFO][o.e.c.r.a.AllocationService] [master] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[.monitoring-es-6-2018.03.09][0]] ...]). [2018-03-12T22:24:05,453][INFO][o.e.c.m.MetaDataCreateIndexService] [master] [.monitoring-kibana-6-2018.03.12] creating index, cause [auto(bulk api)], templates [.monitoring-kibana], shards [1]/[0], mappings [doc] [2018-03-12T22:24:05,601][INFO][o.e.c.r.a.AllocationService] [master] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.monitoring-kibana-6-2018.03.12][0]] ...]). [2018-03-12T22:24:49,189][INFO][o.e.c.m.MetaDataCreateIndexService] [master] [.watcher-history-7-2018.03.12] creating index, cause [auto(bulk api)], templates [.watch-history-7], shards [1]/[0], mappings [doc] [2018-03-12T22:24:49,401][INFO][o.e.c.r.a.AllocationService] [master] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.watcher-history-7-2018.03.12][0]] ...]). [2018-03-12T22:24:49,461][INFO][o.e.c.m.MetaDataMappingService] [master] [.watcher-history-7-2018.03.12/61AZoqRBSGK_saEDEfyyLA] update_mapping [doc] [2018-03-12T22:24:49,475][INFO][o.e.c.m.MetaDataMappingService] [master] [.watcher-history-7-2018.03.12/61AZoqRBSGK_saEDEfyyLA] update_mapping [doc] [2018-03-12T22:24:49,565][INFO][o.e.c.m.MetaDataMappingService] [master] [.watcher-history-7-2018.03.12/61AZoqRBSGK_saEDEfyyLA] update_mapping [doc] [2018-03-12T22:57:10,428][INFO][o.e.n.Node] [master] stopping ... [2018-03-12T22:57:10,467][INFO][o.e.x.w.WatcherService] [master] stopping watch service, reason [shutdown initiated] [2018-03-12T22:57:10,467][INFO][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/816] [Main.cc@168] Ml controller exiting [2018-03-12T22:57:10,468][INFO][o.e.x.m.j.p.NativeController] Native controller process has stopped - no new native processes can be started [2018-03-12T22:57:11,381][INFO][o.e.n.Node] [master] stopped [2018-03-12T22:57:11,381][INFO][o.e.n.Node] [master] closing ... [2018-03-12T22:57:11,406][INFO][o.e.n.Node] [master] closed [2018-03-12T22:57:13,085][INFO][o.e.n.Node] [master] initializing ... [2018-03-12T22:57:13,194][INFO][o.e.e.NodeEnvironment] [master] using [1] data paths, mounts [[/ (/dev/sda1)]], net usable_space [8.7gb], net total_space [11.9gb], types [btrfs] [2018-03-12T22:57:13,194][INFO][o.e.e.NodeEnvironment] [master] heap size [247.5mb], compressed ordinary object pointers [true] [2018-03-12T22:57:13,248][INFO][o.e.n.Node] [master] node name [master], node ID [KC0RJjSzQhiagzzhHcDjag] [2018-03-12T22:57:13,249][INFO][o.e.n.Node] [master] version[6.2.2], pid[7443], build[10b1edd/2018-02-16T19:01:30.685723Z], OS[Linux/4.9.0-6-amd64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_151/25.151-b12] [2018-03-12T22:57:13,249][INFO][o.e.n.Node] [master] JVM arguments [-Xms256m, -Xmx256m, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.c1KRiW3p, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/elasticsearch, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:/var/log/elasticsearch/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/etc/elasticsearch] [2018-03-12T22:57:14,704][ERROR][o.e.b.Bootstrap] Exception ``` ---
**Author:** ![manst](https://avatars.discourse-cdn.com/v4/letter/m/ed655f/32.png) [@manst](https://discuss.elastic.co/u/manst)\ **Post date:** [March 12, 2018, 8:18pm UTC](https://discuss.elastic.co/t/elasticsearch-doesn-t-start-status-1-failure-tls-x-pack/123498/7 "2018-03-12T20:18:26Z")
``` [2018-03-12T22:57:14,704][ERROR][o.e.b.Bootstrap] Exception java.lang.IllegalStateException: failed to load plugin class [org.elasticsearch.xpack.core.XPackPlugin] at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:563) ~[elasticsearch-6.2.2.jar:6.2.2] at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:505) ~[elasticsearch-6.2.2.jar:6.2.2] at org.elasticsearch.node.Node.(Node.java:303) ~[elasticsearch-6.2.2.jar:6.2.2] at org.elasticsearch.node.Node.(Node.java:246) ~[elasticsearch-6.2.2.jar:6.2.2] at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:213) ~[elasticsearch-6.2.2.jar:6.2.2] at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:213) ~[elasticsearch-6.2.2.jar:6.2.2] at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:323) [elasticsearch-6.2.2.jar:6.2.2] at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:121) [elasticsearch-6.2.2.jar:6.2.2] at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) [elasticsearch-6.2.2.jar:6.2.2] at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-6.2.2.jar:6.2.2] at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-6.2.2.jar:6.2.2] at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-6.2.2.jar:6.2.2] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) [elasticsearch-6.2.2.jar:6.2.2] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:85) [elasticsearch-6.2.2.jar:6.2.2] Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?] Caused by: org.elasticsearch.ElasticsearchException: failed to initialize a TrustManagerFactory at org.elasticsearch.xpack.core.ssl.StoreTrustConfig.createTrustManager(StoreTrustConfig.java:72) ~[?:?] at org.elasticsearch.xpack.core.ssl.SSLService.createSslContext(SSLService.java:412) ~[?:?] at java.util.HashMap.computeIfAbsent(HashMap.java:1127) ~[?:1.8.0_151] at org.elasticsearch.xpack.core.ssl.SSLService.loadSSLConfigurations(SSLService.java:464) ~[?:?] at org.elasticsearch.xpack.core.ssl.SSLService.(SSLService.java:91) ~[?:?] at org.elasticsearch.xpack.core.XPackPlugin.(XPackPlugin.java:127) ~[?:?] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?] Caused by: java.nio.file.AccessDeniedException: /etc/elasticsearch/certs/master.p12 at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) ~[?:?] at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:?] at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:?] at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214) ~[?:?] at java.nio.file.Files.newByteChannel(Files.java:361) ~[?:1.8.0_151] [2018-03-12T22:57:14,721][WARN][o.e.b.ElasticsearchUncaughtExceptionHandler] [master] uncaught exception in thread [main] org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to load plugin class [org.elasticsearch.xpack.core.XPackPlugin] at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:125) ~[elasticsearch-6.2.2.jar:6.2.2] at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:112) ~[elasticsearch-6.2.2.jar:6.2.2] at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.2.2.jar:6.2.2] at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.2.2.jar:6.2.2] at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.2.2.jar:6.2.2] Caused by: java.lang.IllegalStateException: failed to load plugin class [org.elasticsearch.xpack.core.XPackPlugin] at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:563) ~[elasticsearch-6.2.2.jar:6.2.2] Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?] at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_151] at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:554) ~[elasticsearch-6.2.2.jar:6.2.2] at org.elasticsearch.node.Node.(Node.java:303) ~[elasticsearch-6.2.2.jar:6.2.2] at org.elasticsearch.node.Node.(Node.java:246) ~[elasticsearch-6.2.2.jar:6.2.2] at org.elasticsearch.bootstrap.Bootstrap$5.(Bootstrap.java:213) ~[elasticsearch-6.2.2.jar:6.2.2] at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:213) ~[elasticsearch-6.2.2.jar:6.2.2] at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:323) ~[elasticsearch-6.2.2.jar:6.2.2] Caused by: org.elasticsearch.ElasticsearchException: failed to initialize a TrustManagerFactory at org.elasticsearch.xpack.core.ssl.StoreTrustConfig.createTrustManager(StoreTrustConfig.java:72) ~[?:?] at org.elasticsearch.xpack.core.ssl.SSLService.createSslContext(SSLService.java:412) ~[?:?] at java.util.HashMap.computeIfAbsent(HashMap.java:1127) ~[?:1.8.0_151] at org.elasticsearch.xpack.core.ssl.SSLService.loadSSLConfigurations(SSLService.java:464) ~[?:?] at org.elasticsearch.xpack.core.ssl.SSLService.(SSLService.java:91) ~[?:?] at org.elasticsearch.xpack.core.XPackPlugin.(XPackPlugin.java:127) ~[?:?] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?] Caused by: java.nio.file.AccessDeniedException: /etc/elasticsearch/certs/master.p12 at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) ~[?:?] at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:?] ``` ---
**Author:** ![manst](https://avatars.discourse-cdn.com/v4/letter/m/ed655f/32.png) [@manst](https://discuss.elastic.co/u/manst)\ **Post date:** [March 12, 2018, 8:19pm UTC](https://discuss.elastic.co/t/elasticsearch-doesn-t-start-status-1-failure-tls-x-pack/123498/8 "2018-03-12T20:19:40Z")
``` root@master:/var/log/elasticsearch# systemctl status elasticsearch ● elasticsearch.service - Elasticsearch Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Mon 2018-03-12 22:57:14 +03; 1min 2s ago Docs: http://www.elastic.co Process: 7443 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=1 Main PID: 7443 (code=exited, status=1/FAILURE) Mar 12 22:57:11 master systemd[1]: Started Elasticsearch. Mar 12 22:57:14 master systemd[1]: elasticsearch.service: Main process exited, code=exited, status=1/FAILURE Mar 12 22:57:14 master systemd[1]: elasticsearch.service: Unit entered failed state. Mar 12 22:57:14 master systemd[1]: elasticsearch.service: Failed with result 'exit-code'. ``` ---
**Author:** ![TimV](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timv/32/13162_2.png) [@TimV](https://discuss.elastic.co/u/TimV)\ **Post date:** [March 12, 2018, 10:46pm UTC](https://discuss.elastic.co/t/elasticsearch-doesn-t-start-status-1-failure-tls-x-pack/123498/9 "2018-03-12T22:46:25Z")
> [@manst](#): > > ```auto > Caused by: java.nio.file.AccessDeniedException: /etc/elasticsearch/certs/master.p12 > at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) ~[?:?] > > ``` This is the key part of the log. What are the permissions on that file? ---
**Author:** ![manst](https://avatars.discourse-cdn.com/v4/letter/m/ed655f/32.png) [@manst](https://discuss.elastic.co/u/manst)\ **Post date:** [March 13, 2018, 6:31pm UTC](https://discuss.elastic.co/t/elasticsearch-doesn-t-start-status-1-failure-tls-x-pack/123498/10 "2018-03-13T18:31:34Z")
`-rw------- 1 root elasticsearch` I've changed permission to rx and java.nio.file.AccessDeniedException - disappeared, but it didn't help - elastic doesn't start ---
**Author:** ![manst](https://avatars.discourse-cdn.com/v4/letter/m/ed655f/32.png) [@manst](https://discuss.elastic.co/u/manst)\ **Post date:** [March 13, 2018, 6:47pm UTC](https://discuss.elastic.co/t/elasticsearch-doesn-t-start-status-1-failure-tls-x-pack/123498/11 "2018-03-13T18:47:50Z")
``` /var/log/elasticsearch# cat my-application.log [2018-03-13T21:43:26,626][INFO][o.e.n.Node] [master] initializing ... [2018-03-13T21:43:26,734][INFO][o.e.e.NodeEnvironment] [master] using [1] data paths, mounts [[/ (/dev/sda1)]], net usable_space [8.7gb], net total_space [11.9gb], types [btrfs] [2018-03-13T21:43:26,735][INFO][o.e.e.NodeEnvironment] [master] heap size [247.5mb], compressed ordinary object pointers [true] [2018-03-13T21:43:26,803][INFO][o.e.n.Node] [master] node name [master], node ID [KC0RJjSzQhiagzzhHcDjag] [2018-03-13T21:43:26,804][INFO][o.e.n.Node] [master] version[6.2.2], pid[2882], build[10b1edd/2018-02-16T19:01:30.685723Z], OS[Linux/4.9.0-6-amd64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_151/25.151-b12] [2018-03-13T21:43:26,804][INFO][o.e.n.Node] [master] JVM arguments [-Xms256m, -Xmx256m, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.FaQnYx8c, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/elasticsearch, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:/var/log/elasticsearch/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/etc/elasticsearch] [2018-03-13T21:43:29,633][INFO][o.e.p.PluginsService] [master] loaded module [aggs-matrix-stats] [2018-03-13T21:43:29,633][INFO][o.e.p.PluginsService] [master] loaded module [analysis-common] [2018-03-13T21:43:29,633][INFO][o.e.p.PluginsService] [master] loaded module [ingest-common] [2018-03-13T21:43:29,633][INFO][o.e.p.PluginsService] [master] loaded module [lang-expression] [2018-03-13T21:43:29,633][INFO][o.e.p.PluginsService] [master] loaded module [lang-mustache] [2018-03-13T21:43:29,633][INFO][o.e.p.PluginsService] [master] loaded module [lang-painless] [2018-03-13T21:43:31,981][ERROR][o.e.b.Bootstrap] Exception java.lang.IllegalArgumentException: unknown setting [transport.port] did you mean any of [transport.tcp.port, transport.type]? [2018-03-13T21:43:31,987][WARN][o.e.b.ElasticsearchUncaughtExceptionHandler] [master] uncaught exception in thread [main] org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: unknown setting [transport.port] did you mean any of [transport.tcp.port, transport.type]? Caused by: java.lang.IllegalArgumentException: unknown setting [transport.port] did you mean any of [transport.tcp.port, transport.type]? root@master:/var/log/elasticsearch# ``` ---
**Author:** ![manst](https://avatars.discourse-cdn.com/v4/letter/m/ed655f/32.png) [@manst](https://discuss.elastic.co/u/manst)\ **Post date:** [March 13, 2018, 6:48pm UTC](https://discuss.elastic.co/t/elasticsearch-doesn-t-start-status-1-failure-tls-x-pack/123498/12 "2018-03-13T18:48:55Z")
``` /etc/elasticsearch# cat elasticsearch.yml # ======================== Elasticsearch Configuration ========================= # # NOTE: Elasticsearch comes with reasonable defaults for most settings. # Before you set out to tweak and tune the configuration, make sure you # understand what are you trying to accomplish and the consequences. # # The primary way of configuring a node is via this file. This template lists # the most important settings you may want to configure for a production cluster. # # Please consult the documentation for further information on configuration options: # https://www.elastic.co/guide/en/elasticsearch/reference/index.html # # ---------------------------------- Cluster ----------------------------------- # # Use a descriptive name for your cluster: # cluster.name: my-application # # ------------------------------------ Node ------------------------------------ # # Use a descriptive name for the node: # node.name: master # Add custom attributes to the node: # #node.attr.rack: r1 # # ----------------------------------- Paths ------------------------------------ # # Path to directory where to store the data (separate multiple locations by comma): # path.data: /var/lib/elasticsearch # # Path to log files: # path.logs: /var/log/elasticsearch # # ----------------------------------- Memory ----------------------------------- # # Lock the memory on startup: # #bootstrap.memory_lock: true # # Make sure that the heap size is set to about half the memory available # on the system and that the owner of the process is allowed to use this # limit. # # Elasticsearch performs poorly when the system is swapping the memory. # # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # network.host: 192.168.222.232 # # Set a custom port for HTTP: # http.port: 9200 http.host: 0.0.0.0 #####http.host: 192.168.222.232 transport.host: 192.168.222.232 transport.port: 9300 # For more information, consult the network module documentation. # # --------------------------------- Discovery ---------------------------------- # # Pass an initial list of hosts to perform discovery when new node is started: # The default list of hosts is ["127.0.0.1", "[::1]"] # discovery.zen.ping.unicast.hosts: ["192.168.222.232, 192.168.222.233:9300"] # # Prevent the "split brain" by configuring the majority of nodes (total number of master-eligible nodes / 2 + 1): # #discovery.zen.minimum_master_nodes: # # For more information, consult the zen discovery module documentation. # # ---------------------------------- Gateway ----------------------------------- # # Block initial recovery after a full cluster restart until N nodes are started: # #gateway.recover_after_nodes: 3 # # For more information, consult the gateway module documentation. # # ---------------------------------- Various ----------------------------------- # # Require explicit names when deleting indices: # #action.destructive_requires_name: true xpack.security.audit.enabled: true xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.keystore.path: certs/master.p12 xpack.security.transport.ssl.truststore.path: /etc/elasticsearch/certs/master.p12 ``` ---
**Author:** ![TimV](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/timv/32/13162_2.png) [@TimV](https://discuss.elastic.co/u/TimV)\ **Post date:** [March 14, 2018, 4:41am UTC](https://discuss.elastic.co/t/elasticsearch-doesn-t-start-status-1-failure-tls-x-pack/123498/13 "2018-03-14T04:41:49Z")
> [@manst](#): > > java.nio.file.AccessDeniedException - disappeared, but it didn't help Well, it clearly did help, because it resolved that error, and now you have a different error. > [@](#): > > ``` > unknown setting [transport.port] did you mean any of [transport.tcp.port, transport.type]? > > ``` This message spells out exactly what is wrong. `transport.port` is not a valid setting. The correct setting is `transport.tcp.port`. ---
**Author:** ![manst](https://avatars.discourse-cdn.com/v4/letter/m/ed655f/32.png) [@manst](https://discuss.elastic.co/u/manst)\ **Post date:** [March 14, 2018, 1:42pm UTC](https://discuss.elastic.co/t/elasticsearch-doesn-t-start-status-1-failure-tls-x-pack/123498/14 "2018-03-14T13:42:06Z")
> [@TimV](#): > > Well, it clearly did help, because it resolved that error, and now you have a different error. ))))) ---
**Author:** ![manst](https://avatars.discourse-cdn.com/v4/letter/m/ed655f/32.png) [@manst](https://discuss.elastic.co/u/manst)\ **Post date:** [March 14, 2018, 9:41pm UTC](https://discuss.elastic.co/t/elasticsearch-doesn-t-start-status-1-failure-tls-x-pack/123498/15 "2018-03-14T21:41:10Z")
``` /var/log/elasticsearch# curl -s 'localhost:9200/_cat/nodes?pretty' -u elastic:pass 192.168.222.232 54 89 13 0.14 0.30 0.27 mdi * master [2018-03-15T00:24:12,930][INFO][o.e.d.DiscoveryModule] [master] using discovery type [zen] [2018-03-15T00:24:13,692][INFO][o.e.n.Node] [master] initialized [2018-03-15T00:24:13,692][INFO][o.e.n.Node] [master] starting ... [2018-03-15T00:24:13,852][INFO][o.e.t.TransportService] [master] publish_address {192.168.222.232:9300}, bound_addresses {192.168.222.232:9300} [2018-03-15T00:24:13,900][INFO][o.e.b.BootstrapChecks] [master] bound or publishing to a non-loopback address, enforcing bootstrap checks [2018-03-15T00:24:14,040][WARN][o.e.d.z.UnicastZenPing] [master] failed to resolve host [192.168.222.232, 192.168.222.233:9300] java.net.UnknownHostException: 192.168.222.232, 192.168.222.233: Name or service not known at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_151] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_151] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151] [2018-03-15T00:24:17,092][INFO][o.e.c.s.MasterService] [master] zen-disco-elected-as-master ([0] nodes joined), reason: new_master {master}{KC0RJjSzQhiagzzhHcDjag}{8mMWffjgSUGmhqTBxba3Aw}{192.168.222.232}{192.168.222.232:9300}{ml.machine_memory=2101534720, ml.max_open_jobs=20, ml.enabled=true} [2018-03-15T00:24:17,098][INFO][o.e.c.s.ClusterApplierService] [master] new_master {master}{KC0RJjSzQhiagzzhHcDjag}{8mMWffjgSUGmhqTBxba3Aw}{192.168.222.232}{192.168.222.232:9300}{ml.machine_memory=2101534720, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {master}{KC0RJjSzQhiagzzhHcDjag}{8mMWffjgSUGmhqTBxba3Aw}{192.168.222.232}{192.168.222.232:9300}{ml.machine_memory=2101534720, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)]]) [2018-03-15T00:24:17,143][INFO][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [master] publish_address {192.168.222.232:9200}, bound_addresses {[::]:9200} [2018-03-15T00:24:17,143][INFO][o.e.n.Node] [master] started [2018-03-15T00:24:17,973][INFO][o.e.x.s.a.AuthenticationService] [master] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic] [2018-03-15T00:24:17,989][INFO][o.e.x.s.a.AuthenticationService] [master] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic] [2018-03-15T00:24:18,169][INFO][o.e.x.s.a.AuthenticationService] [master] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic] [2018-03-15T00:24:18,810][INFO][o.e.x.s.a.AuthenticationService] [master] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic] [2018-03-15T00:24:18,980][INFO][o.e.x.s.a.AuthenticationService] [master] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic] [2018-03-15T00:24:19,114][INFO][o.e.x.s.a.AuthenticationService] [master] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic] [2018-03-15T00:24:19,142][INFO][o.e.l.LicenseService] [master] license [5068e9b2-4634-413c-8dfe-3bfe8d47409f] mode [trial] - valid [2018-03-15T00:24:19,148][WARN][o.e.l.LicenseService] [master] # # License [will expire] on [Tuesday, April 03, 2018]. If you have a new license, please update it. # Otherwise, please reach out to your support contact. # # - upgrade # - Upgrade API is disabled [2018-03-15T00:24:19,158][INFO][o.e.g.GatewayService] [master] recovered [25] indices into cluster_state [2018-03-15T00:24:22,505][INFO][o.e.c.r.a.AllocationService] [master] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[.monitoring-es-6-2018.03.09][0], [.monitoring-kibana-6-2018.03.09][0], [.monitoring-alerts-6][0], [.watcher-history-7-2018.03.09][0]] ...]). root@master:/var/log/elasticsearch# ``` ---
**Author:** ![manst](https://avatars.discourse-cdn.com/v4/letter/m/ed655f/32.png) [@manst](https://discuss.elastic.co/u/manst)\ **Post date:** [March 15, 2018, 6:46pm UTC](https://discuss.elastic.co/t/elasticsearch-doesn-t-start-status-1-failure-tls-x-pack/123498/16 "2018-03-15T18:46:23Z")
I've seeing error and corrected it [quote="manst, post:12, topic:123498"] discovery.zen.ping.unicast.hosts: ["192.168.222.232:9300", "192.168.222.233:9300"] Now my first node see the second. And next bulb of errors: ``` [INFO][o.e.d.z.ZenDiscovery] [master] failed to send join request to master [{data}{KC0RJjSzQhiagzzhHcDjag}{kC1Lw4_2RV6H8ebfXMgXGw}{192.168.222.233}{192.168.222.233:9300}{ml.machine_memory=1722236928, ml.max_open_jobs=20, ml.enabled=true}], reason [RemoteTransportException[[data][192.168.222.233:9300][internal:discovery/zen/join]]; nested: IllegalArgumentException[can't add node {master}{KC0RJjSzQhiagzzhHcDjag}{0lx-KxDlS5iVU-TI1xhMew}{192.168.222.232}{192.168.222.232:9300}{ml.machine_memory=2101534720, ml.max_open_jobs=20, ml.enabled=true}, found existing node {data}{KC0RJjSzQhiagzzhHcDjag}{kC1Lw4_2RV6H8ebfXMgXGw}{192.168.222.233}{192.168.222.233:9300}{ml.machine_memory=1722236928, ml.max_open_jobs=20, ml.enabled=true} with the same id but is a different node instance]; ] ``` solution for this error (you must deleted contents of data folder(/var/lib/elasticsearch/nodes/0) and restarted both the servers ): > [@Setting up elastic search master and data node configuration in two different servers](https://discuss.elastic.co/t/setting-up-elastic-search-master-and-data-node-configuration-in-two-different-servers/98012/2): > > I have got the solution. As I installed through rpm packages in the logs I was getting below issue: [2017-08-24T03:50:01,331][INFO][o.e.d.z.ZenDiscovery] [node-2] failed to send join request to master [{node-1}{sYqhGZc4QyK\_cbZ9\_cz8gQ}{BBHUedE6Roaw0AMiYnQwIg}{172.31.4.119}{172.31.4.119:9300}], reason [RemoteTransportException[[node-1][172.31.4.119:9300][internal:discovery/zen/join]]; nested: IllegalArgumentException[can't add node {node-2}{sYqhGZc4QyK\_cbZ9\_cz8gQ}{FbDpyWy2S7SS1c8TvLsnfQ}{17… Than I should change passwords , because i've had error: ``` /var/log/elasticsearch# curl -s 'localhost:9200/_cat/nodes?pretty' -u elastic:pass { "error" : { "root_cause" : [ { "type" : "security_exception", "reason" : "failed to authenticate user [elastic]", "header" : { "WWW-Authenticate" : "Basic realm=\"security\" charset=\"UTF-8\"" } } ], "type" : "security_exception", "reason" : "failed to authenticate user [elastic]", "header" : { "WWW-Authenticate" : "Basic realm=\"security\" charset=\"UTF-8\"" } }, "status" : 401 ``` ---
**Author:** ![manst](https://avatars.discourse-cdn.com/v4/letter/m/ed655f/32.png) [@manst](https://discuss.elastic.co/u/manst)\ **Post date:** [March 15, 2018, 7:26pm UTC](https://discuss.elastic.co/t/elasticsearch-doesn-t-start-status-1-failure-tls-x-pack/123498/17 "2018-03-15T19:26:12Z")
**Urra))))):** ``` /var/log/elasticsearch# curl -s 'localhost:9200/_cat/nodes?pretty' -u elastic:pass 192.168.222.233 74 45 22 0.24 0.33 0.26 mdi * data 192.168.222.232 55 62 5 0.18 0.14 0.13 mdi - master ``` Thanks everyone a lot for your help. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [April 12, 2018, 7:27pm UTC](https://discuss.elastic.co/t/elasticsearch-doesn-t-start-status-1-failure-tls-x-pack/123498/18 "2018-04-12T19:27:00Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.