Elasticsearch has stopped and no longer works

Hi guys
I was using elastic search (with the nextcloud), but the elastic just stopped after a while.
Try to install other versions or reinstall the same version, but it doesn't work.
Can you help me? Thank you very much!

CentOS Linux release 7.7.1908 (Core)
Php 7.3.13
Mysql 5.7.29
Nextcloud 17.0.2

elasticsearch-6.8.6.rpm
readonlyrest-1.18.10_es6.8.6.zip

[root@nextcloud ~]# sudo -u apache /usr/bin/php /var/www/html/occ fulltextsearch:check
Full text search 1.3.5

- Search Platform:
Elasticsearch 1.4.0
{
    "elastic_host": [
        "http://nextcloud:********@localhost:9200"
    ],
    "elastic_index": "nextcloud",
    "fields_limit": "10000",
    "es_ver_below66": "0",
    "analyzer_tokenizer": "standard"
}

- Content Providers:
Files 1.3.6
{
    "files_local": "1",
    "files_external": "2",
    "files_group_folders": "1",
    "files_encrypted": "0",
    "files_federated": "0",
    "files_size": "20",
    "files_pdf": "1",
    "files_office": "1",
    "files_image": "0",
    "files_audio": "0"
}
[root@nextcloud ~]#

FILE /etc/elasticsearch/readonlyrest.yml
readonlyrest:
access_controle_rules:
name: "Require HTTP Basic Auth"
type: allow
auth_key: nextcloud:nextcloud
indices: ["nextcloud"]

ADD /etc/elasticsearch/elasticsearch.yml
xpack.security.enabled: false

Elasticsearch service FAILED after 10 seconds.

[root@nextcloud ~]# systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Qui 2020-01-16 09:26:24 -03; 27s ago
     Docs: http://www.elastic.co
  Process: 1305 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=1/FAILURE)
 Main PID: 1305 (code=exited, status=1/FAILURE)

Jan 16 09:28:01 nextcloud.local systemd[1]: Started Elasticsearch.
Jan 16 09:26:24 nextcloud.local systemd[1]: elasticsearch.service: main process exited, code=exited, status=1/FAILURE
Jan 16 09:26:24 nextcloud.local systemd[1]: Unit elasticsearch.service entered failed state.
Jan 16 09:26:24 nextcloud.local systemd[1]: elasticsearch.service failed.
[root@nextcloud ~]#

I've tested several versions, updated everything, searched several blogs, but I can't solve the problem. It is really frustrating.

Can any kindness help me? :neutral_face:

Please share what is in the Elasticsearch logs.

1 Like
[2020-01-20T06:24:52,153][INFO ][o.e.e.NodeEnvironment    ] [FItIT-1] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [440gb], net total_space [469.7gb], types [rootfs]
[2020-01-20T06:24:52,159][INFO ][o.e.e.NodeEnvironment    ] [FItIT-1] heap size [990.7mb], compressed ordinary object pointers [true]
[2020-01-20T06:24:52,162][INFO ][o.e.n.Node               ] [FItIT-1] node name derived from node ID [FItIT-1JRt-j8Bcnv5kRwQ]; set [node.name] to override
[2020-01-20T06:24:52,162][INFO ][o.e.n.Node               ] [FItIT-1] version[6.8.6], pid[23041], build[default/rpm/3d9f765/2019-12-13T17:11:52.013738Z], OS[Linux/3.10.0-1062.9.1.el7.x86_64/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_232/25.232-b09]
[2020-01-20T06:24:52,162][INFO ][o.e.n.Node               ] [FItIT-1] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-2452565900209370013, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/elasticsearch, -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:/var/log/elasticsearch/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/etc/elasticsearch, -Des.distribution.flavor=default, -Des.distribution.type=rpm]
[2020-01-20T06:24:58,641][INFO ][t.b.r.c.RorSsl$          ] [FItIT-1] Cannot find SSL configuration is elasticsearch.yml, trying: /etc/elasticsearch/readonlyrest.yml
[2020-01-20T06:24:58,809][ERROR][o.e.b.Bootstrap          ] [FItIT-1] Exception
java.lang.IllegalStateException: failed to load plugin class [tech.beshu.ror.es.ReadonlyRestPlugin]
	at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:614) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:556) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:471) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:163) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.node.Node.<init>(Node.java:339) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.node.Node.<init>(Node.java:266) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) [elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) [elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) [elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-6.8.6.jar:6.8.6]
	at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) [elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) [elasticsearch-6.8.6.jar:6.8.6]
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_232]
	at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:605) ~[elasticsearch-6.8.6.jar:6.8.6]
	... 15 more
Caused by: org.elasticsearch.ElasticsearchException: Cannot parse file /etc/elasticsearch/readonlyrest.yml content. Cause: mapping values are not allowed here
 in 'reader', line 6, column 16:
           auth_key: nextcloud:nextcloud
                   ^

	at tech.beshu.ror.es.ReadonlyRestPlugin.lambda$null$1(ReadonlyRestPlugin.java:119) ~[?:?]
	at tech.beshu.ror.utils.ScalaJavaHelper$.$anonfun$getOrElse$1(ScalaJavaHelper.scala:35) ~[?:?]
	at scala.util.Either.fold(Either.scala:192) ~[?:?]
	at tech.beshu.ror.utils.ScalaJavaHelper$.getOrElse(ScalaJavaHelper.scala:36) ~[?:?]
	at tech.beshu.ror.es.ReadonlyRestPlugin.lambda$new$2(ReadonlyRestPlugin.java:119) ~[?:?]
	at monix.eval.Task$Map.apply(Task.scala:4514) ~[?:?]
	at monix.eval.Task$Map.apply(Task.scala:4510) ~[?:?]
	at monix.eval.internal.TaskRunSyncUnsafe$.apply(TaskRunSyncUnsafe.scala:107) ~[?:?]
	at monix.eval.Task.runSyncUnsafeOpt(Task.scala:1096) ~[?:?]
	at monix.eval.Task.runSyncUnsafe(Task.scala:1063) ~[?:?]
	at tech.beshu.ror.es.ReadonlyRestPlugin.<init>(ReadonlyRestPlugin.java:120) ~[?:?]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_232]
	at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:605) ~[elasticsearch-6.8.6.jar:6.8.6]
	... 15 more
[2020-01-20T06:24:58,826][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [FItIT-1] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: failed to load plugin class [tech.beshu.ror.es.ReadonlyRestPlugin]
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:163) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.8.6.jar:6.8.6]
	at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) ~[elasticsearch-6.8.6.jar:6.8.6]
Caused by: java.lang.IllegalStateException: failed to load plugin class [tech.beshu.ror.es.ReadonlyRestPlugin]
	at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:614) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:556) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:471) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:163) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.node.Node.<init>(Node.java:339) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.node.Node.<init>(Node.java:266) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.8.6.jar:6.8.6]
	... 6 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_232]
	at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:605) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:556) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:471) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:163) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.node.Node.<init>(Node.java:339) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.node.Node.<init>(Node.java:266) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.8.6.jar:6.8.6]
	... 6 more
Caused by: org.elasticsearch.ElasticsearchException: Cannot parse file /etc/elasticsearch/readonlyrest.yml content. Cause: mapping values are not allowed here
 in 'reader', line 6, column 16:
           auth_key: nextcloud:nextcloud
                   ^

	at tech.beshu.ror.es.ReadonlyRestPlugin.lambda$null$1(ReadonlyRestPlugin.java:119) ~[?:?]
	at tech.beshu.ror.utils.ScalaJavaHelper$.$anonfun$getOrElse$1(ScalaJavaHelper.scala:35) ~[?:?]
	at scala.util.Either.fold(Either.scala:192) ~[?:?]
	at tech.beshu.ror.utils.ScalaJavaHelper$.getOrElse(ScalaJavaHelper.scala:36) ~[?:?]
	at tech.beshu.ror.es.ReadonlyRestPlugin.lambda$new$2(ReadonlyRestPlugin.java:119) ~[?:?]
	at monix.eval.Task$Map.apply(Task.scala:4514) ~[?:?]
	at monix.eval.Task$Map.apply(Task.scala:4510) ~[?:?]
	at monix.eval.internal.TaskRunSyncUnsafe$.apply(TaskRunSyncUnsafe.scala:107) ~[?:?]
	at monix.eval.Task.runSyncUnsafeOpt(Task.scala:1096) ~[?:?]
	at monix.eval.Task.runSyncUnsafe(Task.scala:1063) ~[?:?]
	at tech.beshu.ror.es.ReadonlyRestPlugin.<init>(ReadonlyRestPlugin.java:120) ~[?:?]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:?]
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_232]
	at org.elasticsearch.plugins.PluginsService.loadPlugin(PluginsService.java:605) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.plugins.PluginsService.loadBundle(PluginsService.java:556) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.plugins.PluginsService.loadBundles(PluginsService.java:471) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.plugins.PluginsService.<init>(PluginsService.java:163) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.node.Node.<init>(Node.java:339) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.node.Node.<init>(Node.java:266) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:212) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:212) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) ~[elasticsearch-6.8.6.jar:6.8.6]
	at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) ~[elasticsearch-6.8.6.jar:6.8.6]
	... 6 more

I think the error message is pretty clear, no?

Cannot parse file /etc/elasticsearch/readonlyrest.yml content. Cause: mapping values are not allowed here
in 'reader', line 6, column 16:
          auth_key: nextcloud:nextcloud
                  ^

You are using here a third party plugin ReadonlyRestPlugin which is not supported. You should either use the official package and its security features or ask the authors of the plugin you are using for help.

My 2 cents.

May be you should try though:

auth_key: "nextcloud:nextcloud"
1 Like

Hi David, how are you? Hope so.

I published the logs and then realized where the error was: there were two "spaces" in readonlyrest.yml, just where you pointed out.

Now it's running.

Can you help me with one more thing?

After indexing everything, it presented error 82/82 files.
How can I list all of these files?

I have no idea of what it means...

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.