Elastic search does not work during operation, causing problems in restarting

[2023-02-02T18:22:39,230][INFO ][o.e.e.NodeEnvironment    ] [name] using [1] data paths, mounts [[/data (/dev/data)]], net usable_space [55.9gb], net total_space [98.4gb], types [ext4]
[2023-02-02T18:22:39,231][INFO ][o.e.e.NodeEnvironment    ] [name] heap size [2gb], compressed ordinary object pointers [true]
[2023-02-02T18:22:39,352][INFO ][o.e.n.Node               ] [name] node name [name], node ID [NSSQNNryRkeZsMxOuFd7gQ], cluster name [cluster_name], roles [transform, data_frozen, master, remote_cluster_client, data, ml, data_content, data_hot, data_warm, data_cold, ingest]
[2023-02-02T18:22:43,029][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [name] [controller/26337] [Main.cc@122] controller (64 bit): Version 7.16.3 (Build b71e71b93f3e4d) Copyright (c) 2021 Elasticsearch BV
[2023-02-02T18:22:43,529][INFO ][o.e.x.s.a.Realms         ] [name] license mode is [trial], currently licensed security realms are [reserved/reserved,file/default_file,native/default_native]
[2023-02-02T18:22:43,536][INFO ][o.e.x.s.a.s.FileRolesStore] [name] parsed [0] roles from file [/data/elasticsearch/config/roles.yml]
[2023-02-02T18:22:43,830][ERROR][o.e.b.Bootstrap          ] [name] Exception
java.io.UncheckedIOException: Failed to create persistent cache writers
    at org.elasticsearch.xpack.searchablesnapshots.cache.full.PersistentCache.createWriters(PersistentCache.java:346) ~[?:?]
    at org.elasticsearch.xpack.searchablesnapshots.cache.full.PersistentCache.<init>(PersistentCache.java:100) ~[?:?]
    at org.elasticsearch.xpack.searchablesnapshots.SearchableSnapshots.createComponents(SearchableSnapshots.java:333) ~[?:?]
    at org.elasticsearch.node.Node.lambda$new$18(Node.java:733) ~[elasticsearch-7.16.3.jar:7.16.3]
    at java.util.stream.ReferencePipeline$7$1.accept(Unknown Source) ~[?:?]
    at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source) ~[?:?]
    at java.util.stream.AbstractPipeline.copyInto(Unknown Source) ~[?:?]
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source) ~[?:?]
    at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source) ~[?:?]
    at java.util.stream.AbstractPipeline.evaluate(Unknown Source) ~[?:?]
    at java.util.stream.ReferencePipeline.collect(Unknown Source) ~[?:?]
    at org.elasticsearch.node.Node.<init>(Node.java:747) ~[elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.node.Node.<init>(Node.java:309) ~[elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:234) ~[elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:234) ~[elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:434) [elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:166) [elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:157) [elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:77) [elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112) [elasticsearch-cli-7.16.3.jar:7.16.3]
    at org.elasticsearch.cli.Command.main(Command.java:77) [elasticsearch-cli-7.16.3.jar:7.16.3]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:122) [elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80) [elasticsearch-7.16.3.jar:7.16.3]
Caused by: org.apache.lucene.store.LockObtainFailedException: Lock held by another program: /data/elasticsearch/data/nodes/0/snapshot_cache/write.lock
    at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:130) ~[lucene-core-8.10.1.jar:8.10.1 2f24e6a49d48a032df1f12e146612f59141727a9 - mayyasharipova - 2021-10-12 15:13:05]
    at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41) ~[lucene-core-8.10.1.jar:8.10.1 2f24e6a49d48a032df1f12e146612f59141727a9 - mayyasharipova - 2021-10-12 15:13:05]
    at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45) ~[lucene-core-8.10.1.jar:8.10.1 2f24e6a49d48a032df1f12e146612f59141727a9 - mayyasharipova - 2021-10-12 15:13:05]
    at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:923) ~[lucene-core-8.10.1.jar:8.10.1 2f24e6a49d48a032df1f12e146612f59141727a9 - mayyasharipova - 2021-10-12 15:13:05]
    at org.elasticsearch.xpack.searchablesnapshots.cache.full.PersistentCache.createCacheIndexWriter(PersistentCache.java:377) ~[?:?]
    at org.elasticsearch.xpack.searchablesnapshots.cache.full.PersistentCache.createWriters(PersistentCache.java:342) ~[?:?]
    ... 22 more
[2023-02-02T18:22:43,835][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [name] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.io.UncheckedIOException: Failed to create persistent cache writers
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:157) ~[elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:77) ~[elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:112) ~[elasticsearch-cli-7.16.3.jar:7.16.3]
    at org.elasticsearch.cli.Command.main(Command.java:77) ~[elasticsearch-cli-7.16.3.jar:7.16.3]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:122) ~[elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:80) ~[elasticsearch-7.16.3.jar:7.16.3]
Caused by: java.io.UncheckedIOException: Failed to create persistent cache writers
    at org.elasticsearch.xpack.searchablesnapshots.cache.full.PersistentCache.createWriters(PersistentCache.java:346) ~[?:?]
    at org.elasticsearch.xpack.searchablesnapshots.cache.full.PersistentCache.<init>(PersistentCache.java:100) ~[?:?]
    at org.elasticsearch.xpack.searchablesnapshots.SearchableSnapshots.createComponents(SearchableSnapshots.java:333) ~[?:?]
    at org.elasticsearch.node.Node.lambda$new$18(Node.java:733) ~[elasticsearch-7.16.3.jar:7.16.3]
    at java.util.stream.ReferencePipeline$7$1.accept(Unknown Source) ~[?:?]
    at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source) ~[?:?]
    at java.util.stream.AbstractPipeline.copyInto(Unknown Source) ~[?:?]
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source) ~[?:?]
    at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source) ~[?:?]
    at java.util.stream.AbstractPipeline.evaluate(Unknown Source) ~[?:?]
    at java.util.stream.ReferencePipeline.collect(Unknown Source) ~[?:?]
    at org.elasticsearch.node.Node.<init>(Node.java:747) ~[elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.node.Node.<init>(Node.java:309) ~[elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:234) ~[elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:234) ~[elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:434) ~[elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:166) ~[elasticsearch-7.16.3.jar:7.16.3]
    ... 6 more
Caused by: org.apache.lucene.store.LockObtainFailedException: Lock held by another program: /data/elasticsearch/data/nodes/0/snapshot_cache/write.lock
    at org.apache.lucene.store.NativeFSLockFactory.obtainFSLock(NativeFSLockFactory.java:130) ~[lucene-core-8.10.1.jar:8.10.1 2f24e6a49d48a032df1f12e146612f59141727a9 - mayyasharipova - 2021-10-12 15:13:05]
    at org.apache.lucene.store.FSLockFactory.obtainLock(FSLockFactory.java:41) ~[lucene-core-8.10.1.jar:8.10.1 2f24e6a49d48a032df1f12e146612f59141727a9 - mayyasharipova - 2021-10-12 15:13:05]
    at org.apache.lucene.store.BaseDirectory.obtainLock(BaseDirectory.java:45) ~[lucene-core-8.10.1.jar:8.10.1 2f24e6a49d48a032df1f12e146612f59141727a9 - mayyasharipova - 2021-10-12 15:13:05]
    at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:923) ~[lucene-core-8.10.1.jar:8.10.1 2f24e6a49d48a032df1f12e146612f59141727a9 - mayyasharipova - 2021-10-12 15:13:05]
    at org.elasticsearch.xpack.searchablesnapshots.cache.full.PersistentCache.createCacheIndexWriter(PersistentCache.java:377) ~[?:?]
    at org.elasticsearch.xpack.searchablesnapshots.cache.full.PersistentCache.createWriters(PersistentCache.java:342) ~[?:?]
    at org.elasticsearch.xpack.searchablesnapshots.cache.full.PersistentCache.<init>(PersistentCache.java:100) ~[?:?]
    at org.elasticsearch.xpack.searchablesnapshots.SearchableSnapshots.createComponents(SearchableSnapshots.java:333) ~[?:?]
    at org.elasticsearch.node.Node.lambda$new$18(Node.java:733) ~[elasticsearch-7.16.3.jar:7.16.3]
    at java.util.stream.ReferencePipeline$7$1.accept(Unknown Source) ~[?:?]
    at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown Source) ~[?:?]
    at java.util.stream.AbstractPipeline.copyInto(Unknown Source) ~[?:?]
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source) ~[?:?]
    at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source) ~[?:?]
    at java.util.stream.AbstractPipeline.evaluate(Unknown Source) ~[?:?]
    at java.util.stream.ReferencePipeline.collect(Unknown Source) ~[?:?]
    at org.elasticsearch.node.Node.<init>(Node.java:747) ~[elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.node.Node.<init>(Node.java:309) ~[elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:234) ~[elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:234) ~[elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:434) ~[elasticsearch-7.16.3.jar:7.16.3]
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:166) ~[elasticsearch-7.16.3.jar:7.16.3]
    ... 6 more
[2023-02-02T18:22:43,869][INFO ][o.e.x.m.p.NativeController] [name] Native controller process has stopped - no new native processes can be started

The first log I found is as above. I tried everything I found on Google to solve this problem, but it failed.

[2023-02-02T21:00:36,072][INFO ][o.e.e.NodeEnvironment    ] [name] using [1] data paths, mounts [[/data (/dev/data)]], net usable_space [55.6gb], net total_space [98.4gb], types [ext4]
[2023-02-02T21:00:36,072][INFO ][o.e.e.NodeEnvironment    ] [name] heap size [2gb], compressed ordinary object pointers [true]
[2023-02-02T21:00:36,116][INFO ][o.e.n.Node               ] [name] node name [name], node ID [ljP7-iEISP2bDFseryxhhA], cluster name [cluster_name], roles [transform, data_frozen, master, remote_cluster_client, data, ml, data_content, data_hot, data_warm, data_cold, ingest]
[2023-02-02T21:00:39,781][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [name] [controller/30735] [Main.cc@122] controller (64 bit): Version 7.16.3 (Build b71e71b93f3e4d) Copyright (c) 2021 Elasticsearch BV
[2023-02-02T21:00:40,136][INFO ][o.e.x.s.a.Realms         ] [name] license mode is [trial], currently licensed security realms are [reserved/reserved,file/default_file,native/default_native]
[2023-02-02T21:00:40,145][INFO ][o.e.x.s.a.s.FileRolesStore] [name] parsed [0] roles from file [/data/elasticsearch/config/roles.yml]
[2023-02-02T21:00:40,690][INFO ][o.e.i.g.ConfigDatabases  ] [name] initialized default databases [[GeoLite2-Country.mmdb, GeoLite2-City.mmdb, GeoLite2-ASN.mmdb]], config databases [[]] and watching [/data/elasticsearch/config/ingest-geoip] for changes
[2023-02-02T21:00:40,691][INFO ][o.e.i.g.DatabaseNodeService] [name] initialized database registry, using geoip-databases directory [/tmp/elasticsearch-16478923630485953060/geoip-databases/ljP7-iEISP2bDFseryxhhA]
[2023-02-02T21:00:41,185][INFO ][o.e.t.NettyAllocator     ] [name] creating NettyAllocator with the following configs: [name=elasticsearch_configured, chunk_size=1mb, suggested_max_allocation_size=1mb, factors={es.unsafe.use_netty_default_chunk_and_page_size=false, g1gc_enabled=true, g1gc_region_size=4mb}]
[2023-02-02T21:00:41,241][INFO ][o.e.d.DiscoveryModule    ] [name] using discovery type [zen] and seed hosts providers [settings]
[2023-02-02T21:00:41,647][INFO ][o.e.g.DanglingIndicesState] [name] gateway.auto_import_dangling_indices is disabled, dangling indices will not be automatically detected or imported and must be managed manually
[2023-02-02T21:00:42,081][INFO ][o.e.n.Node               ] [name] initialized
[2023-02-02T21:00:42,082][INFO ][o.e.n.Node               ] [name] starting ...
[2023-02-02T21:00:42,113][INFO ][o.e.x.s.c.f.PersistentCache] [name] persistent cache index loaded
[2023-02-02T21:00:42,115][INFO ][o.e.x.d.l.DeprecationIndexingComponent] [name] deprecation component started
[2023-02-02T21:00:42,193][INFO ][o.e.t.TransportService   ] [name] publish_address {127.0.0.1:9300}, bound_addresses {127.0.0.1:9300}
[2023-02-02T21:00:42,454][WARN ][o.e.b.BootstrapChecks    ] [name] memory locking requested for elasticsearch process but memory is not locked
[2023-02-02T21:00:42,456][INFO ][o.e.c.c.Coordinator      ] [name] cluster UUID [eB0SDrwlSP-905wyIowBFQ]
[2023-02-02T21:00:42,578][INFO ][o.e.c.s.MasterService    ] [name] elected-as-master ([1] nodes joined)[{name}{ljP7-iEISP2bDFseryxhhA}{uuqbJScFQsWlGI-8wfv6qA}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 3, version: 63, delta: master node changed {previous [], current [{name}{ljP7-iEISP2bDFseryxhhA}{uuqbJScFQsWlGI-8wfv6qA}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}]}
[2023-02-02T21:00:42,650][INFO ][o.e.c.s.ClusterApplierService] [name] master node changed {previous [], current [{name}{ljP7-iEISP2bDFseryxhhA}{uuqbJScFQsWlGI-8wfv6qA}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}]}, term: 3, version: 63, reason: Publication{term=3, version=63}
[2023-02-02T21:00:42,698][INFO ][o.e.h.AbstractHttpServerTransport] [name] publish_address {127.0.0.1:9200}, bound_addresses {127.0.0.1:9200}
[2023-02-02T21:00:42,699][INFO ][o.e.n.Node               ] [name] started
[2023-02-02T21:00:42,971][INFO ][o.e.l.LicenseService     ] [name] license [9903a757-0105-4659-8056-6974082ed965] mode [basic] - valid
[2023-02-02T21:00:42,972][INFO ][o.e.x.s.a.Realms         ] [name] license mode is [basic], currently licensed security realms are [reserved/reserved,file/default_file,native/default_native]
[2023-02-02T21:00:42,973][INFO ][o.e.x.s.s.SecurityStatusChangeListener] [name] Active license is now [BASIC]; Security is enabled
[2023-02-02T21:00:42,976][INFO ][o.e.g.GatewayService     ] [name] recovered [2] indices into cluster_state
[2023-02-02T21:00:43,346][INFO ][o.e.c.r.a.AllocationService] [name] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[.ds-.logs-deprecation.elasticsearch-default-2023.02.02-000001][0], [.ds-ilm-history-5-2023.02.02-000001][0]]]).

Afterwards, I deleted and reinstalled the Elasticsearch, and the logs I found were as follows.
It also tried everything, but it didn't succeed.

Thereafter, two logs are generated alternately.
I wanted to replace the hard disk, but I couldn't because I couldn't do that.

Please let me know if the hard disk is a problem or if there is another solution.

I think this is the problem

memory locking requested for elasticsearch process but memory is not locked

Is this option turned on? You'll have to see why the memory lock isn't working in your environment.

cluster.name: cluster_name
node.name: name
path.data: /data/elasticsearch/data
path.logs: /data/elasticsearch/logs
bootstrap.memory_lock: true
network.host: 0.0.0.0
http.port: 9200
discovery.seed_hosts: ["0.0.0.0"]
cluster.initial_master_nodes: ["name"]
ingest.geoip.downloader.enabled: false

xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true

My settings are as follows.
I checked if there is a problem with the license, but it is working as basic.

And I have one more piece of equipment of the same spec and it's working well there.

From the second logs, it looks like ES was up and running and status is green, what is the issue there (after re-installation) ?

Lock held by another program: /data/elasticsearch/data/nodes/0/snapshot_cache/write.lock

From the first log, it looks like there was some snapshot in progress which was holding the lock, have you tried deleting that lock file post restarting ES ?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.