My problem is that ELasticsearch won't start, so i checked log /var/log/elasticsearch/elasticsearch.log and I Got an error concerning 'Lucene_87 missing'
Caused by: java.lang.IllegalArgumentException: An SPI class of type org.apache.lucene.codecs.Codec with name 'Lucene87' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath. The current classpath supports the following names: [Lucene70, Lucene80, Lucene84, Lucene86]
at org.apache.lucene.util.NamedSPILoader.lookup(NamedSPILoader.java:116) ~[lucene-core-8.6.2.jar:8.6.2 016993b65e393b58246d54e8ddda9f56a453eb0e - ivera - 2020-08-26 10:53:36]
at org.apache.lucene.codecs.Codec.forName(Codec.java:116) ~[lucene-core-8.6.2.jar:8.6.2 016993b65e393b58246d54e8ddda9f56a453eb0e - ivera - 2020-08-26 10:53:36]
at org.apache.lucene.index.SegmentInfos.readCodec(SegmentInfos.java:445) ~[lucene-core-8.6.2.jar:8.6.2 016993b65e393b58246d54e8ddda9f56a453eb0e - ivera - 2020-08-26 10:53:36]
at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:356) ~[lucene-core-8.6.2.jar:8.6.2 016993b65e393b58246d54e8ddda9f56a453eb0e - ivera - 2020-08-26 10:53:36]
at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:291) ~[lucene-core-8.6.2.jar:8.6.2 016993b65e393b58246d54e8ddda9f56a453eb0e - ivera - 2020-08-26 10:53:36]
at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:61) ~[lucene-core-8.6.2.jar:8.6.2 016993b65e393b58246d54e8ddda9f56a453eb0e - ivera - 2020-08-26 10:53:36]
at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:58) ~[lucene-core-8.6.2.jar:8.6.2 016993b65e393b58246d54e8ddda9f56a453eb0e - ivera - 2020-08-26 10:53:36]
at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:720) ~[lucene-core-8.6.2.jar:8.6.2 016993b65e393b58246d54e8ddda9f56a453eb0e - ivera - 2020-08-26 10:53:36]
at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:81) ~[lucene-core-8.6.2.jar:8.6.2 016993b65e393b58246d54e8ddda9f56a453eb0e - ivera - 2020-08-26 10:53:36]
at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63) ~[lucene-core-8.6.2.jar:8.6.2 016993b65e393b58246d54e8ddda9f56a453eb0e - ivera - 2020-08-26 10:53:36]
at org.elasticsearch.gateway.PersistedClusterStateService.nodeMetadata(PersistedClusterStateService.java:262) ~[elasticsearch-7.9.1.jar:7.9.1]
at org.elasticsearch.env.NodeEnvironment.loadNodeMetadata(NodeEnvironment.java:408) ~[elasticsearch-7.9.1.jar:7.9.1]
at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:329) ~[elasticsearch-7.9.1.jar:7.9.1]
at org.elasticsearch.node.Node.<init>(Node.java:344) ~[elasticsearch-7.9.1.jar:7.9.1]
at org.elasticsearch.node.Node.<init>(Node.java:277) ~[elasticsearch-7.9.1.jar:7.9.1]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:227) ~[elasticsearch-7.9.1.jar:7.9.1]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:227) ~[elasticsearch-7.9.1.jar:7.9.1]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393) ~[elasticsearch-7.9.1.jar:7.9.1]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) ~[elasticsearch-7.9.1.jar:7.9.1]
and this is the Log under journalctl -xe :
Jul 15 14:28:36 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:28:36Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"No living connections"}
Jul 15 14:28:38 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:28:38Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"Unable to revive connection: http://172.31.29.10:9200/"}
Jul 15 14:28:38 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:28:38Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"No living connections"}
Jul 15 14:28:41 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:28:41Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"Unable to revive connection: http://172.31.29.10:9200/"}
Jul 15 14:28:41 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:28:41Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"No living connections"}
Jul 15 14:28:42 srv-wazuh filebeat[705]: 2021-07-15T14:28:42.567Z INFO [monitoring] log/log.go:144 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":5040,"time":{"ms":5}},"total":{"ticks":17190,"time":{"ms":45},"value":17190},"user":>
Jul 15 14:28:43 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:28:43Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"Unable to revive connection: http://172.31.29.10:9200/"}
Jul 15 14:28:43 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:28:43Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"No living connections"}
Jul 15 14:28:46 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:28:46Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"Unable to revive connection: http://172.31.29.10:9200/"}
Jul 15 14:28:46 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:28:46Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"No living connections"}
Jul 15 14:28:48 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:28:48Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"Unable to revive connection: http://172.31.29.10:9200/"}
Jul 15 14:28:48 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:28:48Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"No living connections"}
Jul 15 14:28:51 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:28:51Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"Unable to revive connection: http://172.31.29.10:9200/"}
Jul 15 14:28:51 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:28:51Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"No living connections"}
Jul 15 14:28:52 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:28:52Z","tags":["warning","elasticsearch","monitoring"],"pid":7255,"message":"Unable to revive connection: http://172.31.29.10:9200/"}
Jul 15 14:28:52 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:28:52Z","tags":["warning","elasticsearch","monitoring"],"pid":7255,"message":"No living connections"}
Jul 15 14:28:52 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:28:52Z","tags":["warning","plugins","licensing"],"pid":7255,"message":"License information could not be obtained from Elasticsearch due to Error: No Living connections error"}
Jul 15 14:28:53 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:28:53Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"Unable to revive connection: http://172.31.29.10:9200/"}
Jul 15 14:28:53 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:28:53Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"No living connections"}
Jul 15 14:28:56 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:28:56Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"Unable to revive connection: http://172.31.29.10:9200/"}
Jul 15 14:28:56 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:28:56Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"No living connections"}
Jul 15 14:28:58 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:28:58Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"Unable to revive connection: http://172.31.29.10:9200/"}
Jul 15 14:28:58 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:28:58Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"No living connections"}
Jul 15 14:29:01 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:29:01Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"Unable to revive connection: http://172.31.29.10:9200/"}
Jul 15 14:29:01 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:29:01Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"No living connections"}
Jul 15 14:29:03 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:29:03Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"Unable to revive connection: http://172.31.29.10:9200/"}
Jul 15 14:29:03 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:29:03Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"No living connections"}
Jul 15 14:29:04 srv-wazuh filebeat[705]: 2021-07-15T14:29:04.051Z ERROR [publisher_pipeline_output] pipeline/output.go:154 Failed to connect to backoff(elasticsearch(http://172.31.29.10:9200)): Get "http://172.31.29.10:9200": dial tcp 172.31.29.10:9200: connect: connection refused
Jul 15 14:29:04 srv-wazuh filebeat[705]: 2021-07-15T14:29:04.052Z INFO [publisher_pipeline_output] pipeline/output.go:145 Attempting to reconnect to backoff(elasticsearch(http://172.31.29.10:9200)) with 480 reconnect attempt(s)
Jul 15 14:29:04 srv-wazuh filebeat[705]: 2021-07-15T14:29:04.052Z INFO [publisher] pipeline/retry.go:213 retryer: send wait signal to consumer
Jul 15 14:29:04 srv-wazuh filebeat[705]: 2021-07-15T14:29:04.052Z INFO [publisher] pipeline/retry.go:217 done
Jul 15 14:29:06 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:29:06Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"Unable to revive connection: http://172.31.29.10:9200/"}
Jul 15 14:29:06 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:29:06Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"No living connections"}
Jul 15 14:29:08 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:29:08Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"Unable to revive connection: http://172.31.29.10:9200/"}
Jul 15 14:29:08 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:29:08Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"No living connections"}
Jul 15 14:29:11 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:29:11Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"Unable to revive connection: http://172.31.29.10:9200/"}
Jul 15 14:29:11 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:29:11Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"No living connections"}
Jul 15 14:29:12 srv-wazuh filebeat[705]: 2021-07-15T14:29:12.569Z INFO [monitoring] log/log.go:144 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":5040,"time":{"ms":6}},"total":{"ticks":17200,"time":{"ms":14},"value":17200},"user":>
Jul 15 14:29:13 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:29:13Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"Unable to revive connection: http://172.31.29.10:9200/"}
Jul 15 14:29:13 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:29:13Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"No living connections"}
Jul 15 14:29:16 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:29:16Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"Unable to revive connection: http://172.31.29.10:9200/"}
Jul 15 14:29:16 srv-wazuh kibana[7255]: {"type":"log","@timestamp":"2021-07-15T14:29:16Z","tags":["warning","elasticsearch","data"],"pid":7255,"message":"No living connections"}
than i checked the support matrix : Elastic Support Matrix | Elasticsearch
so i checked Kibana and Elasticsearch version and i found out that are both 7.9.1
wich are not compatible with ubuntu 20.04 (and by the way no one did an upgrade to the OS, So my question how come elasticsearch was running on this os, and what I should do now, I mean shoud i reinstall elasticsearch version 7.10.x or higher instead of 7.9.1 or perform an upgrade to the existing version.