Unable to start elasticsearch after upgrade 7.17 -> 8.2 timeout was exceeded

Can you please direct me how to solve the problem when starting elasticsearch node after its upgrade to 8.2

system startup timeouts despite the fact that I adjusted the timeout to 3 minutes
i followed this procedure

it seems to be related to geoip databases that are not there, but I do not know how to resolve this

tode01prahkz:/data/elasticsearch/log# systemctl start elasticsearch
Job for elasticsearch.service failed because a timeout was exceeded. See "systemctl status elasticsearch.service" and "journalctl -xe" for details.

journalctl

journalctl -xe -u elasticsearch

-- Unit elasticsearch.service has begun starting up.
Jul 29 10:37:04 tode01prahkz systemd[1]: Got notification message from PID 9976, but reception only permitted for main PID 9909
Jul 29 10:39:49 tode01prahkz systemd[1]: elasticsearch.service start operation timed out. Terminating.
Jul 29 10:39:50 tode01prahkz systemd[1]: Got notification message from PID 9976, but reception only permitted for main PID 9909
Jul 29 10:39:52 tode01prahkz systemd[1]: Failed to start Elasticsearch.
-- Subject: Unit elasticsearch.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
-- Unit elasticsearch.service has failed.
--
-- The result is failed.
Jul 29 10:39:52 tode01prahkz systemd[1]: Unit elasticsearch.service entered failed state.
Jul 29 10:39:52 tode01prahkz systemd[1]: elasticsearch.service failed.

the app log:

...
[2022-07-29T10:37:03,767][ERROR][o.e.i.g.DatabaseNodeService] [tode01prahkz] failed to retrieve database [GeoLite2-Country.mmdb]
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
... 
[2022-07-29T10:37:03,767][ERROR][o.e.i.g.DatabaseNodeService] [tode01prahkz] failed to retrieve database [GeoLite2-ASN.mmdb]
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
...     
[2022-07-29T10:37:03,767][ERROR][o.e.i.g.DatabaseNodeService] [tode01prahkz] failed to retrieve database [GeoLite2-City.mmdb]
org.elasticsearch.cluster.block.ClusterBlockException: blocked by: [SERVICE_UNAVAILABLE/1/state not recovered / initialized];
...        
[2022-07-29T10:37:04,348][INFO ][o.e.x.s.a.TokenService   ] [tode01prahkz] refresh keys
[2022-07-29T10:37:04,463][INFO ][o.e.x.s.a.TokenService   ] [tode01prahkz] refreshed keys
[2022-07-29T10:37:04,503][INFO ][o.e.l.LicenseService     ] [tode01prahkz] license [91cf13b1-6265-4d7f-a516-74b68e5f3f5f] mode [basic] - valid
[2022-07-29T10:37:04,504][INFO ][o.e.x.s.a.Realms         ] [tode01prahkz] license mode is [basic], currently licensed security realms are [reserved/reserved,file/default_file,native/default_native]
[2022-07-29T10:37:04,509][INFO ][o.e.h.AbstractHttpServerTransport] [tode01prahkz] publish_address {10.44.163.37:9200}, bound_addresses {[::]:9200}
[2022-07-29T10:37:04,509][INFO ][o.e.n.Node               ] [tode01prahkz] started {tode01prahkz}{bL_sp4qyS_i1g6NyUA24kA}{667jVFsVReaWGxYnrxdv_g}{tode01prahkz}{10.44.163.37}{10.44.163.37:9300}{hilmst}{ml.max_jvm_size=16106127360, xpack.installed=true, ml.machine_memory=33567277056}
[2022-07-29T10:37:04,873][INFO ][o.e.i.g.DatabaseNodeService] [tode01prahkz] successfully loaded geoip database file [GeoLite2-Country.mmdb]
[2022-07-29T10:37:05,001][INFO ][o.e.i.g.DatabaseNodeService] [tode01prahkz] successfully loaded geoip database file [GeoLite2-ASN.mmdb]
[2022-07-29T10:37:06,099][INFO ][o.e.i.g.DatabaseNodeService] [tode01prahkz] successfully loaded geoip database file [GeoLite2-City.mmdb]
[2022-07-29T10:39:49,639][INFO ][o.e.n.Node               ] [tode01prahkz] stopping ...
[2022-07-29T10:39:49,642][INFO ][o.e.x.w.WatcherService   ] [tode01prahkz] stopping watch service, reason [shutdown initiated]
[2022-07-29T10:39:49,642][INFO ][o.e.x.w.WatcherLifeCycleService] [tode01prahkz] watcher has stopped and shutdown
[2022-07-29T10:39:49,953][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [tode01prahkz] [controller/10003] [Main.cc@176] ML controller exiting
[2022-07-29T10:39:49,954][INFO ][o.e.x.m.p.NativeController] [tode01prahkz] Native controller process has stopped - no new native processes can be started
[2022-07-29T10:39:49,957][INFO ][o.e.c.c.Coordinator      ] [tode01prahkz] master node [{tode03prahkz}{V1Fj3JEaTNyUbS21SkWWgQ}{boDTTy3DT4OOKFLpozIluA}{tode03prahkz}{10.44.163.39}{10.44.163.39:9300}{himst}] disconnected, restarting discovery
[2022-07-29T10:39:50,489][INFO ][o.e.n.Node               ] [tode01prahkz] stopped
[2022-07-29T10:39:50,489][INFO ][o.e.n.Node               ] [tode01prahkz] closing ...
[2022-07-29T10:39:50,499][INFO ][o.e.i.g.DatabaseReaderLazyLoader] [tode01prahkz] evicted [0] entries from cache after reloading database [/tmp/elasticsearch-8551826250694294177/geoip-databases/bL_sp4qyS_i1g6NyUA24kA/GeoLite2-Country.mmdb]
[2022-07-29T10:39:50,499][INFO ][o.e.i.g.DatabaseReaderLazyLoader] [tode01prahkz] evicted [0] entries from cache after reloading database [/tmp/elasticsearch-8551826250694294177/geoip-databases/bL_sp4qyS_i1g6NyUA24kA/GeoLite2-ASN.mmdb]
[2022-07-29T10:39:50,499][INFO ][o.e.i.g.DatabaseReaderLazyLoader] [tode01prahkz] evicted [0] entries from cache after reloading database [/tmp/elasticsearch-8551826250694294177/geoip-databases/bL_sp4qyS_i1g6NyUA24kA/GeoLite2-City.mmdb]
[2022-07-29T10:39:50,500][INFO ][o.e.n.Node               ] [tode01prahkz] closed

it hangs on line successfully loaded geoip database file [GeoLite2-City.mmdb]
which is weird because it complains the file does not exists but later it writes this successfully loaded line and later it timeouts and exits..

the file does not exists here

tode01prahkz:/usr/share/elasticsearch# find . |grep -i GeoLite2-City.mmdb
tode01prahkz:/usr/share/elasticsearch# ls -l /usr/share/elasticsearch/modules/ingest-geoip
total 2140
-rw-r--r-- 1 root root   56179 Jul  6 17:17 geoip2-3.0.0.jar
-rw-r--r-- 1 root root  100197 Jul  6 17:17 ingest-geoip-8.3.2.jar
-rw-r--r-- 1 root root   75717 Jul  6 17:17 jackson-annotations-2.13.1.jar
-rw-r--r-- 1 root root  374588 Jul  6 17:17 jackson-core-2.13.1.jar
-rw-r--r-- 1 root root 1534017 Jul  6 17:17 jackson-databind-2.13.1.jar
-rw-r--r-- 1 root root   30891 Jul  6 17:17 maxmind-db-2.0.0.jar
-rw-r--r-- 1 root root    1937 Jul  6 17:17 plugin-descriptor.properties
-rw-r--r-- 1 root root    1081 Jul  6 17:17 plugin-security.policy
1 Like

in order to get rid of geoip err I tried

PUT _cluster/settings
{
    "persistent": {
       "ingest.geoip.downloader.enabled": false
    }
}

but it seems does not help
the err message dissapeared

there is no ERR message is elasticsearch.log anymore
the only err I receive is


Jul 30 11:45:04 tode01prahkz systemd[1]: Got notification message from PID 1147, but reception only permitted for main PID 1080
Jul 30 11:47:49 tode01prahkz systemd[1]: elasticsearch.service start operation timed out. Terminating.
Jul 30 11:47:50 tode01prahkz systemd[1]: Got notification message from PID 1147, but reception only permitted for main PID 1080
Jul 30 11:47:52 tode01prahkz systemd[1]: Failed to start Elasticsearch.

I found out the node joins the clster but it is shutted down later
the problem is this err


Jul 30 12:05:17 tode01prahkz systemd[1]: Starting Elasticsearch...
Jul 30 12:05:33 tode01prahkz systemd[1]: Got notification message from PID 3780, but reception only permitted for main PID 3714
Jul 30 12:08:18 tode01prahkz systemd[1]: elasticsearch.service start operation timed out. Terminating.
Jul 30 12:08:19 tode01prahkz systemd[1]: Got notification message from PID 3780, but reception only permitted for main PID 3714
Jul 30 12:08:21 tode01prahkz systemd[1]: Failed to start Elasticsearch.
Jul 30 12:08:21 tode01prahkz systemd[1]: Unit elasticsearch.service entered failed state.
Jul 30 12:08:21 tode01prahkz systemd[1]: elasticsearch.service failed.

the issue is here with JAVA
see err complain of pid 23374 and 23308

tode01prahkz:/data/elasticsearch/log# systemctl status elasticsearch
â—Ź elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/elasticsearch.service.d
           └─startup-timeout.conf
   Active: activating (start) since Sat 2022-07-30 14:48:26 CEST; 25s ago
     Docs: https://www.elastic.co
 Main PID: 23308 (java)
   CGroup: /system.slice/elasticsearch.service
           ├─23308 /usr/share/elasticsearch/jdk/bin/java -Xms4m -Xmx64m -XX:+UseSerialGC -Dcli.name=server -Dcli.script=/usr/share/elasticsearch/bin/elasticsearch -Dcli.libs=lib/tools/server-...
           ├─23374 /usr/share/elasticsearch/jdk/bin/java -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -Djava.security.manager=allow -XX:+AlwaysPreTouch -Xss1m -D...
           └─23400 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller

Jul 30 14:48:26 tode01prahkz systemd[1]: Starting Elasticsearch...
Jul 30 14:48:41 tode01prahkz systemd[1]: Got notification message from PID 23374, but reception only permitted for main PID 23308

This shows that something asked the Elasticsearch process to shutdown.
Maybe there's a systemd timeout you can increase to let Elasticsearch start up?

thank you @warkolm I have upated the timeout to 3minutes
by looking at java CGroup there are two java processes

CGroup: /system.slice/elasticsearch.service
├─23308 /usr/share/elasticsearch/jdk/bin/java -Xms4m -Xmx64m -XX:+U
├─23374 /usr/share/elasticsearch/jdk/bin/java -Des.networkaddress.c
└─23400 /usr/share/elasticsearch/modules/x-pack-ml/platform/

but in 7.17 version this looked

tode01prahkz:~# systemctl status elasticsearch
...
Main PID: 13760 (java)
CGroup: /system.slice/elasticsearch.service
├─13760 /usr/share/elasticsearch/jdk/bin/java -Xshare:auto -Des.n...
└─13965 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux

I have downgraded the elastic back to 7.17 and I will try it again.

@warkolm pls do you know whether there were any changes within JAVA setup I should process before moving from v7 to v8

when I run the upgrade process again I am getting the same err message

it starts the cluster but after timeout the cluster stops

tode01prahkz:~# systemctl status elasticsearch
â—Ź elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
Drop-In: /etc/systemd/system/elasticsearch.service.d
└─startup-timeout.conf
Active: activating (start) since Tue 2022-08-02 08:35:13 CEST; 1min 13s ago
Docs: https://www.elastic.co
Main PID: 22212 (java)
CGroup: /system.slice/elasticsearch.service
├─22212 /usr/share/elasticsearch/jdk/bin/java -Xms4m -Xmx64m -XX:+UseSerialGC -Dcli.name=server -Dcli.script=/usr/share/elasticsearch/bin/elasticsearch -Dcli.libs=lib/tools/serve...
├─22284 /usr/share/elasticsearch/jdk/bin/java -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -Djava.security.manager=allow -XX:+AlwaysPreTouch -Xss1m ...
└─22309 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller

Aug 02 08:35:13 tode01prahkz systemd[1]: Starting Elasticsearch...
Aug 02 08:35:31 tode01prahkz systemd[1]: Got notification message from PID 22284, but reception only permitted for main PID 22212

this is how it looks after timeout

> tode01prahkz:~# systemctl status elasticsearch
> â—Ź elasticsearch.service - Elasticsearch
>    Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
>   Drop-In: /etc/systemd/system/elasticsearch.service.d
>            └─startup-timeout.conf
>    Active: failed (Result: timeout) since Tue 2022-08-02 08:38:16 CEST; 3min 49s ago
>      Docs: https://www.elastic.co
>   Process: 22212 ExecStart=/usr/share/elasticsearch/bin/systemd-entrypoint -p ${PID_DIR}/elasticsearch.pid --quiet (code=exited, status=143)
>  Main PID: 22212 (code=exited, status=143)
> 
> Aug 02 08:35:13 tode01prahkz systemd[1]: Starting Elasticsearch...
> Aug 02 08:35:31 tode01prahkz systemd[1]: Got notification message from PID 22284, but reception only permitted for main PID 22212
> Aug 02 08:38:13 tode01prahkz systemd[1]: elasticsearch.service start operation timed out. Terminating.
> Aug 02 08:38:14 tode01prahkz systemd[1]: Got notification message from PID 22284, but reception only permitted for main PID 22212
> Aug 02 08:38:16 tode01prahkz systemd[1]: Failed to start Elasticsearch.
> Aug 02 08:38:16 tode01prahkz systemd[1]: Unit elasticsearch.service entered failed state.
> Aug 02 08:38:16 tode01prahkz systemd[1]: elasticsearch.service failed.

after reinstall It shows different err in elastic.log which may be the root cause
I still do not know how to resolve it but I am gooing to search this issue

> ...
> [2022-08-02T08:49:22,597][INFO ][o.e.x.s.a.RealmsAuthenticator] [tode01prahkz] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
> [2022-08-02T08:49:22,597][INFO ][o.e.x.s.a.RealmsAuthenticator] [tode01prahkz] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
> [2022-08-02T08:49:22,598][INFO ][o.e.x.s.a.RealmsAuthenticator] [tode01prahkz] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
> [2022-08-02T08:49:22,597][INFO ][o.e.x.s.a.RealmsAuthenticator] [tode01prahkz] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
> [2022-08-02T08:49:22,669][INFO ][o.e.x.s.a.RealmsAuthenticator] [tode01prahkz] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
> [2022-08-02T08:49:29,462][INFO ][o.e.n.Node               ] [tode01prahkz] stopping ...
> [2022-08-02T08:49:29,463][INFO ][o.e.x.w.WatcherService   ] [tode01prahkz] stopping watch service, reason [shutdown initiated]
> [2022-08-02T08:49:29,464][INFO ][o.e.x.w.WatcherLifeCycleService] [tode01prahkz] watcher has stopped and shutdown
> [2022-08-02T08:49:29,750][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [tode01prahkz] [controller/23860] [Main.cc@176] ML controller exiting
> [2022-08-02T08:49:29,750][INFO ][o.e.x.m.p.NativeController] [tode01prahkz] Native controller process has stopped - no new native processes can be started
> [2022-08-02T08:49:29,752][INFO ][o.e.c.c.Coordinator      ] [tode01prahkz] master node [{tode03prahkz}{V1Fj3JEaTNyUbS21SkWWgQ}{boDTTy3DT4OOKFLpozIluA}{tode03prahkz}{10.44.163.39}{10.44.163.39:9300}{himst}] disconnected, restarting discovery
> [2022-08-02T08:49:29,807][INFO ][o.e.n.Node               ] [tode01prahkz] stopped
> [2022-08-02T08:49:29,807][INFO ][o.e.n.Node               ] [tode01prahkz] closing ...
> [2022-08-02T08:49:29,815][INFO ][o.e.n.Node               ] [tode01prahkz] closed

Do you know why the node complains of realsm auth problem ?

[2022-08-02T09:16:20,407][INFO ][o.e.n.Node               ] [tode01prahkz] starting ...
[2022-08-02T09:16:20,432][INFO ][o.e.x.s.c.f.PersistentCache] [tode01prahkz] persistent cache index loaded
[2022-08-02T09:16:20,433][INFO ][o.e.x.d.l.DeprecationIndexingComponent] [tode01prahkz] deprecation component started
[2022-08-02T09:16:20,516][INFO ][o.e.t.TransportService   ] [tode01prahkz] publish_address {10.44.163.37:9300}, bound_addresses {[::]:9300}
[2022-08-02T09:16:21,940][INFO ][o.e.b.BootstrapChecks    ] [tode01prahkz] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2022-08-02T09:16:21,955][WARN ][o.e.c.c.ClusterBootstrapService] [tode01prahkz] this node is locked into cluster UUID [XtdLLc-TTrCiKcJ5WhtFLw] but [cluster.initial_master_nodes] is set to [tode01prahkz, tode02prahkz, tode03prahkz]; remove this setting to avoid possible data loss caused by subsequent cluster bootstrap attempts
[2022-08-02T09:16:23,133][INFO ][o.e.c.s.ClusterApplierService] [tode01prahkz] master node changed {previous [], current [{tode03prahkz}{V1Fj3JEaTNyUbS21SkWWgQ}{boDTTy3DT4OOKFLpozIluA}{tode03prahkz}{10.44.163.39}{10.44.163.39:9300}{himst}]}, added {{todk01prahkz}{fcb4e-ArSKeozVMFaTfAew}{pKVSzPs0QXmgDe3kYsqVOg}{todk01prahkz}{10.44.163.40}{10.44.163.40:9300}, {tode03prahkz}{V1Fj3JEaTNyUbS21SkWWgQ}{boDTTy3DT4OOKFLpozIluA}{tode03prahkz}{10.44.163.39}{10.44.163.39:9300}{himst}, {tode02prahkz}{HY5GXbZvRVSczTsrEZG_DQ}{9sHZi7hiTXOGdZMHbkONZw}{tode02prahkz}{10.44.163.38}{10.44.163.38:9300}{himst}}, term: 22, version: 525973, reason: ApplyCommitRequest{term=22, version=525973, sourceNode={tode03prahkz}{V1Fj3JEaTNyUbS21SkWWgQ}{boDTTy3DT4OOKFLpozIluA}{tode03prahkz}{10.44.163.39}{10.44.163.39:9300}{himst}{xpack.installed=true, data=hot, transform.node=true}}
[2022-08-02T09:16:23,240][INFO ][o.e.c.s.ClusterSettings  ] [tode01prahkz] updating [cluster.routing.allocation.enable] from [all] to [primaries]
[2022-08-02T09:16:23,241][INFO ][o.e.c.s.ClusterSettings  ] [tode01prahkz] updating [cluster.routing.allocation.node_concurrent_incoming_recoveries] from [2] to [5]
[2022-08-02T09:16:23,241][INFO ][o.e.c.s.ClusterSettings  ] [tode01prahkz] updating [cluster.routing.allocation.node_concurrent_outgoing_recoveries] from [2] to [5]
[2022-08-02T09:16:23,241][INFO ][o.e.c.s.ClusterSettings  ] [tode01prahkz] updating [cluster.routing.allocation.enable] from [all] to [primaries]
[2022-08-02T09:16:23,241][INFO ][o.e.c.s.ClusterSettings  ] [tode01prahkz] updating [ingest.geoip.downloader.enabled] from [true] to [false]
[2022-08-02T09:16:23,242][INFO ][o.e.c.s.ClusterSettings  ] [tode01prahkz] updating [xpack.monitoring.elasticsearch.collection.enabled] from [true] to [false]
[2022-08-02T09:16:23,242][INFO ][o.e.c.s.ClusterSettings  ] [tode01prahkz] updating [xpack.monitoring.collection.enabled] from [false] to [true]
[2022-08-02T09:16:23,842][INFO ][o.e.x.s.a.TokenService   ] [tode01prahkz] refresh keys
[2022-08-02T09:16:23,956][INFO ][o.e.x.s.a.TokenService   ] [tode01prahkz] refreshed keys
[2022-08-02T09:16:23,985][INFO ][o.e.l.LicenseService     ] [tode01prahkz] license [91cf13b1-6265-4d7f-a516-74b68e5f3f5f] mode [basic] - valid
[2022-08-02T09:16:23,986][INFO ][o.e.x.s.a.Realms         ] [tode01prahkz] license mode is [basic], currently licensed security realms are [reserved/reserved,file/default_file,native/default_native]
[2022-08-02T09:16:23,991][INFO ][o.e.h.AbstractHttpServerTransport] [tode01prahkz] publish_address {10.44.163.37:9200}, bound_addresses {[::]:9200}
[2022-08-02T09:16:23,991][INFO ][o.e.n.Node               ] [tode01prahkz] started {tode01prahkz}{aCiSpclBTmGc1gvS7TImoA}{l9ewzpBsT-SF75UfzwlCkg}{tode01prahkz}{10.44.163.37}{10.44.163.37:9300}{hilmst}{ml.max_jvm_size=16106127360, xpack.installed=true, ml.machine_memory=33567244288}
[2022-08-02T09:16:32,607][INFO ][o.e.x.s.a.RealmsAuthenticator] [tode01prahkz] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2022-08-02T09:16:32,607][INFO ][o.e.x.s.a.RealmsAuthenticator] [tode01prahkz] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2022-08-02T09:16:32,607][INFO ][o.e.x.s.a.RealmsAuthenticator] [tode01prahkz] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2022-08-02T09:16:32,607][INFO ][o.e.x.s.a.RealmsAuthenticator] [tode01prahkz] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]
[2022-08-02T09:16:32,608][INFO ][o.e.x.s.a.RealmsAuthenticator] [tode01prahkz] Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]

before timeout it is normally connected to the cluster and I can access it with password

xxxxx@tode01prahkz:~$ curl -u elastic:xxx "http://localhost:9200/_cat/nodes?v&h=name,version,node.role,master"
name         version node.role master
tode03prahkz 7.17.0  himst     *
tode02prahkz 7.17.0  himst     -
todk01prahkz 7.17.0  -         -
tode01prahkz 8.3.2   hilmst    -

Because something is trying to login with the elastic user with a wrong password.

@warkolm
password is not stored in elasticsearch.yml nor in other conf files and nodes are joined based on keys but not elastic password.

i did just: systemctl start elasticsearch

Where do you think the incorrect login is comming from?

false alarm: this auth err was caused by metricbeat

Authentication of [elastic] was terminated by realm [reserved] - failed to authenticate user [elastic]

I forgot to fix this conf.

however the original issue still remains unresolved

It seems I have resolved it by adding NotifyAccess=all
to service.d conf file

since then it starts normally

tode01prahkz:/etc/systemd/system/elasticsearch.service.d# cat /etc/systemd/system/elasticsearch.service.d/startup-timeout.conf
[Service]
TimeoutStartSec=180
NotifyAccess=all

3 Likes

it is related to

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.