Native controller process has stopped : no new native processes can be started

Elastic Stack Elasticsearch
1

Hello support

I need your help in order to fix a crash of elasticsearch just after the starting with the message 'Native controller process has stopped - no new native processes can be started"

Elasticsearch is started a systemd service.
I don't get this issue when I start manually the process

Elasticsearch release 6.7.1
OS : Redhat 7.1

Logs:

[2020-01-07T15:40:59,709][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [sidre-bac-tac] [controller/5278] [Main.cc@109] controller (64 bit): Version 6.7.1 (Build e1f492de67a719) Copyright (c) 2019 Elasticsearch BV
[2020-01-07T15:41:00,498][DEBUG][o.e.a.ActionModule ] [sidre-bac-tac] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2020-01-07T15:41:00,864][INFO ][o.e.d.DiscoveryModule ] [sidre-bac-tac] using discovery type [zen] and host providers [settings]
[2020-01-07T15:41:01,686][INFO ][o.e.n.Node ] [sidre-bac-tac] initialized
[2020-01-07T15:41:01,686][INFO ][o.e.n.Node ] [sidre-bac-tac] starting ...
[2020-01-07T15:41:01,882][INFO ][o.e.t.TransportService ] [sidre-bac-tac] publish_address {10.229.3.157:9300}, bound_addresses {10.238.95.156:9300}, {10.229.3.157:9300}
[2020-01-07T15:41:01,914][INFO ][o.e.b.BootstrapChecks ] [sidre-bac-tac] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2020-01-07T15:41:05,264][INFO ][o.e.c.s.ClusterApplierService] [sidre-bac-tac] detected_master {sidre-bac-rjs}{hm0uUOdvSR28mmeT6N4cFw}{UNjiXXzsSbmUaJ5JPHqPmw}{10.229.3.155}{10.229.3.155:9300}{ml.machine_memory=8189480960, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true}, added {{sidre-bac-rjs}{hm0uUOdvSR28mmeT6N4cFw}{UNjiXXzsSbmUaJ5JPHqPmw}{10.229.3.155}{10.229.3.155:9300}{ml.machine_memory=8189480960, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true},}, reason: apply cluster state (from master [master {sidre-bac-rjs}{hm0uUOdvSR28mmeT6N4cFw}{UNjiXXzsSbmUaJ5JPHqPmw}{10.229.3.155}{10.229.3.155:9300}{ml.machine_memory=8189480960, ml.max_open_jobs=20, xpack.installed=true, ml.enabled=true} committed version [14]])
[2020-01-07T15:41:05,572][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [sidre-bac-tac] Failed to clear cache for realms []
[2020-01-07T15:41:05,576][INFO ][o.e.x.s.a.TokenService ] [sidre-bac-tac] refresh keys
[2020-01-07T15:41:05,843][INFO ][o.e.x.s.a.TokenService ] [sidre-bac-tac] refreshed keys
[2020-01-07T15:41:05,881][INFO ][o.e.l.LicenseService ] [sidre-bac-tac] license [596d39b3-5ad0-422e-8b50-126d73681e02] mode [basic] - valid
[2020-01-07T15:41:05,930][INFO ][o.e.h.n.Netty4HttpServerTransport] [sidre-bac-tac] publish_address {10.229.3.157:9200}, bound_addresses {10.238.95.156:9200}, {10.229.3.157:9200}
[2020-01-07T15:41:05,930][INFO ][o.e.n.Node ] [sidre-bac-tac] started
[2020-01-07T15:41:50,207][INFO ][o.e.x.m.p.NativeController] [sidre-bac-tac] Native controller process has stopped - no new native processes can be started
[2020-01-07T15:41:50,209][INFO ][o.e.n.Node ] [sidre-bac-tac] stopping ...
[2020-01-07T15:41:50,219][INFO ][o.e.x.w.WatcherService ] [sidre-bac-tac] stopping watch service, reason [shutdown initiated]
[2020-01-07T15:41:50,674][INFO ][o.e.n.Node ] [sidre-bac-tac] stopped
[2020-01-07T15:41:50,675][INFO ][o.e.n.Node ] [sidre-bac-tac] closing ...
[2020-01-07T15:41:50,688][INFO ][o.e.n.Node ] [sidre-bac-tac] closed

2

Hello,

Welcome and thanks for posting in the forums.

I am not sure that the INFO-level message you posted points to the underlying issue. It doesn't seem like that message gets logged in a case that would shut down the node.

Are you retrieving these log messages with journald or by looking in a file?

-William

3

I retrieve these log messages in the file generated by elasticsearch and in the directory "path.logs"

4

If you're running Elasticsearch under systemd, you might be able to find an informative systemd error in the output of journalctl --user elasticsearch.service or systemctl status elasticsearch.

I hope this is helpful. Pinpointing these failures can be tricky.

-William

5

In journalctl, I get only 'failed to start Elasticsearch'

6

Issue fixed. "Type=simple" replaced by "Type=forking"in the service script

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.