ES 5.6 inmemory-node recovers faster than standalone ES server

[kayshu]

Hi,

During some testing with a ES5.6 with about 4GB of data (3k shards), with an inmemory-node we are getting really fast shard recovery during startup (5 times faster than standalone ES server).
A standalone ES server takes about 20 minutes on my machine to recover completely (i.e all shards change from unassigned to started state) whereas a in-memory node recovers and starts all the shard in just 4 minutes. Is this expected behaviour ? what could be the cause of this.

[spinscale (Alexander Reelsen)]

can you explain what an in-memory node is? Elasticsearch does not have such a feature, as all data is persisted to disk.

That said, 3k shards for a single node are too much and you should reduce. Also, in order to speed up recoveries on node start up, looking at synced flush might make some sense.

Lastly, you should consider to upgrade to one of the more recent elasticsearch versions, coming with tons of bugfixes and performance improvements.`

[kayshu]

Sorry for not explaining it well, by an in-memory node, i meant an embedded node. i.e creating the node directly using org.elasticsearch.node.Node

Something like this

    Settings settings = Settings.builder()
                        .put("path.home", homePath)
                        .build();
    node = new Node(settings);
    node.start();

also the 3k shards is just for testing. (but will number of shards affect the cluster performance even if data size is less ?)

[kayshu]

@spinscale

So after much digging, it looks like the slow recovery was because of the Elasticsearch's Security Manager. Since the Embedded node was started directly and had no interference from the security manager, hence it recovered faster.
I know that the feature to disable the security manager has been removed. But what are the disadvantages of disabling the SecurityManager?