Intermittent issues with Elasticsearch. Where do I even start?

Elastic Stack Elasticsearch
1

Using

Magento 2.3.4
Elasticsearch 7.16.3
java 1.8.0_312

I installed it on Ubuntu 20.04 with 2cpu and 2g RAM.

/etc/default/Elasticsearch

ES_HEAP_SIZE=1g
ES_STARTUP_SLEEP_TIME=5
MAX_OPEN_FILES=65535
MAX_LOCKED_MEMORY=unlimited

/etc/Elasticsearch/Elasticsearch.yml

cluster.name: Magento
network.host: 127.0.0.1
discovery.type: single-node

sudo systemctl restart Elasticsearch.service

This works fine... until I try to reindex Magento.
php bin/magento indexer:reindex catalogsearch_fulltext

I get: Could not ping search engine: No alive nodes found in your cluster
curl -X GET "127.0.0.1:9200"
curl: (7) Failed to connect to 127.0.0.1 port 9200: Connection refused

journalctl -xe

Jan 21 09:20:24 Magento kernel: oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,t>
Jan 21 09:20:24 Magento kernel: Out of memory: Killed process 4414 (java) total-vm:3774312kB, anon-rss:1287532kB, file-r>
Jan 21 09:20:24 Magento kernel: oom_reaper: reaped process 4414 (java), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
Jan 21 09:20:24 Magento sshd[4694]: Received disconnect from 112.85.42.2 port 46033:11:  [preauth]
Jan 21 09:20:24 Magento sshd[4694]: Disconnected from 112.85.42.2 port 46033 [preauth]
Jan 21 09:20:24 Magento systemd[1]: elasticsearch.service: Main process exited, code=killed, status=9/KILL

Then my log has just a warning::

 [Magento] the default discovery settings are unsuitable for production use; at least one of [discovery.seed_hosts, discovery.seed_providers, cluster.initial_master_nodes]

So... What am I doing wrong???

2

For Elasticsearch it is recommended to set the heap to 50% of memory allocated to it as long as this stays below roughly 30GB. If you are assigning 1GB to heap that implies that Elasticsearch has exclusive access to the full 2GB of the node, which will not be the case if you are running other processes there. You could lower the heap size by 50%, but I would not recommend going lower than that. If you have a lot of things running on the node you may simply need to increase the amount of RAM available.

3

I suppose you are using single node elaticsearch cluster with development mode and not production mode. I'm not familiar with Magento but reading the error message of Magento, isn't it the cause of problem? Another possibility seems memory shortage for Magento.

As far as Elasticsearch works fine without Magento, You had better to ask to Magent support.

4

Hi Tomo_M thanks for your answer. Magento is just the name of my server. I am running a production mode and the issue is probably from Elasticsearch. As Christian mentioned, I will need to upgrade my server to avoid intermittent shortage.

5

Thanks. How many CPU/RAM do you recommend for an optimal Magento shop of about 10K products?

6

I would first try reducing the heap size to 512MB and see if that works. If not, a 1GB like you have now should be fine but you may need additional memory for other processes.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.