Windows 2012 Servers are not responding due to memory issue

Elastic Stack Elasticsearch
1

I have 3 windows servers with elastic search installed and clustered. These servers are not responding after 1 week, I cannot RDP into the vms. When I look into vsphere I notice that its utilizing max memory, So immediately I used to reboot the vms in vsphere. These VMs have 2 CPUs and 8GB Ram each, What is the recommended CPUs and RAM for ES servers(clustered-3)? This is the java memory set. Is there any problem with my java memorysetting or do I need to make changes in the yaml file? Can any one reply i'm facing this issue frequently.

2

@dubul,

How did you configure your 3 node cluster?
All 3 nodes Master and data nodes or 1 master and 2 data nodes?

3

node.master: true
node.data: true

I have selected both

4

This is my es config file. This config will be similar on all the three nodes except the nodename and networkhost will be different.

     # ======================== Elasticsearch Configuration =========================
    #
    # NOTE: Elasticsearch comes with reasonable defaults for most settings.
    #       Before you set out to tweak and tune the configuration, make sure you
    #       understand what are you trying to accomplish and the consequences.
    #
    # The primary way of configuring a node is via this file. This template lists
    # the most important settings you may want to configure for a production cluster.
    #
    # Please see the documentation for further information on configuration options:
    # <http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html>
    #
    # ---------------------------------- Cluster -----------------------------------
    #
    # Use a descriptive name for your cluster:
    #
     cluster.name: elasticsearch
    #
    # ------------------------------------ Node ------------------------------------
    #
    # Use a descriptive name for the node:
    #
     node.name: dt1es01p01
     node.master: true 
     node.data: true
    #
    # Add custom attributes to the node:
    #
    # node.rack: r1
    #
    # ----------------------------------- Paths ------------------------------------
    #
    # Path to directory where to store the data (separate multiple locations by comma):
    #
     path.data: f:\esdata
    #
    # Path to log files:
    #
     path.logs: f:\eslogs
    #
    # ----------------------------------- Memory -----------------------------------
    #
    # Lock the memory on startup:
    #
    # bootstrap.mlockall: true
    #
    # Make sure that the `ES_HEAP_SIZE` environment variable is set to about half the memory
    # available on the system and that the owner of the process is allowed to use this limit.
    #
    # Elasticsearch performs poorly when the system is swapping the memory.
    #
    # ---------------------------------- Network -----------------------------------
    #
    # Set the bind address to a specific IP (IPv4 or IPv6):
    #
     network.host: dt1es01p01
    #
    # Set a custom port for HTTP:
    #
     http.port: 9200
    #
    # For more information, see the documentation at:
    # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html>
    #
    # --------------------------------- Discovery ----------------------------------
    #
    # Pass an initial list of hosts to perform discovery when new node is started:
    # The default list of hosts is ["127.0.0.1", "[::1]"]
    #
     discovery.zen.ping.unicast.hosts: ["dt1es01p01", "dt1es01p02", "dt1es01p03"]
    #
    # Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1):
    #
     discovery.zen.minimum_master_nodes: 2
    #
    # For more information, see the documentation at:
    # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html>
    #
    # ---------------------------------- Gateway -----------------------------------
    #
    # Block initial recovery after a full cluster restart until N nodes are started:
    #
    # gateway.recover_after_nodes: 3
    #
    # For more information, see the documentation at:
    # <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html>
    #
    # ---------------------------------- Various -----------------------------------
    #
    # Disable starting multiple nodes on a single system:
    #
    # node.max_local_storage_nodes: 1
    #
    # Require explicit names when deleting indices:
    #
    # action.destructive_requires_name: true
5

@dubul,
uncomment this setting bootstrap.mlockall: true in yaml file.
Also change the Initial memory pool value to match with your Maximum memory pool.

6

Thank you @iluvcode

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.