Deleting index suddenly

Elastic Stack Elasticsearch
1

Hi everyone,
I've been having a very annoying problem lately. Something deleting my indexes suddenly. Here is my situations;

  1. 9200 port is closed to outside. Only server ip can access the port.
  2. There is a lot of docker containers runs my apps. (Ruby on Rails)
  3. It happened a week ago, yesterday and today.
  4. Thats log file:
[2022-09-28T01:01:00,001][INFO ][o.e.x.m.MlDailyMaintenanceService] [node-1] triggering scheduled [ML] maintenance tasks
[2022-09-28T01:01:00,053][INFO ][o.e.x.m.a.TransportDeleteExpiredDataAction] [node-1] Deleting expired data
[2022-09-28T01:01:00,120][INFO ][o.e.x.m.j.r.UnusedStatsRemover] [node-1] Successfully deleted [0] unused stats documents
[2022-09-28T01:01:00,122][INFO ][o.e.x.m.a.TransportDeleteExpiredDataAction] [node-1] Completed deletion of expired ML data
[2022-09-28T01:01:00,123][INFO ][o.e.x.m.MlDailyMaintenanceService] [node-1] Successfully completed [ML] maintenance task: triggerDeleteExpiredDataTask
[2022-09-28T02:42:24,224][INFO ][o.e.t.TcpTransport       ] [node-1] close connection exception caught on transport layer [Netty4TcpChannel{localAddress=/[SERVER_IP]:9300, remoteAddress=/167.248.133.120:53836, profile=default}], disconnecting from relevant node: Connection reset
[2022-09-28T02:42:24,364][WARN ][o.e.t.TcpTransport       ] [node-1] SSL/TLS request received but SSL/TLS is not enabled on this node, got (16,3,1,0), [Netty4TcpChannel{localAddress=/[SERVER_IP]:9300, remoteAddress=/167.248.133.120:60736, profile=default}], closing connection
[2022-09-28T02:42:25,970][WARN ][o.e.t.TcpTransport       ] [node-1] invalid internal transport message format, got (5a,47,0,0), [Netty4TcpChannel{localAddress=/[SERVER_IP]:9300, remoteAddress=/167.248.133.120:54630, profile=default}], closing connection
[2022-09-28T02:42:27,275][INFO ][o.e.t.TcpTransport       ] [node-1] close connection exception caught on transport layer [Netty4TcpChannel{localAddress=/[SERVER_IP]:9300, remoteAddress=/162.142.125.9:50404, profile=default}], disconnecting from relevant node: Connection reset
[2022-09-28T02:42:27,414][WARN ][o.e.t.TcpTransport       ] [node-1] SSL/TLS request received but SSL/TLS is not enabled on this node, got (16,3,1,0), [Netty4TcpChannel{localAddress=/[SERVER_IP]:9300, remoteAddress=/162.142.125.9:41160, profile=default}], closing connection
[2022-09-28T02:42:28,988][WARN ][o.e.t.TcpTransport       ] [node-1] invalid internal transport message format, got (5a,47,0,0), [Netty4TcpChannel{localAddress=/[SERVER_IP]:9300, remoteAddress=/162.142.125.9:50058, profile=default}], closing connection
[2022-09-28T02:42:30,346][INFO ][o.e.t.TcpTransport       ] [node-1] close connection exception caught on transport layer [Netty4TcpChannel{localAddress=/[SERVER_IP]:9300, remoteAddress=/167.94.138.118:55144, profile=default}], disconnecting from relevant node: Connection reset
[2022-09-28T02:42:30,558][WARN ][o.e.t.TcpTransport       ] [node-1] SSL/TLS request received but SSL/TLS is not enabled on this node, got (16,3,1,0), [Netty4TcpChannel{localAddress=/[SERVER_IP]:9300, remoteAddress=/167.94.138.118:37658, profile=default}], closing connection
[2022-09-28T02:42:32,158][WARN ][o.e.t.TcpTransport       ] [node-1] invalid internal transport message format, got (5a,47,0,0), [Netty4TcpChannel{localAddress=/[SERVER_IP]:9300, remoteAddress=/167.94.138.118:39190, profile=default}], closing connection
[2022-09-28T04:30:00,013][INFO ][o.e.x.s.SnapshotRetentionTask] [node-1] starting SLM retention snapshot cleanup task
[2022-09-28T04:30:00,016][INFO ][o.e.x.s.SnapshotRetentionTask] [node-1] there are no repositories to fetch, SLM retention snapshot cleanup task complete
[2022-09-28T05:49:34,647][INFO ][o.e.t.TcpTransport       ] [node-1] close connection exception caught on transport layer [Netty4TcpChannel{localAddress=/[SERVER_IP]:9300, remoteAddress=/167.94.138.47:57266, profile=default}], disconnecting from relevant node: Connection reset
[2022-09-28T05:49:34,798][WARN ][o.e.t.TcpTransport       ] [node-1] SSL/TLS request received but SSL/TLS is not enabled on this node, got (16,3,1,0), [Netty4TcpChannel{localAddress=/[SERVER_IP]:9300, remoteAddress=/167.94.138.47:41438, profile=default}], closing connection
[2022-09-28T05:49:36,401][WARN ][o.e.t.TcpTransport       ] [node-1] invalid internal transport message format, got (5a,47,0,0), [Netty4TcpChannel{localAddress=/[SERVER_IP]:9300, remoteAddress=/167.94.138.47:39864, profile=default}], closing connection
[2022-09-28T07:24:17,942][WARN ][o.e.t.TcpTransport       ] [node-1] invalid internal transport message format, got (4a,52,4d,49), [Netty4TcpChannel{localAddress=/[SERVER_IP]:9300, remoteAddress=/87.236.176.153:36785, profile=default}], closing connection
[2022-09-28T07:24:24,218][WARN ][o.e.t.TcpTransport       ] [node-1] invalid internal transport message format, got (0,0,0,31), [Netty4TcpChannel{localAddress=/[SERVER_IP]:9300, remoteAddress=/87.236.176.153:54701, profile=default}], closing connection
[2022-09-28T07:24:24,352][WARN ][o.e.t.TcpTransport       ] [node-1] invalid internal transport message format, got (0,0,0,6e), [Netty4TcpChannel{localAddress=/[SERVER_IP]:9300, remoteAddress=/87.236.176.153:44163, profile=default}], closing connection
[2022-09-28T07:24:27,530][WARN ][o.e.t.TcpTransport       ] [node-1] invalid internal transport message format, got (12,1,0,1a), [Netty4TcpChannel{localAddress=/[SERVER_IP]:9300, remoteAddress=/87.236.176.153:54509, profile=default}], closing connection
[2022-09-28T07:24:30,715][WARN ][o.e.t.TcpTransport       ] [node-1] invalid internal transport message format, got (3,0,0,2c), [Netty4TcpChannel{localAddress=/[SERVER_IP]:9300, remoteAddress=/87.236.176.153:34451, profile=default}], closing connection
[2022-09-28T07:24:37,016][WARN ][o.e.t.TcpTransport       ] [node-1] SSL/TLS request received but SSL/TLS is not enabled on this node, got (16,3,1,0), [Netty4TcpChannel{localAddress=/[SERVER_IP]:9300, remoteAddress=/87.236.176.153:45369, profile=default}], closing connection
[2022-09-28T10:06:03,572][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [ucarmak_products_production_20220927195522798/F8U6htWXRWWtzmQMaPpJaQ] deleting index
[2022-09-28T10:06:06,422][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [tasmakina_products_production_20220927193759937/LRs6jHZOSBCNVq_rV4AZUA] deleting index
[2022-09-28T10:06:09,470][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [sutkonagi_products_production_20220927182657047/D46hFQskQS2cYCc7-LVo7A] deleting index
[2022-09-28T10:06:12,421][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [stokdevi_products_production_20220927193058370/eYV2OMMmQZujtELpalWgSw] deleting index

  1. Resource Usages

    Screenshot
    Screenshot1648×553 138 KB

  2. elasticsearch.yml

cluster.name: my_cluster_name 
node.name: node-1
action.destructive_requires_name: true # added this line yesterday after index deleting issue
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0
http.port: 9200
discovery.seed_hosts: []

xpack.security.enabled: false
xpack.security.enrollment.enabled: false
xpack.security.http.ssl:
  enabled: false
  keystore.path: certs/http.p12
xpack.security.transport.ssl:
  enabled: false
  1. Here is ILM policies:
{
  "90-days-default": {
    "version": 1,
    "modified_date": "2022-04-20T14:56:54.056Z",
    "policy": {
      "phases": {
        "warm": {
          "min_age": "2d",
          "actions": {
            "shrink": {
              "number_of_shards": 1
            },
            "forcemerge": {
              "max_num_segments": 1
            }
          }
        },
        "cold": {
          "min_age": "30d",
          "actions": {}
        },
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_primary_shard_size": "50gb",
              "max_age": "30d"
            }
          }
        },
        "delete": {
          "min_age": "90d",
          "actions": {
            "delete": {
              "delete_searchable_snapshot": true
            }
          }
        }
      },
      "_meta": {
        "managed": true,
        "description": "built-in ILM policy using the hot, warm, and cold phases with a retention of 90 days"
      }
    },
    "in_use_by": {
      "indices": [],
      "data_streams": [],
      "composable_templates": []
    }
  },
  "ilm-history-ilm-policy": {
    "version": 1,
    "modified_date": "2022-04-20T14:56:54.244Z",
    "policy": {
      "phases": {
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_primary_shard_size": "50gb",
              "max_age": "30d"
            }
          }
        },
        "delete": {
          "min_age": "90d",
          "actions": {
            "delete": {
              "delete_searchable_snapshot": true
            }
          }
        }
      },
      "_meta": {
        "managed": true,
        "description": "default policy for the ILM history indices"
      }
    },
    "in_use_by": {
      "indices": [],
      "data_streams": [],
      "composable_templates": [
        "ilm-history"
      ]
    }
  },
  "watch-history-ilm-policy-16": {
    "version": 1,
    "modified_date": "2022-04-20T14:56:54.214Z",
    "policy": {
      "phases": {
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_primary_shard_size": "50gb",
              "max_age": "3d"
            }
          }
        },
        "delete": {
          "min_age": "4d",
          "actions": {
            "delete": {
              "delete_searchable_snapshot": true
            }
          }
        }
      },
      "_meta": {
        "managed": true,
        "description": "default policy for the watcher history indices"
      }
    },
    "in_use_by": {
      "indices": [],
      "data_streams": [],
      "composable_templates": [
        ".watch-history-16"
      ]
    }
  },
  "30-days-default": {
    "version": 1,
    "modified_date": "2022-04-20T14:56:54.154Z",
    "policy": {
      "phases": {
        "warm": {
          "min_age": "2d",
          "actions": {
            "shrink": {
              "number_of_shards": 1
            },
            "forcemerge": {
              "max_num_segments": 1
            }
          }
        },
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_primary_shard_size": "50gb",
              "max_age": "30d"
            }
          }
        },
        "delete": {
          "min_age": "30d",
          "actions": {
            "delete": {
              "delete_searchable_snapshot": true
            }
          }
        }
      },
      "_meta": {
        "managed": true,
        "description": "built-in ILM policy using the hot and warm phases with a retention of 30 days"
      }
    },
    "in_use_by": {
      "indices": [],
      "data_streams": [],
      "composable_templates": []
    }
  },
  "synthetics": {
    "version": 1,
    "modified_date": "2022-04-20T14:56:53.875Z",
    "policy": {
      "phases": {
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_primary_shard_size": "50gb",
              "max_age": "30d"
            }
          }
        }
      },
      "_meta": {
        "managed": true,
        "description": "default policy for the synthetics index template installed by x-pack"
      }
    },
    "in_use_by": {
      "indices": [],
      "data_streams": [],
      "composable_templates": [
        "synthetics"
      ]
    }
  },
  ".monitoring-8-ilm-policy": {
    "version": 1,
    "modified_date": "2022-04-20T14:56:54.017Z",
    "policy": {
      "phases": {
        "warm": {
          "min_age": "0ms",
          "actions": {
            "forcemerge": {
              "max_num_segments": 1
            }
          }
        },
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_primary_shard_size": "50gb",
              "max_age": "3d"
            }
          }
        },
        "delete": {
          "min_age": "3d",
          "actions": {
            "delete": {
              "delete_searchable_snapshot": true
            }
          }
        }
      },
      "_meta": {
        "defaults": {
          "delete_min_age": "Using value of [3d] based on the monitoring plugin default"
        },
        "description": "Index lifecycle policy generated for [monitoring-*-8] data streams"
      }
    },
    "in_use_by": {
      "indices": [],
      "data_streams": [],
      "composable_templates": [
        ".monitoring-beats-mb",
        ".monitoring-kibana-mb",
        ".monitoring-es-mb",
        ".monitoring-logstash-mb"
      ]
    }
  },
  "slm-history-ilm-policy": {
    "version": 1,
    "modified_date": "2022-04-20T14:56:54.274Z",
    "policy": {
      "phases": {
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_primary_shard_size": "50gb",
              "max_age": "30d"
            }
          }
        },
        "delete": {
          "min_age": "90d",
          "actions": {
            "delete": {
              "delete_searchable_snapshot": true
            }
          }
        }
      },
      "_meta": {
        "managed": true,
        "description": "default policy for the SLM history indices"
      }
    },
    "in_use_by": {
      "indices": [],
      "data_streams": [],
      "composable_templates": [
        ".slm-history"
      ]
    }
  },
  ".fleet-actions-results-ilm-policy": {
    "version": 1,
    "modified_date": "2022-04-20T14:56:54.333Z",
    "policy": {
      "phases": {
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_size": "300gb",
              "max_age": "30d"
            }
          }
        },
        "delete": {
          "min_age": "90d",
          "actions": {
            "delete": {
              "delete_searchable_snapshot": true
            }
          }
        }
      },
      "_meta": {
        "managed": true,
        "description": "default policy for fleet action results indices"
      }
    },
    "in_use_by": {
      "indices": [],
      "data_streams": [],
      "composable_templates": []
    }
  },
  "365-days-default": {
    "version": 1,
    "modified_date": "2022-04-20T14:56:54.185Z",
    "policy": {
      "phases": {
        "warm": {
          "min_age": "2d",
          "actions": {
            "shrink": {
              "number_of_shards": 1
            },
            "forcemerge": {
              "max_num_segments": 1
            }
          }
        },
        "cold": {
          "min_age": "30d",
          "actions": {}
        },
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_primary_shard_size": "50gb",
              "max_age": "30d"
            }
          }
        },
        "delete": {
          "min_age": "365d",
          "actions": {
            "delete": {
              "delete_searchable_snapshot": true
            }
          }
        }
      },
      "_meta": {
        "managed": true,
        "description": "built-in ILM policy using the hot, warm, and cold phases with a retention of 365 days"
      }
    },
    "in_use_by": {
      "indices": [],
      "data_streams": [],
      "composable_templates": []
    }
  },
  "7-days-default": {
    "version": 1,
    "modified_date": "2022-04-20T14:56:54.124Z",
    "policy": {
      "phases": {
        "warm": {
          "min_age": "2d",
          "actions": {
            "shrink": {
              "number_of_shards": 1
            },
            "forcemerge": {
              "max_num_segments": 1
            }
          }
        },
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_primary_shard_size": "50gb",
              "max_age": "7d"
            }
          }
        },
        "delete": {
          "min_age": "7d",
          "actions": {
            "delete": {
              "delete_searchable_snapshot": true
            }
          }
        }
      },
      "_meta": {
        "managed": true,
        "description": "built-in ILM policy using the hot and warm phases with a retention of 7 days"
      }
    },
    "in_use_by": {
      "indices": [],
      "data_streams": [],
      "composable_templates": []
    }
  },
  ".deprecation-indexing-ilm-policy": {
    "version": 1,
    "modified_date": "2022-04-20T14:56:54.304Z",
    "policy": {
      "phases": {
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_primary_shard_size": "10gb",
              "max_age": "30d"
            }
          }
        },
        "delete": {
          "min_age": "30d",
          "actions": {
            "delete": {
              "delete_searchable_snapshot": true
            }
          }
        }
      },
      "_meta": {
        "managed": true,
        "description": "ILM policy used for deprecation logs"
      }
    },
    "in_use_by": {
      "indices": [],
      "data_streams": [],
      "composable_templates": [
        ".deprecation-indexing-template"
      ]
    }
  },
  "metrics": {
    "version": 1,
    "modified_date": "2022-04-20T14:56:53.987Z",
    "policy": {
      "phases": {
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_primary_shard_size": "50gb",
              "max_age": "30d"
            }
          }
        }
      },
      "_meta": {
        "managed": true,
        "description": "default policy for the metrics index template installed by x-pack"
      }
    },
    "in_use_by": {
      "indices": [],
      "data_streams": [],
      "composable_templates": [
        "metrics"
      ]
    }
  },
  "ml-size-based-ilm-policy": {
    "version": 1,
    "modified_date": "2022-04-20T14:56:53.928Z",
    "policy": {
      "phases": {
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_primary_shard_size": "50gb"
            }
          }
        }
      },
      "_meta": {
        "managed": true,
        "description": "default policy for machine learning state and stats indices"
      }
    },
    "in_use_by": {
      "indices": [],
      "data_streams": [],
      "composable_templates": [
        ".ml-state",
        ".ml-stats"
      ]
    }
  },
  "180-days-default": {
    "version": 1,
    "modified_date": "2022-04-20T14:56:54.096Z",
    "policy": {
      "phases": {
        "warm": {
          "min_age": "2d",
          "actions": {
            "shrink": {
              "number_of_shards": 1
            },
            "forcemerge": {
              "max_num_segments": 1
            }
          }
        },
        "cold": {
          "min_age": "30d",
          "actions": {}
        },
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_primary_shard_size": "50gb",
              "max_age": "30d"
            }
          }
        },
        "delete": {
          "min_age": "180d",
          "actions": {
            "delete": {
              "delete_searchable_snapshot": true
            }
          }
        }
      },
      "_meta": {
        "managed": true,
        "description": "built-in ILM policy using the hot, warm, and cold phases with a retention of 180 days"
      }
    },
    "in_use_by": {
      "indices": [],
      "data_streams": [],
      "composable_templates": []
    }
  },
  "logs": {
    "version": 1,
    "modified_date": "2022-04-20T14:56:53.959Z",
    "policy": {
      "phases": {
        "hot": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_primary_shard_size": "50gb",
              "max_age": "30d"
            }
          }
        }
      },
      "_meta": {
        "managed": true,
        "description": "default policy for the logs index template installed by x-pack"
      }
    },
    "in_use_by": {
      "indices": [],
      "data_streams": [],
      "composable_templates": [
        "logs"
      ]
    }
  }
}

Can anyone help about that?
Thanks.

2

Welcome to our community! :smiley:

Can you elaborate what indices are being deleted?

3

Thanks!
All indices are deleted. Here is a full list.

[2022-09-28T10:06:03,572][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [ucarmak_products_production_20220927195522798/F8U6htWXR
WWtzmQMaPpJaQ] deleting index
[2022-09-28T10:06:06,422][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [tasmakina_products_production_20220927193759937/LRs6jHZ
OSBCNVq_rV4AZUA] deleting index
[2022-09-28T10:06:09,470][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [sutkonagi_products_production_20220927182657047/D46hFQs
kQS2cYCc7-LVo7A] deleting index
[2022-09-28T10:06:12,421][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [stokdevi_products_production_20220927193058370/eYV2OMMm
QZujtELpalWgSw] deleting index
[2022-09-28T10:06:15,389][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [rainsskyline_products_production_20220927191940885/GyPK
hKAQS-uJlBx41nmf4g] deleting index
[2022-09-28T10:06:18,185][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [rainsparadise_products_production_20220927200614996/S_IGwDcjQTiivK_D0vamuQ] deleting index
[2022-09-28T10:06:21,060][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [niyugen_products_production_20220927192607522/ygWcBeYkQkSIRsFA-u7pZQ] deleting index
[2022-09-28T10:06:24,421][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [nezahatsahin_products_production_20220927191616025/WLou-DFwS0ali7qCRTOqtw] deleting index
[2022-09-28T10:06:27,186][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [neroenoir_products_production_20220927194552037/AJEFPrjSQDKEDONd1BPbVQ] deleting index
[2022-09-28T10:06:30,140][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [muratoglu_products_production_20220927192132430/Wwss9lC5QWWn9bBDfKaKcg] deleting index
[2022-09-28T10:06:33,324][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [mobilyadevi_products_production_20220927194500497/P_O1IF18TBiHeUaN6Rpcug] deleting index
[2022-09-28T10:06:37,541][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [mlbutik_products_production_20220927195226691/2WKv-CxfRcy9XuwO1sDkSA] deleting index
[2022-09-28T10:06:40,257][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [lensver_products_production_20220927191307217/q9dGnKk0S1GLs-_vhxPaOA] deleting index
[2022-09-28T10:06:43,494][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [hilaltokalak_products_production_20220927194003970/s5djdW0pToChEUdZtjmHTQ] deleting index
[2022-09-28T10:06:47,392][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [guneskuyumcu_products_production_20220927190120719/YmsqMacsRWOTmrUwjrOo5w] deleting index
[2022-09-28T10:06:51,459][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [gunday_products_production_20220927194405107/FoQ1aayyRKW9yV_xjdJXCg] deleting index
[2022-09-28T10:06:55,198][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [esnastore_products_production_20220927193647104/XHXdwrV9QuimJ8p8ajmIsA] deleting index
[2022-09-28T10:06:58,047][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [erbaatugla_products_production_20220927200617511/qwSWBlz7TxOSRDNsX4Kl4w] deleting index
[2022-09-28T10:07:05,507][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [emilico_products_production_20220927192740502/nw381unORTC-0qlS0OxnIQ] deleting index
[2022-09-28T10:07:09,117][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [dkbutik_products_production_20220927195359602/EUqLUOB4Qku5UZgTpDoxpQ] deleting index
[2022-09-28T10:07:12,307][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [yeninesilaktar_products_production_20220927190227549/khXm41nmSaqfmXXXNo-dnw] deleting index
[2022-09-28T10:07:17,219][INFO ][o.e.c.m.MetadataDeleteIndexService] [node-1] [defnemine_products_production_20220927195703686/JHFEL_iZStiqIJfsLurkrQ] deleting index
4

What version of Elasticsearch are you using?
Why do you have Security disabled?

5

Version 8.1.3.
Security disabled because I didn't need it. I think. Because 9200 port disabled for outside access. Only docker containers can access the port.

6

Well something outside of Elasticsearch is asking for these indices to be deleted, as it's not Elasticsearch deleting them.

You might want to enable security so you can control what is happening.

7

Is port 9300 open to outside access?

Have you verified that the IP addresses listed in the logs are the one you would expect to connect to your cluster?

Have you had any new indices unexpectedly created?

It does look like something or someone is deleting indices so if you do not have an bug in some application it is possible your cluster is not as secured as you think it is.

8

Is port 9300 open to outside access?

Hey thanks for reply. When I check it, I realized the port 9200 open to outside, somehow. Thanks for all replies and sorry for taking your time.

9

You might want to enable security so you can control what is happening.

You're right, that's my bad. Thanks for replies.

10

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.