Changing RAM/DISK for “logging-and-metrics” leads to an health issue

[apaulsen (Agnes)]

We saw high memory pressure and disk usage on logging-and-metrics.

We've decided to increase RAM/Disk via UI, editing the DATA setting to 2GB and 64GB disk space.
Intermediately we’ve got a split brain situation.

We've restarted the deployment and got the following picture:

How to get rid of the old DATA_DEFAULT (her in ZONE-3)?.

TIA,
Agnes

[Alex_Piggott]

Normal steps are:

• Turn system_owned to false in the advanced editor (data section)
• Override the instance capacity for the "overwhelmed" node
  • (ensure you have sufficient capacity on the allocator first, eg put it into maint mode, move off other instances as needed to get enough space .. eg looks like 1GB would be sufficient here)
  • Later version of ECE let you use the UI (there would be three dots in the top right of the instance boxes after turning system owned off if so) ...
    • if not you have to use the API: https://www.elastic.co/guide/en/cloud-enterprise/2.2/Clusters_-Elasticsearch-CRUD-_Configuration.html#set-es-cluster-instances-settings-overrides (you need restart_after_update=true)
• Now the cluster should recover. Then you can reapply the original "upscale plan" and it should work

[apaulsen (Agnes)]

@Alex_Piggott

• Turn system_owned to false in the advanced editor (data section)
• Override the instance capacity for the "overwhelmed" node

Perfect! "logging-and-metrics" is back to normal mode with more memory and disk space.

( + I turned system_owned afterwards back to true)

Thanks!!