How to avoid daily creation of .monitoring system indices (v7.14)

Elastic Stack Elasticsearch
1

Hi All,

I see daily system indices (.monitoring...) getting created. How can we avoid creation of these indices daily and instead create them on monthly basis.

I am aware of ILM and using it for application indices to rollover based on some condition.

green  open   .monitoring-es-7-2021.10.01       SAO_SkHbSxmqT0YUSQw   1   0     626178       181374    304.9mb        304.9mb
green  open   .monitoring-es-7-2021.10.03       FGVARsvWmZb9N_lQ7Vw   1   0     624380       181460    305.2mb        305.2mb
green  open   .monitoring-es-7-2021.10.02       nbHdeGWgQHHLOom3aA   1   0     625650       181890    304.3mb        304.3mb
green  open   .monitoring-kibana-7-2021.10.05   tVK2B5hG6HY0hkFTkOw   1   0      17278            0      2.6mb          2.6mb
green  open   .monitoring-kibana-7-2021.10.04   xvYEyR9aTHfTbp0dIjg   1   0      17280            0      2.7mb          2.7mb
green  open   .monitoring-kibana-7-2021.10.03   hyI8qT3m5qwhNqh-Lgg   1   0      17278            0      2.6mb          2.6mb
green  open   .monitoring-kibana-7-2021.10.02   bMYpHICzQJROBW2ylgQ   1   0      17280            0      2.6mb          2.6mb
green  open   .monitoring-kibana-7-2021.10.01   amPxFHlagpLOFA8DU8g   1   0      17280            0      2.7mb          2.7mb
green  open   .monitoring-kibana-7-2021.10.06   Ci1sGNHKRrij_dvXT3Q   1   0       5556            0   1009.3kb       1009.3kb
green  open   .monitoring-kibana-7-2021.09.30   0gZH3uRbTI25d_mnRMw   1   0      17278            0      2.6mb          2.6mb
green  open   .monitoring-logstash-7-2021.10.02 HFS4YT6YTVuuA4VWARQ   1   0    1001660            0     83.9mb         83.9mb
green  open   .monitoring-logstash-7-2021.10.01 a4-Gch4ZQvCE69VNLWw   1   0    1001776            0       84mb           84mb
green  open   .monitoring-logstash-7-2021.10.06 vChEl446QI_zM7LwNuw   1   0     322132            0     29.4mb         29.4mb
green  open   .monitoring-logstash-7-2021.10.05 EeeOKxrCS9KUfn4QeeSQ   1   0    1001776            0     84.2mb         84.2mb
green  open   .monitoring-logstash-7-2021.10.04 d7FIrIl5Ry2AGlplw2og   1   0    1001660            0     84.1mb         84.1mb
green  open   .monitoring-logstash-7-2021.10.03 ML8o7Ay-Qpnupq4mTYA   1   0    1001776            0       84mb           84mb
green  open   .monitoring-es-7-2021.09.30       zSBX2-3oQNCFzf2JrRRg   1   0     600528       143600    289.7mb        289.7mb
green  open   .monitoring-logstash-7-2021.09.30 0CvqSmj_T7aw_td1SOQ   1   0    1001660            0     84.4mb         84.4mb
green  open   .monitoring-es-7-2021.10.05       T7oViKrCQmekVNoQIXzw   1   0     625914       181632    306.5mb        306.5mb
green  open   .monitoring-es-7-2021.10.04       KOAaguTkTS25-xaCReZw   1   0     625562       181976    305.8mb        305.8mb
green  open   .monitoring-es-7-2021.10.06       Ldwc1RH2Q-SjXNHJow   1   0     201444       239638    119.4mb        119.4mb

for beats indices i have configured like below in logstash.conf, so they are getting created monthly.

/etc/logstash/conf.d/logstash.conf

.
.
.
output {
   elasticsearch {
    hosts => ['http://10.10.10.242:9200']
    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM}"
    user => elastic
    password => "${es_pwd}"
  }
}

yellow open   filebeat-7.14.0-2021.09           BY-6ohQuuEpTYc0JOrg   1   1    5561027            0      1.2gb          1.2gb
yellow open   filebeat-7.14.0-2021.10           7lgIaIwQTWGePj80_UQ   1   1     302390            0     98.6mb         98.6mb
yellow open   heartbeat-7.14.0-2021.09          EfiKRgjOSxuMFgGx9iw   1   1     922396            0    472.3mb        472.3mb
yellow open   metricbeat-7.14.0-2021.09         OD-HNSAaT-qCoK3z2LA   1   1    2648148            0      1.9gb          1.9gb
yellow open   heartbeat-7.14.0-2021.10          weD-2RRFTNHbAwj2uBg   1   1     462654            0    452.1mb        452.1mb
yellow open   metricbeat-7.14.0-2021.10         hCG-_J7gSrGQdl3YSRg   1   1    1260879            0        1gb            1gb

how can we do these for .monitoring indices?

Thanks,

2

I just noticed at any time there are only 7 such indices. So the older one are getting automatically deleted.

green  open   .monitoring-kibana-7-2021.10.05   tVK2B5zfRhY0hkFTkOw   1   0      17278            0      2.6mb          2.6mb
green  open   .monitoring-kibana-7-2021.10.04   xvY0ZPEaTHfTbp0dIjg   1   0      17280            0      2.7mb          2.7mb
green  open   .monitoring-kibana-7-2021.10.03   hyIp368m5qwhNqh-Lgg   1   0      17278            0      2.6mb          2.6mb
green  open   .monitoring-kibana-7-2021.10.02   bMYpHIJKzoROBW2ylgQ   1   0      17280            0      2.6mb          2.6mb
green  open   .monitoring-kibana-7-2021.10.01   amPxFHcagpLOFA8DU8g   1   0      17280            0      2.7mb          2.7mb
green  open   .monitoring-kibana-7-2021.10.06   Ci1sGNrij_dvz83XT3Q   1   0       5556            0        1mb            1mb
green  open   .monitoring-kibana-7-2021.09.30   0gZH3uI2Lsg5d_mnRMw   1   0      17278            0      2.6mb          2.6mb

green  open   .monitoring-logstash-7-2021.10.02 HFS4YTVu4jquA4VWARQ   1   0    1001660            0     83.9mb         83.9mb
green  open   .monitoring-logstash-7-2021.10.01 a4-4ZQvGEACE69VNLWw   1   0    1001776            0       84mb           84mb
green  open   .monitoring-logstash-7-2021.10.06 vChE6QImNC_zM7LwNuw   1   0     322132            0     30.8mb         30.8mb
green  open   .monitoring-logstash-7-2021.10.05 EeerCS9K_4Ufn4QeeSQ   1   0    1001776            0     84.2mb         84.2mb
green  open   .monitoring-logstash-7-2021.10.04 d7FIrRy2AG5elplw2og   1   0    1001660            0     84.1mb         84.1mb
green  open   .monitoring-logstash-7-2021.10.03 MLAy-QVOTpnupq4mTYA   1   0    1001776            0       84mb           84mb
green  open   .monitoring-logstash-7-2021.09.30 0Cvj_T7aw1_tddP1SOQ   1   0    1001660            0     84.4mb         84.4mb
3

Yes. That's how it works by default. 7 days of monitoring data. So there's no need to change the default naming IMO.

In case you want to change something, you should read from How monitoring works | Elasticsearch Guide [7.15] | Elastic

If something is unclear, please tell which method you are using to monitor your cluster, from the legacy to the newest one.

1 Like
4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.