Index Lifecycle Management Metricbeat

Hi @Ankita_Pachauri

What version?

Below this is for 8.x but should work for 7.17 as well...

And I always add that although renaming default indices is supported however if you are just getting to know The Elastic Stack there are tons of reasons to just use the defaults, when you move away from the defaults it is critical you understand all those relationship.

Another approach is simply tagging these host(s) with a tag
tags: ["metricbeateoprodsec"]

and leave everything as default and just filter by the tag in each of the areas and set the metric ILM policy to what you want

These are all the things affected when changing index / data stream names

  • Data Stream
  • Index Template
  • ILM Policy
  • Dashboards
  • Infrastructure App.
  • Probably more I am not remembering

So assuming you want to do this...Leverage the framework

Here is my fully functional metricbeat.yml

You will want to run setup Once then turn off the overwrite and dashboards loading.

Run setup first
filebeat setup -e

then start filebeat
filebeat -e
or
systemctl start filebeat

# =========================== Modules configuration ============================

metricbeat.config.modules:
  # Glob pattern for configuration loading
  path: ${path.config}/modules.d/*.yml

  # Set to true to enable config reloading
  reload.enabled: false

  # Period on which files under path should be checked for changes
  #reload.period: 10s

# ======================= Elasticsearch template setting =======================

setup.template.enabled: true  
setup.template.name: "metricbeateoprodsec-%{[agent.version]}" 
setup.template.pattern: "metricbeateoprodsec-%{[agent.version]}" 
setup.template.overwrite: true ## <- Set this to true Once then take out or set to false. 
setup.ilm.policy_name: metricbeateoprodsec-policy 
setup.template.settings:
  index.number_of_shards: 1
  index.number_of_replicas: 1
  index.codec: best_compression
  #_source.enabled: false

# Important if you want the dashboards to work
# Set this to true Once then set to false. Or comment out BOTH lines, Otherwise metricbeat will reload the dashboards EVERY TIME!
setup.dashboards.enabled: true 
setup.dashboards.index: "metricbeateoprodsec-*"


# =================================== Kibana ===================================

setup.kibana:

# ---------------------------- Elasticsearch Output ----------------------------
output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["http://localhost:9200"]
  index: "metricbeateoprodsec-%{[agent.version]}"

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

Now you have a Data Stream, Index Template and ILM Policy and have set the Built in dashboards to work with all the changes

Data Stream

Index Template

ILM Policy

Now for the ILM you have an ILM Policy go in and adjust as you wish

Don't forget to fix the Infrasture App (No Spaces)

Dashboards