Should I put the index template & policy in ElasticSearch Cluster? or In Logstash?

I am having trouble finding the path to put my template and policy in ES or Logstash.

I know logstash stores default template path at vendor -> Path

When I referred ES 7.8.0 documentation for policy (Here) & it also says a Note :
Custom ILM policies must already exist on the Elasticsearch cluster before they can be used.

output {
  elasticsearch {
    ilm_rollover_alias => "custom"
    ilm_pattern => "000001"
    ***ilm_policy => "custom_policy"***
  }
}

Does ilm_policy accepts policy path or only policy id? (Check Documentation)

Both documents says policy should already exists on Elastic Cluster.

Can somebody tell me the path where exactly I need to copy my Policy and Template in Elastic Cluster?
or where should I put it in Logstash Cluster config folder?

I have also seen OpenDistro Documentaion (But it talks only about API)

My setup as of now:

I am using Opendistro(1.9.0) for ElasticSearch(7.8.0) along with Logstash(7.8.0) and Kibana on Kubernetes.

I created Index template (logstash-msd-template.json) & Policy (delete-logstash-msd-policy.json) and placed then on Logstash Pod under /config folder

bash-4.2$ pwd
/usr/share/logstash/config
bash-4.2$ ls -ltr
total 26
-rw-r--r-- 1 logstash root  286 Jun 14 21:28 pipelines.yml
-rw-rw-r-- 1 logstash root  342 Jun 14 21:28 logstash-sample.conf
-rw-r--r-- 1 root     root   49 Jul 24 13:45 logstash.yml
**-rw-r--r-- 1 root     root 1448 Jul 24 13:45 logstash-msd-template.json
-rw-r--r-- 1 root     root  982 Jul 24 13:45 delete-logstash-msd-policy.json**
bash-4.2$

and here is my Logstash Output Config and referenced it as below:

    output {
    elasticsearch {
        template_overwrite => true
        manage_template => true
        template => "/usr/share/logstash/config/logstash-msd-template.json"
        template_name => "logstash-msd-template"
        **. . .**
        ilm_enabled => false
        index => "logstash-%{[@metadata][tenant]}-blah-%{+YYYY.MM.dd}"
      }
    }

My Index Template (logstash-msd-template.json): In which I have referenced policy as ("opendistro.index_state_management.policy_id": "delete-logstash-msd-policy.json")

{
 "order": 1,
 "version": 60001,
 "index_patterns": [
 "logstash-msd-*"
 ],
"settings": {
"opendistro.index_state_management.policy_id": "delete-logstash-msd-policy",
"index": {
  "number_of_shards": "1",
  "refresh_interval": "5s"
}
},
 "mappings": {
 "dynamic_templates": [
  {
    "message_field": {
      "path_match": "message",
      "mapping": {
        "norms": false,
        "type": "text"
      },
      "match_mapping_type": "string"
    }
  },
  {
    "string_fields": {
      "mapping": {
        "norms": false,
        "type": "text",
        "fields": {
          "keyword": {
            "ignore_above": 256,
            "type": "keyword"
          }
        }
      },
      "match_mapping_type": "string",
      "match": "*"
    }
  }
],
"properties": {
  "@timestamp": {
    "type": "date"
  },
  "geoip": {
    "dynamic": true,
    "properties": {
      "ip": {
        "type": "ip"
      },
      "latitude": {
        "type": "half_float"
      },
      "location": {
        "type": "geo_point"
      },
      "longitude": {
        "type": "half_float"
      }
    }
  },
  "@version": {
    "type": "keyword"
  }
}
},
"aliases": {}
}

My Policy (delete-logstash-msd-policy.json):

{
  "policy": {
  "policy_id": "delete-logstash-msd-policy",
  "description": "A simple default policy that changes the states of msd indexes",
  "last_updated_time": 1595340380667,
  "schema_version": 1,
  "error_notification": null,
  "default_state": "hot",
  "states": [
  {
    "name": "hot",
    "actions": [],
    "transitions": [
      {
        "state_name": "delete",
        "conditions": {
          "min_index_age": "2h"
        }
      }
    ]
  },
  {
    "name": "delete",
    "actions": [
      {
        "delete": {}
      }
    ],
    "transitions": []
  }
]
}
}

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.