# Ilm with Logstash - moving from hot-warm-cold-delete

**URL:** <https://discuss.elastic.co/t/ilm-with-logstash-moving-from-hot-warm-cold-delete/196070>\
**Category:** Logstash\
**Tags:** ilm-index-lifecycle-management\
**Created:** [August 21, 2019, 8:46am UTC](https://discuss.elastic.co/t/ilm-with-logstash-moving-from-hot-warm-cold-delete/196070 "2019-08-21T08:46:00Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![pastechecker](https://avatars.discourse-cdn.com/v4/letter/p/0ea827/32.png) [@pastechecker](https://discuss.elastic.co/u/pastechecker)\
**Post date:** [August 21, 2019, 8:46am UTC](https://discuss.elastic.co/t/ilm-with-logstash-moving-from-hot-warm-cold-delete/196070/1 "2019-08-21T08:46:00Z")

</div>

Hello.

I am a little bit confused about the order of operation in regards to the ILM and Logstash. I am preparing the configuration before attempting to do the ingestion. I will have a lot of data and want to plan in advance if I get the concept and necessary steps right.

I want to accomplish the following tasks:

- 
  1. All log data is indexed by Logstash directly to the hot nodes.

- 
  1. Want to move data from hot to warm and later to cold and discard.

**My steps will be:**

- **1**. Create index "lifecycle hot-warm-cold-delete-180days".
- **2**. Create index template "logs\_indexing".
- **3**. Create index alias "logs-000001" with write permissions.
- **4**. Enable ilm in logstash as per configuration below.

My questions:

- 
  1. Is this a proper order of operation?

- 
  1. Should I configure the index alias "logs-000001" in advance or the Logstash will create for me? I assume that yes, because the index name will follow the convention: "logs-2019.34", etc. so when Logstash will create the index, it will create the index plus the index\_alias.

- 
  1. As data will flow, will I get the next rolled over index named "logs-000002", "logs-000003", etc. without manual intervention having the Logstash configuration above?

**#logstash elasticsearch config**

> output {  
> elasticsearch {  
> id =\> "LOGS"  
> index =\> "logs-%{+xxxx.ww}"  
> hosts =\> ["localhost:9200"]  
> action =\> "index"  
> manage\_template =\> "true"  
> template\_name =\> "logs\_indexing"  
> ilm\_enabled =\> "true"  
> ilm\_rollover\_alias =\> "logs"  
> ilm\_pattern =\> "000001"  
> ilm\_policy =\> "logs\_rollover\_policy"  
> enable\_metric =\> "true"  
> }  
> }

**Where my template\_name "logs\_indexing" is:**

> {  
> "index\_patterns": ["logs\*"],  
> "mappings": {  
> "dynamic\_templates": [{  
> "strings": {  
> "match\_mapping\_type": "string",  
> "mapping": {  
> "type": "keyword"  
> }  
> }  
> }, {  
> "integers": {  
> "match\_mapping\_type": "long",  
> "mapping": {  
> "type": "integer"  
> }  
> }  
> }, {  
> "floating\_points\_double": {  
> "match\_mapping\_type": "double",  
> "mapping": {  
> "type": "float"  
> }  
> }  
> }  
> ]  
> },  
> "settings": {  
> "index": {  
> "number\_of\_shards": 1,  
> "number\_of\_replicas": 1,  
> "refresh\_interval": "120s",  
> "routing.allocation.require.data": "hot",  
> "lifecycle.name": "hot-warm-cold-delete-180days",  
> "lifecycle.rollover\_alias": "logs",  
> "codec": "best\_compression"  
> }  
> }  
> }

**My hot-warm-cold-delete-180days is:**

> {  
> "policy": {  
> "phases": {  
> "hot": {  
> "actions": {  
> "rollover": {  
> "max\_size":"50gb",  
> "max\_age":"3d"  
> },  
> "set\_priority": {  
> "priority": 50  
> }  
> }  
> },  
> "warm": {  
> "min\_age": "30d",  
> "actions": {  
> "forcemerge": {  
> "max\_num\_segments": 1  
> },  
> "shrink": {  
> "number\_of\_shards": 1  
> },  
> "allocate": {  
> "require": {  
> "data": "warm"  
> }  
> },  
> "set\_priority": {  
> "priority": 25  
> }  
> }  
> },  
> "cold": {  
> "min\_age": "120d",  
> "actions": {  
> "set\_priority": {  
> "priority": 0  
> },  
> "freeze": {},  
> "allocate": {  
> "require": {  
> "data": "cold"  
> }  
> }  
> }  
> },  
> "delete": {  
> "min\_age": "180d",  
> "actions": {  
> "delete": {}  
> }  
> }  
> }  
> }  
> }

**Alias index:**

> PUT logs-000001  
> {  
> "aliases": {  
> "logs":{  
> "is\_write\_index": true  
> }  
> }  
> }

Thank you for feedback.

---

<div class="post-metadata">

**Author:** ![pastechecker](https://avatars.discourse-cdn.com/v4/letter/p/0ea827/32.png) [@pastechecker](https://discuss.elastic.co/u/pastechecker)\
**Post date:** [August 27, 2019, 10:59am UTC](https://discuss.elastic.co/t/ilm-with-logstash-moving-from-hot-warm-cold-delete/196070/2 "2019-08-27T10:59:45Z")

</div>

I was wrong with the config.

output {  
elasticsearch {  
id =\> "LOGS"  
hosts =\> ["localhost:9200"]  
action =\> "index"  
ilm\_enabled =\> "true"  
ilm\_rollover\_alias =\> "logs"  
ilm\_pattern =\> "000001"  
ilm\_policy =\> "logs\_rollover\_policy"  
enable\_metric =\> "true"  
}  
}

The configuration above is being used at the moment.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 24, 2019, 11:12am UTC](https://discuss.elastic.co/t/ilm-with-logstash-moving-from-hot-warm-cold-delete/196070/3 "2019-09-24T11:12:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.