Unknown setting [index.lifecycle.name] please check that any required plugins

Elastic Stack Elasticsearch
1

Hello Team,

I am exploring ILM option but I am running into below error, can anyone shade some light?

following below doc:
https://www.elastic.co/guide/en/elasticsearch/reference/6.7/index-lifecycle-error-handling.html

PUT /ilm/policy/shrink-the-index
{
"policy": {
"phases": {
"warm": {
"min_age": "5d",
"actions": {
"shrink": {
"number_of_shards": 4
}
}
}
}
}
}

PUT /myindex
{
"settings": {
"index.number_of_shards": 2,
"index.lifecycle.name": "shrink-the-index"
}
}
ERROR:
{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "unknown setting [index.lifecycle.name] please check that any required plugins are installed, or check the breaking changes documentation for removed settings"
}
],
"type": "illegal_argument_exception",
"reason": "unknown setting [index.lifecycle.name] please check that any required plugins are installed, or check the breaking changes documentation for removed settings"
},
"status": 400
}

Cluster details:
[elk@elk2 templates]$ curl -XGET http://10.1.28.176:9200
{
"name" : "elk-2",
"cluster_name" : "elk-test",
"cluster_uuid" : "Q79e7lNpS56fzQJvQ8BLvQ",
"version" : {
"number" : "6.7.1",
"build_flavor" : "oss",
"build_type" : "tar",
"build_hash" : "2f32220",
"build_date" : "2019-04-02T15:59:27.961366Z",
"build_snapshot" : false,
"lucene_version" : "7.7.0",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}

Thanks
Chandra

2

ILM is part of the free Basic license, but you are running the OSS version which doesn't contain this.

3

How about HOT & WARM architecture features?

And I really appreciate if you can give example to write data into HOT node using templates from logstash.

I was looking at this document, is this available in OSS?

Thank
Chandra

4

Can anyone help me on this?

Thanks
Chandra

5

Read this and specifically the "Also be patient" part.

It's fine to answer on your own thread after 2 or 3 days (not including weekends) if you don't have an answer.

6

Is there a reason you don't want to run with a Basic license?

You can still setup hot/warm as per that blog post, but ILM is definitely the preferred path.

7

@dadoonet.. thanks will keep in mind :slight_smile:

Hi @warkolm, basic licence is free?

Thanks
Chandra

8

Hi Warkolm,

and where can I download free basic software? I don't see any option to download free basic version, all I see is elasticsearch/beta/alpha/oss

Thanks
Chandra

9

The default version contains a basic license.
Just download the default version from https://www.elastic.co/downloads/elasticsearch

10

Thanks @dadoonet but after I install default one I remember it was saying as trial version and it will expire after 14 days or so.

Do I have to purchase licence for basic? because in about 2 months we are planing to PROD.

Thanks
Chandra

11

You don't need to do anything to activate a basic license.
But if you activated a trial (30 days) then you need to revert to basic (you can do that from Kibana or using API) or buy a license if you want to use one or many features which are available under the commercial license.

12

Thanks @dadoonet.. Appreciate your help! will install basic then.

13

I noticed one more thing, I am not sure if I can ask this question in same ticket or you want me to open different ticket for this.

This is my node config

Node-1
"cluster_name" : "elk-test",
"nodes" : {
"szZvzywsRuGCTEzXhLU2-Q" : {
"name" : "elk-1",
"transport_address" : "10.1.28.175:9300",
"host" : "10.1.28.175",
"ip" : "10.1.28.175",
"version" : "6.7.1",
"build_flavor" : "oss",
"build_type" : "tar",
"build_hash" : "2f32220",
"total_indexing_buffer" : 421645516,
"roles" : [
"master",
"data",
"ingest"
],
"attributes" : {
"data" : "hot"
},
"settings" : {
"cluster" : {
"name" : "elk-test"
},
"node" : {
"attr" : {
"data" : "hot"
},
"name" : "elk-1"
},
"path" : {
"data" : [
"/opt/data/elastic",
"/home/elk/opt1/data/elastic_1"
],
Node-2
XZlIzGZkSe2BE_4LKNaOcQ" : {
"name" : "elk-2",
"transport_address" : "10.1.28.176:9300",
"host" : "10.1.28.176",
"ip" : "10.1.28.176",
"version" : "6.7.1",
"build_flavor" : "oss",
"build_type" : "tar",
"build_hash" : "2f32220",
"total_indexing_buffer" : 421645516,
"roles" : [
"master",
"data",
"ingest"
],
"attributes" : {
"data" : "warm"
},
"settings" : {
"cluster" : {
"name" : "elk-test"
},
"node" : {
"attr" : {
"data" : "warm"
},
"name" : "elk-2"
},
"path" : {
"data" : [
"/opt/data/elastic",
"/home/elk/opt1/data/elastic_1"
],

My Template:

[elk@elk2 44SXSucURSKaGSVxmoq1Uw]$ cat /opt/logstash/config/templates/swift_proxy_log_sizing.json
{
"template": "swift_proxy_logs",
"index_patterns": ["swift-proxy-logs-*"],
"settings": {
"index.routing.allocation.require.box_type": "hot",
"index.refresh_interval": "5s",
"index.codec": "best_compression",
"number_of_shards": 5,
"number_of_replicas": 0
},
"aliases": {
"swift_proxy_log_write_alias": {}
}
}
questions:

1. I am writing into HOT node index swift-proxy-logs- which is elk-1 as per my template config but I see data is writing into WARM node as well, anything I am doing wrong here?*

HOT Node: elk-1
[root@elk1 44SXSucURSKaGSVxmoq1Uw]# ll
total 0
drwxrwxr-x. 5 elk elk 46 May 9 13:05 0
drwxrwxr-x. 5 elk elk 46 May 9 13:05 2
drwxrwxr-x. 2 elk elk 23 May 9 13:06 _state
[root@elk1 44SXSucURSKaGSVxmoq1Uw]# cd /home/elk/opt1/data/elastic_1/nodes/0/indices/44SXSucURSKaGSVxmoq1Uw
[root@elk1 44SXSucURSKaGSVxmoq1Uw]# ll
total 0
drwxrwxr-x. 5 elk elk 46 May 9 13:05 4
drwxrwxr-x. 2 elk elk 23 May 9 13:06 _state
[root@elk1 44SXSucURSKaGSVxmoq1Uw]# du -sh /opt/data/elastic/nodes/0/indices/44SXSucURSKaGSVxmoq1Uw;du -sh /home/elk/opt1/data/elastic_1/nodes/0/indices/44SXSucURSKaGSVxmoq1Uw
1.6G /opt/data/elastic/nodes/0/indices/44SXSucURSKaGSVxmoq1Uw
785M /home/elk/opt1/data/elastic_1/nodes/0/indices/44SXSucURSKaGSVxmoq1Uw

WARM Node: elk-2
[elk@elk2 44SXSucURSKaGSVxmoq1Uw] ll total 0 drwxrwxr-x. 5 elk elk 46 May 9 13:05 1 drwxrwxr-x. 2 elk elk 23 May 9 13:06 _state [elk@elk2 44SXSucURSKaGSVxmoq1Uw] cd /home/elk/opt1/data/elastic_1/nodes/0/indices/44SXSucURSKaGSVxmoq1Uw
[elk@elk2 44SXSucURSKaGSVxmoq1Uw] ll total 0 drwxrwxr-x. 5 elk elk 46 May 9 13:05 3 drwxrwxr-x. 2 elk elk 23 May 9 13:06 _state [elk@elk2 44SXSucURSKaGSVxmoq1Uw] du -sh /opt/data/elastic/nodes/0/indices/44SXSucURSKaGSVxmoq1Uw;du -sh /home/elk/opt1/data/elastic_1/nodes/0/indices/44SXSucURSKaGSVxmoq1Uw
785M /opt/data/elastic/nodes/0/indices/44SXSucURSKaGSVxmoq1Uw
785M /home/elk/opt1/data/elastic_1/nodes/0/indices/44SXSucURSKaGSVxmoq1Uw

1. if you add up all the above index shards size its coming around 3 GB but cat index showing only 1.8 GB, why?
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open swift-proxy-logs-2019.05.09-1 44SXSucURSKaGSVxmoq1Uw 5 0 3357698 0 1.8gb 1.8gb

14

It's better to create a new topic on that question, as it's a totally different problem :wink:

15

No worries will do that, Thanks @warkolm

16

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.