ILM problems

I have been spinning my wheels for a while on this. I can't figure out where I am missing something.

Logstash output

output {
#       ******* this is for debuging what has been parsed
#       file {
#               path => "/var/log/logstash/rubydebug"
#               codec => rubydebug
#       }

        elasticsearch {
                hosts => ["localhost:9200"]
                index => "alienvault"
}}

Here is the Index Template

{
  "index": {
    "lifecycle": {
      "name": "alienvault-policy"
    },
    "number_of_shards": "1",
    "auto_expand_replicas": "0-1",
    "number_of_replicas": "0"
  }
}

and here is the ILM policy

PUT _ilm/policy/alienvault-policy
    {
      "policy": {
        "phases": {
          "hot": {
            "min_age": "0ms",
            "actions": {
              "rollover": {
                "max_age": "1d"
              },
              "set_priority": {
                "priority": 100
              }
            }
          },
          "warm": {
            "actions": {}
          },
          "delete": {
            "min_age": "91d",
            "actions": {
              "delete": {}
            }
          }
        }
      }
    }

Seems like no matter what i do I get this error

I am hoping someone cat help direct me in what I am missing.

TIA

You need to create an initial index, see https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started-index-lifecycle-management.html

section - "To begin, we will want to bootstrap our first index to write to."

Substituting your names should be something like this:

PUT alienvault-000001
{
  "aliases": {
    "alienvault": {
      "is_write_index": true
    }
  }
}

I don't think you will have a problem with your existing "alienvault-" index, but if you don't need the data, delete it first. If you do need that data, after ingesting is going to alienvault-000001, you can reindex the old into the new, then delete the old.

Good luck :slight_smile:

So i did bootstrap it. Had no problem building alienvault-000001 and writing to it.

The error that comes up now is.

Index lifecycle error

illegal_argument_exception: setting [index.lifecycle.rollover_alias] for index [alienvault-000001] is empty or not defined

it is still writing to alienvault-000001

I did find a logstash setting but that did not seem to change anything

 elasticsearch {
                hosts => ["localhost:9200"]
                ilm_rollover_alias => "alienvault"
                ilm_pattern => "000001"
                ilm_policy => "alienvault-policy"



"alienvault-000001" : {
  "index" : "alienvault-000001",
  "managed" : true,
  "policy" : "alienvault-policy",
  "lifecycle_date" : "2020-01-22T21:15:05.426Z",
  "lifecycle_date_millis" : 1579727705426,
  "age" : "16.16h",
  "phase" : "hot",
  "phase_time" : "2020-01-23T13:08:57.537Z",
  "phase_time_millis" : 1579784937537,
  "action" : "rollover",
  "action_time" : "2020-01-22T21:18:24.572Z",
  "action_time_millis" : 1579727904572,
  "step" : "ERROR",
  "step_time" : "2020-01-23T13:18:24.472Z",
  "step_time_millis" : 1579785504472,
  "failed_step" : "check-rollover-ready",
  "step_info" : {
    "type" : "illegal_argument_exception",
    "reason" : "setting [index.lifecycle.rollover_alias] for index [alienvault-000001] is empty or not defined",
    "stack_trace" : """java.lang.IllegalArgumentException: setting [index.lifecycle.rollover_alias] for index [alienvault-000001] is empty or not defined
	at org.elasticsearch.xpack.core.ilm.WaitForRolloverReadyStep.evaluateCondition(WaitForRolloverReadyStep.java:50)
	at org.elasticsearch.xpack.ilm.IndexLifecycleRunner.runPeriodicStep(IndexLifecycleRunner.java:142)
	at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggerPolicies(IndexLifecycleService.java:304)
	at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggered(IndexLifecycleService.java:242)
	at org.elasticsearch.xpack.core.scheduler.SchedulerEngine.notifyListeners(SchedulerEngine.java:175)
	at org.elasticsearch.xpack.core.scheduler.SchedulerEngine$ActiveSchedule.run(SchedulerEngine.java:203)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:830)
"""
  },
  "phase_execution" : {
    "policy" : "alienvault-policy",
    "phase_definition" : {
      "min_age" : "0ms",
      "actions" : {
        "rollover" : {
          "max_age" : "1d"
        },
        "set_priority" : {
          "priority" : 100
        }
      }
    },
    "version" : 13,
    "modified_date" : "2020-01-23T12:57:13.062Z",
    "modified_date_in_millis" : 1579784233062
  }
}
  }
}

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.