DWbank
(Darin)
January 22, 2020, 8:47pm
1
I have been spinning my wheels for a while on this. I can't figure out where I am missing something.
Logstash output
output {
# ******* this is for debuging what has been parsed
# file {
# path => "/var/log/logstash/rubydebug"
# codec => rubydebug
# }
elasticsearch {
hosts => ["localhost:9200"]
index => "alienvault"
}}
Here is the Index Template
{
"index": {
"lifecycle": {
"name": "alienvault-policy"
},
"number_of_shards": "1",
"auto_expand_replicas": "0-1",
"number_of_replicas": "0"
}
}
and here is the ILM policy
PUT _ilm/policy/alienvault-policy
{
"policy": {
"phases": {
"hot": {
"min_age": "0ms",
"actions": {
"rollover": {
"max_age": "1d"
},
"set_priority": {
"priority": 100
}
}
},
"warm": {
"actions": {}
},
"delete": {
"min_age": "91d",
"actions": {
"delete": {}
}
}
}
}
}
Seems like no matter what i do I get this error
I am hoping someone cat help direct me in what I am missing.
TIA
rugenl
(Len Rugen)
January 22, 2020, 8:59pm
2
You need to create an initial index, see https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started-index-lifecycle-management.html
section - "To begin, we will want to bootstrap our first index to write to."
Substituting your names should be something like this:
PUT alienvault-000001
{
"aliases": {
"alienvault": {
"is_write_index": true
}
}
}
I don't think you will have a problem with your existing "alienvault-" index, but if you don't need the data, delete it first. If you do need that data, after ingesting is going to alienvault-000001, you can reindex the old into the new, then delete the old.
Good luck
DWbank
(Darin)
January 23, 2020, 12:20pm
3
So i did bootstrap it. Had no problem building alienvault-000001 and writing to it.
The error that comes up now is.
Index lifecycle error
illegal_argument_exception: setting [index.lifecycle.rollover_alias] for index [alienvault-000001] is empty or not defined
it is still writing to alienvault-000001
I did find a logstash setting but that did not seem to change anything
elasticsearch {
hosts => ["localhost:9200"]
ilm_rollover_alias => "alienvault"
ilm_pattern => "000001"
ilm_policy => "alienvault-policy"
"alienvault-000001" : {
"index" : "alienvault-000001",
"managed" : true,
"policy" : "alienvault-policy",
"lifecycle_date" : "2020-01-22T21:15:05.426Z",
"lifecycle_date_millis" : 1579727705426,
"age" : "16.16h",
"phase" : "hot",
"phase_time" : "2020-01-23T13:08:57.537Z",
"phase_time_millis" : 1579784937537,
"action" : "rollover",
"action_time" : "2020-01-22T21:18:24.572Z",
"action_time_millis" : 1579727904572,
"step" : "ERROR",
"step_time" : "2020-01-23T13:18:24.472Z",
"step_time_millis" : 1579785504472,
"failed_step" : "check-rollover-ready",
"step_info" : {
"type" : "illegal_argument_exception",
"reason" : "setting [index.lifecycle.rollover_alias] for index [alienvault-000001] is empty or not defined",
"stack_trace" : """java.lang.IllegalArgumentException: setting [index.lifecycle.rollover_alias] for index [alienvault-000001] is empty or not defined
at org.elasticsearch.xpack.core.ilm.WaitForRolloverReadyStep.evaluateCondition(WaitForRolloverReadyStep.java:50)
at org.elasticsearch.xpack.ilm.IndexLifecycleRunner.runPeriodicStep(IndexLifecycleRunner.java:142)
at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggerPolicies(IndexLifecycleService.java:304)
at org.elasticsearch.xpack.ilm.IndexLifecycleService.triggered(IndexLifecycleService.java:242)
at org.elasticsearch.xpack.core.scheduler.SchedulerEngine.notifyListeners(SchedulerEngine.java:175)
at org.elasticsearch.xpack.core.scheduler.SchedulerEngine$ActiveSchedule.run(SchedulerEngine.java:203)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:830)
"""
},
"phase_execution" : {
"policy" : "alienvault-policy",
"phase_definition" : {
"min_age" : "0ms",
"actions" : {
"rollover" : {
"max_age" : "1d"
},
"set_priority" : {
"priority" : 100
}
}
},
"version" : 13,
"modified_date" : "2020-01-23T12:57:13.062Z",
"modified_date_in_millis" : 1579784233062
}
}
}
}
system
(system)
Closed
February 20, 2020, 12:20pm
4
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.