FORCE index rollover

Elastic Stack Elasticsearch
1

Hello,
I sheep logs from multiples servers via the winlogbeat agent that are stored with index name ecs-agent-windows. This index is managed by a test0 policy (ILM) and configured with the agent-wingbeat template.

The logs are always added to ecs-agent-windows, even though the index is at the complete phase.

Is there any why to force the rollover the index with a new index name serie to resolve this issue from elasticsearch, knowing that I don’t have access to winlogbeat agent.

The index settings

> {
>   "ecs-agent-windows": {
>     "settings": {
>       "index": {
>         "lifecycle": {
>           "name": "test0",
>           "rollover_alias": "ecs-agent-windows1",
>           "indexing_complete": "true"
>         },
>         "routing": {
>           "allocation": {
>             "include": {
>               "_tier_preference": "data_warm,data_hot"
>             }
>           }
>         },
>         "number_of_shards": "1",
>         "provided_name": "ecs-agent-windows",
>         "creation_date": "1663002046263",
>         "priority": "50",
>         "number_of_replicas": "0",
>         "uuid": "OgE-V8WVRBqW56Hwwmo5Bg",
>         "version": {
>           "created": "8030299"
>         }
>       }
>     }
>   }
> }

The policy settings

{
"test0": {
"version": 1,
"modified_date": "2022-10-05T17:17:50.872Z",
"policy": {
"phases": {
"warm": {
"min_age": "1s",
"actions": {
"set_priority": {
"priority": 50
}
}
},
"hot": {
"min_age": "0ms",
"actions": {
"set_priority": {
"priority": 100
},
"rollover": {
"max_primary_shard_size": "50gb",
"max_age": "10m",
"max_docs": 500
}
}
}
}
},
"in_use_by": {
"indices": [
"ecs-agent-windows",
"ecs-agent-filebeat-2022.10.05-000002",
"ecs-agent-filebeat-2022.10.05-000001",
"ecs-agent-filebeat"
],
"data_streams": ,
"composable_templates": [
"ecs-agent-windows",
"agent-filebeat",
"agent-winlogbeat"
]
}
}
}

The ILM explain:

> {
>   "indices": {
>     "ecs-agent-windows": {
>       "index": "ecs-agent-windows",
>       "managed": true,
>       "policy": "test0",
>       "index_creation_date_millis": 1663002046263,
>       "time_since_index_creation": "23.8d",
>       "lifecycle_date_millis": 1663002046263,
>       "age": "23.8d",
>       "phase": "warm",
>       "phase_time_millis": 1665015792294,
>       "action": "complete",
>       "action_time_millis": 1665016742506,
>       "step": "complete",
>       "step_time_millis": 1665016742506,
>       "phase_execution": {
>         "policy": "test0",
>         "phase_definition": {
>           "min_age": "1s",
>           "actions": {
>             "set_priority": {
>               "priority": 50
>             }
>           }
>         },
>         "version": 1,
>         "modified_date_in_millis": 1664990270872
>       }
>     }
>   }
> }

Best regards,

2

Hey, if i understood right you want to add rotation capability to your indices.

You can try adding date at the end of your indices name, so it would be limited with 24 hour. New logs will be added to new indices.

Just change your elasticsearch output within logstash config as below;

index           => "ecs-agent-windows-%{+YYYY.MM.dd}"
3

Hi FALEN,

Thanks for you replay.
The output of the winlogbeat agent not point to logstash, it store data directly in elasticsearch. My need is to rollover the index automaticaly via the ilm policy defined.

Best regards,

4

If you are not using Logstash, then you should edit the question and change the tag to direct it to the correct forum.

1 Like
5

Hello leandrojmp,

It was a mistake, done.

Best regards,

6

Hello,
Can I have an expert who can help me to solve this probleme

Regards,

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.