Overwriting supplied index micro-%{appName}%{+YYYY.MM.dd} with rollover alias vehicle-service

Elastic Stack Logstash
1

input {
tcp {
port => 5000
codec => json
}
}

output {
if [appName] =="user-service"{
elasticsearch {
hosts => ["http://localhost:9200"]
index => "micro-%{appName}%{+YYYY.MM.dd}" # Use date-based index names
ilm_enabled => true # Enable ILM
ilm_rollover_alias => "user-service" # Define an alias for the rollover
}
}
else if [appName] == "vehicle-service" {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "micro-%{appName}%{+YYYY.MM.dd}"
ilm_enabled => true # Enable ILM
ilm_rollover_alias => "vehicle-service" # Define an alias for the rollover
}
}
else if [appName] == "search-service" {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "micro-%{appName}%{+YYYY.MM.dd}"
ilm_enabled => true # Enable ILM
ilm_rollover_alias => "search-service" # Define an alias for the rollover
}
}
else if [appName] == "common-service" {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "micro-%{appName}%{+YYYY.MM.dd}"
ilm_enabled => true # Enable ILM
ilm_rollover_alias => "common-service" # Define an alias for the rollover
}
}
else if [appName] == "api-gateway" {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "micro-%{appName}%{+YYYY.MM.dd}"
ilm_enabled => true # Enable ILM
ilm_rollover_alias => "api-gateway" # Define an alias for the rollover
}
}
else if [appName] == "asset-service" {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "micro-%{appName}%{+YYYY.MM.dd}"
ilm_enabled => true # Enable ILM
ilm_rollover_alias => "asset-service" # Define an alias for the rollover
}
}
else if [appName] == "booking-service" {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "micro-%{appName}%{+YYYY.MM.dd}"
ilm_enabled => true # Enable ILM
ilm_rollover_alias => "booking-service" # Define an alias for the rollover
}
}
else if [appName] == "discovery-server" {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "micro-%{appName}%{+YYYY.MM.dd}"
ilm_enabled => true # Enable ILM
ilm_rollover_alias => "discovery-server" # Define an alias for the rollover
}
}
else if [appName] == "fare-service" {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "micro-%{appName}%{+YYYY.MM.dd}"
ilm_enabled => true # Enable ILM
ilm_rollover_alias => "fare-service" # Define an alias for the rollover
}
}
else if [appName] == "qr-service" {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "micro-%{appName}%{+YYYY.MM.dd}"
ilm_enabled => true # Enable ILM
ilm_rollover_alias => "qr-service" # Define an alias for the rollover
}
}

}

2

You cannot use the index option if you set ilm_enabled to true. It gets overwritten. Sounds like this is working as designed.

3

if i am deleting the ilm_enabled line then the index is not created

i am doing this in logstash.conf

4

Please provide at least one complete example of one of your elasticsearch outputs (with secrets redacted) and what index you expect to it create. With so little information it is hard to answer.

5

[2023-12-01T09:25:28,500][INFO ][logstash.outputs.elasticsearch][main] Using a default mapping template {:es_version=>8, :ecs_compatibility=>:v8}
[2023-12-01T09:25:28,507][WARN ][logstash.outputs.elasticsearch][main] Restored connection to ES instance {:url=>""}
[2023-12-01T09:25:28,509][INFO ][logstash.outputs.elasticsearch][main] Elasticsearch version determined (8.11.1) {:es_version=>8}
[2023-12-01T09:25:28,509][WARN ][logstash.outputs.elasticsearch][main] Detected a 6.x and above cluster: the type event field won't be used to determine the document _type {:es_version=>8}
[2023-12-01T09:25:28,518][INFO ][logstash.outputs.elasticsearch][main] Not eligible for data streams because config contains one or more settings that are not compatible with data streams: {"index"=>"micro-%{appName}%{+YYYY.MM.dd}", "ilm_rollover_alias"=>"asset-service", "ilm_enabled"=>"true"}
[2023-12-01T09:25:28,519][INFO ][logstash.outputs.elasticsearch][main] Data streams auto configuration (data_stream => auto or unset) resolved to false
[2023-12-01T09:25:28,530][WARN ][logstash.outputs.elasticsearch][main] Overwriting supplied index micro%{+YYYY.MM.dd} with rollover alias vehicle-service2

i want my indices to create appending date in name and i am using elk and giving name of indices in logstash.conf , as you can see above is my all logstash.conf file so if i use ilm_enabled then my indices are overrided by ilm_rollover_alias and if i don't use then it is not created
also i am new to Elasticsearch my requirement is to create elk stack to see logs and to make snapshots or backup of logs older than 1 month so when i restore the logs it conflicts with the name of indices as my indices name is yellow open vehicle-service-2023.11.30-000001
and it is not changing with date so i need to change its name

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.