Hi there,
Is there a way to manage ILM when the index name is built in the ingestion pipeline?
In my scenario, metricbeats send data, specifing the target pipeline
output.elasticsearch:
hosts: ["${ELASTICSEARCH_HOST}:${ELASTICSEARCH_PORT}"]
username: "${ELASTICSEARCH_USERNAME}"
password: "${ELASTICSEARCH_PASSWORD}"
pipeline: "${ELASTICSEARCH_PIPELINE}"
ssl:
verification_mode: full
certificate_authorities: ["/etc/ssl/elastic/tls.crt"]
then, the pipeline splits data based on the metric content.
PUT _ingest/pipeline/my_pipeline
{
"processors": [
{
"set": {
"field": "_index",
"value": "my_index"
}
},
{
"set": {
"if": "<something>",
"field": "_index",
"value": "{{_index}}_foo"
}
}
]
}
I need this in order to be able to give permissions to users only to a specific index set.
In this scenario, the first index does not exists before the first data and the ILM won't work.
Create the first index "manually" would be possible but very very tricky.