Manage ILM with ingestion pipeline

Hi there,

Is there a way to manage ILM when the index name is built in the ingestion pipeline?

In my scenario, metricbeats send data, specifing the target pipeline

output.elasticsearch:
  hosts: ["${ELASTICSEARCH_HOST}:${ELASTICSEARCH_PORT}"]
  username: "${ELASTICSEARCH_USERNAME}"
  password: "${ELASTICSEARCH_PASSWORD}"
  pipeline: "${ELASTICSEARCH_PIPELINE}"
  ssl:
    verification_mode: full
    certificate_authorities: ["/etc/ssl/elastic/tls.crt"]

then, the pipeline splits data based on the metric content.

PUT _ingest/pipeline/my_pipeline
{
  "processors": [
    {
      "set": {
        "field": "_index",
        "value": "my_index"
      }
    },
    {
      "set": {
        "if": "<something>",
        "field": "_index",
        "value": "{{_index}}_foo"
      }
    }
  ]
}

I need this in order to be able to give permissions to users only to a specific index set.

In this scenario, the first index does not exists before the first data and the ILM won't work.
Create the first index "manually" would be possible but very very tricky.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.