Dynamic namespace on APM integration

[juliana_cg (Juliana)]

Hello,

I'm using Fleet and Elastic Agent on ES 7.17.1. Before, we used an apm standalone server and we could decide how to name our indexes thanks to parameters on Elasticsearch output, like this:

  indices:
    - index: "apm-%{[observer.version]}-%{[kubernetes.namespace]}-error"
      when.contains:
        processor.event: "error"

    - index: "apm-%{[observer.version]}-%{[kubernetes.namespace]}-transaction"
      when.contains:
        processor.event: "transaction"

It seems this parameters no longer work on fleet Elasticsearch output configuration or on APM integration. I thought we could handle it with the namespace option of the APM integration, but it seems you can only define one namespace per apm.

Is there a way to set up namespace dynamically, so it can be equal to our kubernates namespace?

I've thought of manually setting up index name with ingest pipelines, but since we now use datastreams instead of indices i'm not sure it'll work as expected.

I've also wondered if using set processor to set datastream.namespace to kubernetes.namespace would correctly route de data to proper datastream, and if so, which of the many apm ingest pipelines should be edited.

Thanks!

[stuart.nelson]

Hey, unfortunately this isn't supported due to limitation in the elastic-agent / fleet. You could definitely open a feature request in the elastic-agent repository. The best work-around at the moment is to have an agent policy per kubernetes namespace, and configure the apm integration with that desired namespace.

[juliana_cg (Juliana)]

I see, thanks for the suggestion. Do you think setting up an Elastic agent to output to logstash could also work out?