Autodiscover for multiple dynamic ports per container

[mng12689 (Michael Ng)]

Hello,

I have metricbeat installed on an EC2 instance that contains multiple docker containers, each with an Envoy proxy inside. Each container exposes 2 Envoy ports on the host (one for application traffic and one for metrics), and both are dynamically allocated by ECS. I want metricbeat to pull Envoy metrics from each container on the metrics port, and I understand that Autodiscover is the tool of choice when dealing with containers and dynamic port mapping. However, since BOTH ports are dynamically mapped, its unclear how to tell autodiscover which one to choose (or even which one it would choose if left to its own devices?). Is there a way to configure autodiscover for my use case? The container port for the metrics could be static even if the host port exposing it is dynamic, so in theory it would be possible to select the correct host port given the host_port:container_port mapping exposed by a docker start event, but I currently see no way to do this.

Note: I may be able to configure Envoy to forward certain types of traffic to the metrics port, but that is undesirable for performance reasons (I don't want higher priority application traffic to contend for resources with metrics traffic).

[exekias (Carlos Pérez Aradros)]

Hi @mng12689

This is a fair request, I guess using ${data.port} in your template settings would still work, but you would see some errors from the module, as it would be trying to contact the wrong port in half of the launched configurations.

Why don't you hardcode the metrics port? take into account that you don't need to expose it, as Metricbeat should have access to the container port anyway.

Best regards