Using Prometheus remote_write to send metrics to Elastic cloud

[Maduranga]

Hello,

I have a Prometheus server which is sending currently sending metrics to Grafana using remote_write.
I would like to use our Elastic cloud Kibana to get the Prometheus metrics.
Would like advice on how to authenticate and the URL for the remote_write.
What should be the URL and the authentication method for the Elastic cloud?

Thank you,
Maduranga

[stephenb (Stephen Brown)]

Perhaps take a look at this

Prometheus remote_write metricset | Metricbeat Reference [8.4] | Elastic

[Maduranga]

Thank you @stephenb
I checked that document but it doesn't mention what credentials we should set when connecting to Elastic cloud.

Thank you,
Maduranga

[stephenb (Stephen Brown)]

@Maduranga

Thank you @stephenb
I checked that document but it doesn't mention what credentials we should set when connecting to Elastic cloud.

Thank you,
Maduranga

That is an module...

The elastic cloud credential go in the metricbeat output section not the module sections

Configure the output for Elasticsearch Service on Elastic Cloud | Metricbeat...

In genereal metricbeat has

• General Settings
• Inputs or Modules <--- The prometheus moduls
• Outputs <--- Elastic Search Service Settings

[Maduranga]

Thank you again @stephenb
I am little confused.
I am trying to send the Prometheus metrics from Prometheus server to the Elastic cloud using prometheus.yml files remote_write.
The architecture in here: Prometheus Monitoring | Elastic
So there will be no metricbeat module in the Prometheus server.

I suppose the the URL for remote_write is the Elastic search endpoint URL, but I am not sure how to use the cloud.id and cloud.auth.

Regards,
Maduranga

[stephenb (Stephen Brown)]

Yes if you look at page you reference you must run metricbeat somewhere ... where you want to run it is up to you?

The Data Flow as shown is

Your App -> Prometheus Exporter -> Prometheus Server (Remote Write) -> Metricbeat With Prometheus Module (Install where you want) -> Elasticsearch Service on Elastic Cloud

You need to install and run Metricbeat to read the remote_write and then Ship the data y ... You could install it on the same server that the prometheus server or on just another server somewhere else

This picture shows Self Managed Elastic Search ... For Elastic Cloud you need to install metricbeat on you own VM Somewhere... I would put it on or near the prometheus server

[Maduranga]

Thank you @stephenb
I get it now. I am using a Fleet agent with Prometheus enabled.