Use logstash to connect VMware vCenter API?

Elastic Stack Logstash
1

I'm trying to utilize the elastic logstash to obtain VMware vcenter datacenter metrics via API but failed to connect them.
The vCenter version is 8.0.1.
I had successfully configured "syslog" as input and recieved logs from vCenter before therefore I assumed that the basic connection is good.

However when i try "http_poller" to access vCenter API, it didn't work. No data is recieved.
The basic codes that i utilized in the input section are shown below:

input{
      http_poller {
                   rls => {
    	                   vcenter_api => {
        			                       method => get
        			                       url => "https://<vcenter_IP>/rest/vcenter"
       				                                headers => {
						                                       "Content-Type" => "application/json"
           					                                    }
            			                   user => "<vcenter_account>"
            			                   password => "<vcenter_account_password>"
            			                   }
                           }
                           request_timeout => 60
                           schedule => { cron => "* * * * * UTC" }      #run every minutes
                           codec => "json"
                   }
      }

I'm new to the elastic world, can anyone provide some hints please?

2

For Metrics I would highly recommend the metricbeat VSphere module

You can setup

Metricbeat -> Elasticsearch (Highly Recommend doing this first!!)

or more complex

Metricbeat -> Logstash -> Elasticsearch

To Get started use the Metricbeat Quickstart Guide (Follow it closely)

And instead of the sample nginx module use the vsphere module

3

This is nice! :grinning:
Thx for the help.

The reason why I wanna connect logstash to vSphere is due to the need for monitoring datacenter events and metrics, such as audit failure or disk utilization performance...etc, and the "syslog" is limited to do so.
If the metricbeat vsphere module is the best way to achieve this then I'm going to research it now.

By the way, is there any recommended modules/methods to utilize if i wanna do the same thing to other VMware products?(e.g. vRealize, NSX)

4

Filebeat or logstash for log files like syslog etc

Metricbeat for metrics.

And of course there's the new elastic agent that has many integrations, but that's another subject.

As to your other systems. I don't think there's out of the box solutions, but if there are log shippers, log files, Prometheus scrapers or other shippers elastic is open, you just need to POST json to it.

5

thank you stephen, the information is helpful.
may i confirm some last details please?
According to "Quick start: installation and configuration", if I want to acquire metrics from VMware vCenter server then the typical metricbeat should be installed on it? (the vcenter is utilizing Photon OS so i assume it's supported.)

The "vSphere module"(and also the "modules lists") only described that three metricsets are supported(datastore, vm, host), so currently these are the only options to collect data from?

6

No, it doesn't need to be installed directly on the ESX host. Just a VM that can access the vCenter API.

That's the only metrics that have a predefined module.

You can use any other method as long as you can ship the metrics. I'm not that familiar with all the vCenter / vSphere metrics.

Elastic can also ingest Prometheus metrics using metricbeat.

7

this is a little bit suprising, I thought installing metricbeat on the vCenter server is good enough and the overall topology would be simpler.

8

I am not quite clear what the "This" is that is surprising

But You can run metricbeat wherever you want... I am just not sure metricbeat is formally supported on Photon OS.

See the support matrix here

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.