VMware logging

Elastic Stack Logstash
1

Hey, all.

I want to get logs from my VMware hosts and VMs into logstash. I am getting logs from the hosts though, in truth, most of what I see is rather meaningless. I'm not able to figure out how to get individual VM logs (without setting up forwarders on each indiviual VM). Has anyone set up logging to capture both meaningful ESXi host and VM logs?

Thanks so much.

Diggy

2

What logs from inside the VMs are you interested in? And in what way are they accessible from the host server?

3

Magnus,

The best way to answer that is to give an example from a proprietary log aggregator/analyzer that we've trialed. It uses a custom API to pull logs from a vCenter server. I had to identify the vCenter server by IP address, and provide access user name and password. There was nothing I needed to do in vCenter. After that, I was able to look at "reports" (actually "canned" searches) that would, for example, tell me if a new folder was created in a ESX host, or if a VM was created, renamed, reconfigured, etc., or if there was a change to a datastore. Now, I realize that this may be an unfair comparison (it is, after all, an expensive, proprietary, application). However, given that most environments are highly virtualized, and that VMware is the platform of choice, it would be a great thing if ELK could return similar data.

A bit more experimentation (I can't do a lot of that in our current VMware implementation; we will be setting up a better test environment), but I did shut down an earlier version of an ELK VM, and got back the following:

message <166>2016-01-26T20:50:32.049Z dev-vmware.mydomain.local Hostd: [FFF81B90 info 'ha-eventmgr' opID=843656FD-000006D9-3-f9] Event 840 : Guest OS shut down for test-log-logstash on dev-vmware.NetAtlantic.local in ha-datacenter

So, there is some good stuff going on vis-a-vis VMware logs. But, I want to make it even better. Hence, my question remains: has anyone set up logging to capture both meaningful ESXi host and VM logs? Or, specifically to Magnus, are there any other config tweaks I can do to logstash to mine VMware fully?

Hope I'm clear about what I'm asking, and I'm not being a pita.

Diggy