Metricbeat 7.11.1 fail in system fsstat and filesystem: The device is not ready

Sommerdal · February 26, 2021, 1:04pm

I keep getting this:
module/wrapper.go:259 Error fetching data for metricset system.fsstat: filesystem list: GetFilesystemType failed: GetVolumeInformationW failed: The device is
module/wrapper.go:259 Error fetching data for metricset system.filesystem: error getting filesystem list: GetFilesystemType failed: GetVolumeInformationW failed: The device is not ready.

The enviroment is:
agent.type metricbeat
agent.version 7.11.1
ecs.version 1.7.0
error.message error getting filesystem list: GetFilesystemType failed: GetVolumeInformationW failed: The device is not ready.
event.dataset system.filesystem
event.module system
host.architecture x86_64
host.os.build 9600.19542
host.os.family windows
host.os.kernel 6.3.9600.19538 (winblue_ltsb_escrow.191014-1700)
host.os.name Windows Server 2012 R2 Standard
host.os.platform windows
host.os.version 6.3
metricset.name filesystem
service.type system

anderssonjohan · March 1, 2021, 11:43am

We also have this issue when upgrading from 7.10 to 7.11.
The reason is this change to metricbeat: [Metricbeat] Filesystem Metricset: Inconsistent `system.filesystem.type` values on Windows · Issue #22501 · elastic/beats · GitHub

The change adds a call to Kernel32.GetVolumeInformationW to all drives on the system, but this will fail for certain drive types.

For example, on our system we get the drives A:\ C:\ D:\ L:\ where A is a floppy and D is a CD-ROM.
Both these two returns false when calling GetVolumeInformationW and that will stop metricbeat from working.

Right now you can only ignore certain filesystem types, but that filter will be applied on the result from making the syscall. Since the syscall fails, metricbeat stops reporting filesystem metrics.

anderssonjohan · February 26, 2021, 2:24pm

I posted a script here that confirms the issue on our system:

gist.github.com

https://gist.github.com/anderssonjohan/482ef06f5b40fc1c36c7d4f2ac1a107b

get-volumeinformation.ps1

$pinvokes = @'
[DllImport("Kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
  [return: MarshalAs(UnmanagedType.Bool)]
  public extern static bool GetVolumeInformationW(
    string rootPathName,
    System.Text.StringBuilder volumeNameBuffer,
    int volumeNameSize,
    out uint volumeSerialNumber,
    out uint maximumComponentLength,
    out uint fileSystemFlags,

This file has been truncated. show original

readme.md

Metricbeat fails using version 7.11 when GetVolumeInformationW returns false, which it does for floppy and cd-rom volumes.

Example output from this script:

```
PS> get-volumeinformation.ps1
Drives found by GetLogicalDriveStringsW: A:\ C:\ D:\ L:\
GetVolumeInformationW for A:\ returned False
A:\ volume name:
A:\ filesystem name:

This file has been truncated. show original

When the GetVolumeInformationW returns false/error it will get to this line:

github.com

andrewstucki/gosigar/blob/c03570f334158d0daf162e9f300507c9d3ad4cf1/sigar_windows.go#L141


		return errors.Wrap(err, "GetAccessPaths failed")
	}
	for _, drive := range drives {
		dt, err := windows.GetDriveType(drive)
		if err != nil {
			return errors.Wrapf(err, "GetDriveType failed")
		}
		fsType, err := windows.GetFilesystemType(drive)
		if err != nil {
			return errors.Wrapf(err, "GetFilesystemType failed")
		}
		self.List = append(self.List, FileSystem{
			DirName:     drive,
			DevName:     drive,
			TypeName:    dt.String(),
			SysTypeName: fsType,
		})
	}
	return nil

Sommerdal · March 1, 2021, 11:44am

Downgrading to mb 7.10 solved the problem.

Thx

swtrux · March 11, 2021, 8:12pm

I am experiencing the same issue and although you have found the cause there does not seem to be a solution/workaround other than downgrading to 7.10. Is there a bug open on this issue?

anderssonjohan · March 11, 2021, 8:38pm

@swtrux Yes, the original issue was re-opened and a fix was posted yesterday. Please see this comment [Metricbeat] Filesystem Metricset: Inconsistent `system.filesystem.type` values on Windows · Issue #22501 · elastic/beats · GitHub

swtrux · March 11, 2021, 9:03pm

Thanks for the quick response, that is great news. So at this time from what I can tell they have a fix but still have not disclosed what version the fix will be in. Is that true?

swtrux · March 23, 2021, 5:41pm

7.12.0 dropped today and it looks like the fix DID NOT make it in.

Aymeric_Caroff · April 19, 2021, 8:04pm

I'm wondering why the Github issue got closed since it's been confirmed that the error is still present in 7.12.0. I hope we are not dropping the ball here, there's still a bunch of Windows VMs out there .