# Filesystem stats not including xfs device

**URL:** <https://discuss.elastic.co/t/filesystem-stats-not-including-xfs-device/176703>\
**Category:** Beats\
**Tags:** metricbeat\
**Created:** [April 12, 2019, 8:21pm UTC](https://discuss.elastic.co/t/filesystem-stats-not-including-xfs-device/176703 "2019-04-12T20:21:18Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Eric\_Herbrandson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eric_herbrandson/32/44069_2.png) [@Eric\_Herbrandson](https://discuss.elastic.co/u/Eric_Herbrandson)\
**Post date:** [April 12, 2019, 8:21pm UTC](https://discuss.elastic.co/t/filesystem-stats-not-including-xfs-device/176703/1 "2019-04-12T20:21:19Z")

</div>

I have 2 devices I expect to see logs for "xvda" and "nvme0n1". The second doesn't appear in kibana. It is a xfs filesystem. I do see one error showing up repeatedly in the metricbeat logs, but it's hard to tell if it's related. The error is...

ERROR elasticsearch/client.go:374 Failed to encode event: unsupported float value: NaN

Output from "lsblk"  
xvda 202:0 0 50G 0 disk  
[redacted]  
nvme0n1 259:0 0 1.7T 0 disk /data

Output from "cat /proc/filesystems"

| nodev | sysfs |
| --- | --- |
| nodev | rootfs |
| nodev | ramfs |
| nodev | bdev |
| nodev | proc |
| nodev | cpuset |
| nodev | cgroup |
| nodev | cgroup2 |
| nodev | tmpfs |
| nodev | devtmpfs |
| nodev | debugfs |
| nodev | tracefs |
| nodev | securityfs |
| nodev | sockfs |
| nodev | bpf |
| nodev | pipefs |
| nodev | hugetlbfs |
| nodev | devpts |
| nodev | autofs |
| nodev | pstore |
| nodev | mqueue |
| nodev | selinuxfs |
| | ext3 |
| | ext2 |
| | ext4 |
| nodev | xenfs |
| | xfs |
| | vfat |
| nodev | overlay |
| nodev | binfmt\_misc |

Output from "cat /etc/fstab"  
/dev/nvme0n1 /data xfs defaults,noatime,nodiratime 0 0

Output from "df -h"  
Filesystem Size Used Avail Use% Mounted on  
[redacted]  
/dev/nvme0n1 1.8T 1.7T 114G 94% /data

What am I doing wrong?

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [April 19, 2019, 8:13pm UTC](https://discuss.elastic.co/t/filesystem-stats-not-including-xfs-device/176703/2 "2019-04-19T20:13:31Z")

</div>

Hi @Eric_Herbrandson and welcome to discuss 🙂

I tried to reproduce this issue with an xfs device but I couldn't, the device was properly reported.

Could you send the output of `stat -f /dev/nvme0n1`?

> [@Eric\_Herbrandson](#):
>
> ERROR elasticsearch/client.go:374 Failed to encode event: unsupported float value: NaN

This error could be caused by other metricsets, could you also try to disable all metricsets except `filesystem` to check if this is the module generating this error?

Thanks!

---

<div class="post-metadata">

**Author:** ![Eric\_Herbrandson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eric_herbrandson/32/44069_2.png) [@Eric\_Herbrandson](https://discuss.elastic.co/u/Eric_Herbrandson)\
**Post date:** [April 22, 2019, 2:01pm UTC](https://discuss.elastic.co/t/filesystem-stats-not-including-xfs-device/176703/3 "2019-04-22T14:01:41Z")

</div>

Thanks so much for your reply @jsoriano! Your response got me thinking and almost certainly this issue is because metricbeat is running in side a docker container. I can't believe I forgot to mention that in my original post ☹

What's the best way to resolve that? Do I just need to make sure some path on that device is mounted into the container?

---

<div class="post-metadata">

**Author:** ![ruflin](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ruflin/32/3116_2.png) [@ruflin](https://discuss.elastic.co/u/ruflin)\
**Post date:** [April 23, 2019, 11:08am UTC](https://discuss.elastic.co/t/filesystem-stats-not-including-xfs-device/176703/4 "2019-04-23T11:08:33Z")

</div>

There are 2 parts here I think. First you need to mount it into the container in case you run metricbeat inside the container.

Second, the error you see should not happen any means we hit some edge cases. Any chance you could provide the output from the command @jsoriano posted above?

---

<div class="post-metadata">

**Author:** ![Eric\_Herbrandson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eric_herbrandson/32/44069_2.png) [@Eric\_Herbrandson](https://discuss.elastic.co/u/Eric_Herbrandson)\
**Post date:** [April 23, 2019, 2:09pm UTC](https://discuss.elastic.co/t/filesystem-stats-not-including-xfs-device/176703/5 "2019-04-23T14:09:12Z")

</div>

Here is the ouput:

File: "/dev/nvme0n1"  
ID: 0 Namelen: 255 Type: tmpfs  
Block size: 4096 Fundamental block size: 4096  
Blocks: Total: 7853870 Free: 7853870 Available: 7853870  
Inodes: Total: 7853870 Free: 7853537

However, all of the commands and output I've listed are from the host, not the container. I assume you want them from the container instead? However, `stat -f /dev/nvme0n1` from inside the container errors with "stat: cannot read file system information for '/dev/nvme0n1': No such file or directory"

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [April 26, 2019, 11:28am UTC](https://discuss.elastic.co/t/filesystem-stats-not-including-xfs-device/176703/6 "2019-04-26T11:28:46Z")

</div>

@Eric_Herbrandson it should be possible to access the devices if you mount `/dev` as a volume in `/hostfs/dev` and you start metricbeat with `--system.hostfs=/hostfs`. If this doesn't work we might need to take a deeper look.

Another option is to use the `--device` docker run flag, for example `--device /dev/nvme0n1` would expose this device in the container.

---

<div class="post-metadata">

**Author:** ![Eric\_Herbrandson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eric_herbrandson/32/44069_2.png) [@Eric\_Herbrandson](https://discuss.elastic.co/u/Eric_Herbrandson)\
**Post date:** [April 26, 2019, 6:16pm UTC](https://discuss.elastic.co/t/filesystem-stats-not-including-xfs-device/176703/7 "2019-04-26T18:16:39Z")

</div>

@jsoriano That did the trick! Though, I did also need to add the `-e` argument before it started working. Thanks so much for the help!

I am still getting the error "elasticsearch/client.go:374 Failed to encode event: unsupported float value: NaN". Would you like to try to track that down? If so, would you like to continue discussing in this thread or would you prefer we start a separate thread for that?

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [April 29, 2019, 9:52am UTC](https://discuss.elastic.co/t/filesystem-stats-not-including-xfs-device/176703/8 "2019-04-29T09:52:54Z")

</div>

Good to read that it is working for you now! For the record, what did the trick? Mounting `/dev` under `/hostfs`, or the `--device` flag?

The `-e` argument only changes the logs configuration so the are written to stderr. This shouldn't affect any other thing.

> [@Eric\_Herbrandson](#):
>
> I am still getting the error "elasticsearch/client.go:374 Failed to encode event: unsupported float value: NaN". Would you like to try to track that down? If so, would you like to continue discussing in this thread or would you prefer we start a separate thread for that?

Are you still missing events? To identify what metricset is creating the incorrect events it'd be good if you could try to enable them one by one to try to isolate the problem.

And what version of metricbeat are you using?

---

<div class="post-metadata">

**Author:** ![Eric\_Herbrandson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eric_herbrandson/32/44069_2.png) [@Eric\_Herbrandson](https://discuss.elastic.co/u/Eric_Herbrandson)\
**Post date:** [April 29, 2019, 3:25pm UTC](https://discuss.elastic.co/t/filesystem-stats-not-including-xfs-device/176703/9 "2019-04-29T15:25:54Z")

</div>

Mounting `/dev` did it.

I don't know that we're missing events, but I guess it could be possible. I've narrowed it down to the `diskio` metricset. When I remove that, the errors go away.

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [April 29, 2019, 3:55pm UTC](https://discuss.elastic.co/t/filesystem-stats-not-including-xfs-device/176703/10 "2019-04-29T15:55:26Z")

</div>

A fix for the diskio metricset was recently merged: [https://github.com/elastic/beats/pull/11676](https://github.com/elastic/beats/pull/11676)

This will be available in 6.7.2 and 7.0.1.

---

<div class="post-metadata">

**Author:** ![Eric\_Herbrandson](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/eric_herbrandson/32/44069_2.png) [@Eric\_Herbrandson](https://discuss.elastic.co/u/Eric_Herbrandson)\
**Post date:** [April 29, 2019, 4:52pm UTC](https://discuss.elastic.co/t/filesystem-stats-not-including-xfs-device/176703/11 "2019-04-29T16:52:56Z")

</div>

Great. Thank you so much for all the help!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 27, 2019, 4:52pm UTC](https://discuss.elastic.co/t/filesystem-stats-not-including-xfs-device/176703/12 "2019-05-27T16:52:59Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
