# Metricbeat can't make visualization due to null value in index fields value **URL:** \ **Category:** Metrics\ **Created:** [April 22, 2022, 8:35am UTC](https://discuss.elastic.co/t/metricbeat-cant-make-visualization-due-to-null-value-in-index-fields-value/302996 "2022-04-22T08:35:36Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![PRASHANT\_MEHTA](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prashant_mehta/32/101764_2.png) [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Post date:** [April 22, 2022, 8:35am UTC](https://discuss.elastic.co/t/metricbeat-cant-make-visualization-due-to-null-value-in-index-fields-value/302996/1 "2022-04-22T08:35:36Z")
Hello All, I am unable to create TSVB visulization TOP N in kibana,as I'm trying to display system.filesystem.used.pct of mount point but both these fiels have null values(-) in index. Any suggestion would be helpful! ![image](https://us1.discourse-cdn.com/elastic/original/3X/7/5/75a5fa899063571cf493701e69bc3c082b8495be.png) ![image](https://us1.discourse-cdn.com/elastic/original/3X/e/1/e1ab957b4d99a152fc52bd3986058e62ce98faf1.png) Thanx, Prashant ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [April 22, 2022, 7:57pm UTC](https://discuss.elastic.co/t/metricbeat-cant-make-visualization-due-to-null-value-in-index-fields-value/302996/2 "2022-04-22T19:57:19Z")
Hi @PRASHANT_MEHTA Couple items... I am a little confused. What version are you on? What are you actually try to show? That is a Table View not Top N I would put the following in the panel filter so only file system documents show up. `event.dataset : "system.filesystem"` ---
**Author:** ![PRASHANT\_MEHTA](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prashant_mehta/32/101764_2.png) [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Post date:** [April 25, 2022, 10:22am UTC](https://discuss.elastic.co/t/metricbeat-cant-make-visualization-due-to-null-value-in-index-fields-value/302996/3 "2022-04-25T10:22:46Z")
Hello @stephenb , Thanks for looking into this! I'm using kibana,metricbeat,elastic 7.9.1 version.I'm trying to implement a use case through TSVM(option TOP N).Which would display only those mount points which have reached 90% utilization and dont require below 90%. So I'm implementing this using TOP N ,but challenge I face is that I need to display hostName also.Currently only filesystem used % and filesystem name can be seen,but I need the hostname as well.This cant be achived in kibana directly I guess,so I'm trying to append the hostname with mountpoint name through javascript processor,but host name as an event is not picked ,I'm not sure why. Even sometimes visulization shows wrong data like 0% and green bar.Like below,Ideally through panel filter I will filter system.filesystem.used.pct \> 0.9.....But this cant be seen at time and at time it does....Seems bug. ## SEEMS BUG ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/6/96ee7b4875f2bcce55f0729a34f08abe8730d9ce.png) ## ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/f/0f8ccbd58d255aaa95524d73f09fc674b98866dc.png) Aggregation: ![image](https://us1.discourse-cdn.com/elastic/original/3X/0/0/00ab7467e22b5d529a3ac655b6b940c459a5b9af.png) Correct Requirement:[mountpoint hostname---------------------filesystem used % value] Below Image- only red bar should be shown as alert which can be achived through panel filter \> 0.9(90%),as I cant show three values in visulization so I am trying to create new field that would concatenate(mountpoint- hostname---------------------------------file.system.used.pct) ![image](https://us1.discourse-cdn.com/elastic/optimized/3X/c/2/c25dd5388be51bfc1e9e6e22664f91f1004e1995_2_690x30.png) Can u assist how can this be best achived and why JS processor in metricbeat dosent't pick up hostname event? ```auto var console = require('console'); function process(event) { var fileSystemUsedPercent = event.Get("system.filesystem.used.pct");//filesystem used percentage console.info("File system used percentage value [%s]", fileSystemUsedPercent); var mountPoint = event.Get("system.filesystem.mount_point"); //mountpoint name console.info("mountPoint [%s]", mountPoint); var hostName = event.Get(host.name); //hostname [DOSENT PICK UP HOST EVENT THOUGH PRESENT IN INDEX] console.info("hostName [%s]", hostName); if(fileSystemUsedPercent > 0.8) { event.Put("filesystem.status","critical"); event.Put("filesystem.mount.host",mountPoint + ' ' + hostName);//display hostname and mountpoint if greater than treshold console.info("filesystem.status [%s]", event.Get("filesystem.status")); } return event; } ``` Many Thanks, Prashant ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [April 25, 2022, 4:58pm UTC](https://discuss.elastic.co/t/metricbeat-cant-make-visualization-due-to-null-value-in-index-fields-value/302996/4 "2022-04-25T16:58:44Z")
Hi @PRASHANT_MEHTA Lets back up a bit. 1st 7.9 is ancient / EOL and I had to set up a 7.9.3 stack because so much has changed.... if you were on a recent release there would be some pretty simple ways to solve this. I think what you want to do is "As an ops user I was display the Filesystems with the Highest `system.filesystem.used.pct` and show them sorted by max and display both host.name and `system.filesystem.mount_point`" Ok let/s work on that. 1st The Top Hits aggregation you are trying to use is absolutely the **wrong** aggregation... Top Hits means "Most Common" not Max Value so that will never work your aggregation is asking for the average of the most common value.. and that is why the graph looks wrong . So First lets see if we can get the Top N working first (without the hostname) just for learning purposes.. ![Screen Shot 2022-04-25 at 9.51.48 AM](https://us1.discourse-cdn.com/elastic/original/3X/f/7/f77673df0c5eb0ec95e45c83587eb8b337f798cf.png) Panel Options, Says use the last value (but drop the last partial bucket) Make sure the bucket size / interval \>=1min (the collection rate) and filter on the data set. ![Screen Shot 2022-04-25 at 9.52.17 AM](https://us1.discourse-cdn.com/elastic/original/3X/4/b/4b78a9808c0f3184180488384d39a13abea66df2.png) Now find the Average of the Filesystem % and sort them Descending ![Screen Shot 2022-04-25 at 9.52.48 AM](https://us1.discourse-cdn.com/elastic/original/3X/1/3/137550a947537a080c602969bb3c869fb8094a69.png) And set to Percent ![Screen Shot 2022-04-25 at 9.52.56 AM](https://us1.discourse-cdn.com/elastic/original/3X/6/9/692379db032b46824272e3bd5923c62bf18ba4b4.png) That is the correct way to build that... Try that and make sure it works, come back when you have it working ... then perhaps we can figure out the concatenate the host + mount point. I am not familiar with the Java Script processor. (again in a new stack this is a 5 min fix) ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [April 25, 2022, 5:23pm UTC](https://discuss.elastic.co/t/metricbeat-cant-make-visualization-due-to-null-value-in-index-fields-value/302996/5 "2022-04-25T17:23:32Z")
in 7.9.3. You can get what you want by using a data table You need to hit "update" after you make each change ![Screen Shot 2022-04-25 at 10.18.14 AM](https://us1.discourse-cdn.com/elastic/original/3X/9/4/941242827cb53350ea7f39c8c16b28b6950e52c9.png) Select metricbeat-\* ![Screen Shot 2022-04-25 at 10.18.30 AM](https://us1.discourse-cdn.com/elastic/original/3X/d/2/d2addd5116c201eece1510528b78422ffb89e570.png) Select your Metric ![Screen Shot 2022-04-25 at 10.18.41 AM](https://us1.discourse-cdn.com/elastic/original/3X/9/6/964869c38ed940195dafe5678941236438f1db57.png) Split Rows By Host ![Screen Shot 2022-04-25 at 10.18.52 AM](https://us1.discourse-cdn.com/elastic/original/3X/8/3/8305fae1697e5d36eac9e5aa35b8c835ce22097e.png) The mount point ![Screen Shot 2022-04-25 at 10.18.58 AM](https://us1.discourse-cdn.com/elastic/original/3X/1/e/1e2aa40af85f51125f6e9af4cf315d17c9967958.png) And you should get this ![Screen Shot 2022-04-25 at 10.19.02 AM](https://us1.discourse-cdn.com/elastic/original/3X/9/8/9804c3146ea35c9f6dfe5fd1b9a83c11307532ec.png) You can use custom labels to make it look nicer and sorry my data is boring. ---
**Author:** ![PRASHANT\_MEHTA](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prashant_mehta/32/101764_2.png) [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Post date:** [May 4, 2022, 3:31pm UTC](https://discuss.elastic.co/t/metricbeat-cant-make-visualization-due-to-null-value-in-index-fields-value/302996/6 "2022-05-04T15:31:05Z")
@stephenb Thanx for helping in this! It worked fine.Can you please suggest ELK version above 7.9.1 i.e not log4j vulnerable and non-paid/open source version that can be used? Many Thanks, Prashant ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [May 4, 2022, 4:03pm UTC](https://discuss.elastic.co/t/metricbeat-cant-make-visualization-due-to-null-value-in-index-fields-value/302996/7 "2022-05-04T16:03:48Z")
I would use 7.17.3 or 8.2 with the Elastic Basic License which is Free You can see the features [here](https://www.elastic.co/subscriptions) under the FREE AND OPEN - BASIC Heading Once you are on a newer version I would highly recommend looking at / using Lens @PRASHANT_MEHTA Ohh the new version would solve your other issues of concatenating the host + filesystem... 5 minute task with new runtime fields... ---
**Author:** ![PRASHANT\_MEHTA](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prashant_mehta/32/101764_2.png) [@PRASHANT\_MEHTA](https://discuss.elastic.co/u/PRASHANT_MEHTA)\ **Post date:** [May 13, 2022, 4:47pm UTC](https://discuss.elastic.co/t/metricbeat-cant-make-visualization-due-to-null-value-in-index-fields-value/302996/8 "2022-05-13T16:47:57Z")
Hello All, Probable solution found and here is the code and system.yml for reference. ```auto /** The ProcessDataUpdater implements to combine the host name, process name and process id data in a separate filed. */ var console = require('console'); var params = {hostname: ""}; /** Gets the script parameter passed from the system.yml file. With this context, host name is passed as input. */ function register(scriptParams) { params = scriptParams; } /** Entry point to run the logic. It gets the host name, process name and process id form the current event and creates a new field with combining the those values. Also, it returns the updated event. */ function process(event) { var hostName = params.hostname; var processId = event.Get("process.pid"); var processName = event.Get("process.name"); event.Put("process.details", hostName + "_" + processName + "_" + processId) var processDetails = event.Get("process.details"); return event; } ``` system.yml ```auto # Module: system # Docs: https://www.elastic.co/guide/en/beats/metricbeat/7.8/metricbeat-module-system.html - module: system period: 10s metricsets: - load - memory - network - module: system period: 60s metricsets: - fsstat - filesystem filesystem.ignore_types: [ssysfs, rootfs, ramfs, bdev, cgroup, cpuset,debugfs, securityfs, sockfs, dax, bpf, pipefs, anon_inodefs, configfs, devpts, hugetlbfs, autofs, pstore, mqueue, selinuxfs, rpc_pipefs, binfmt_misc, overlay,nfs4,nfs,proc,sysfs] - module: system period: 10s metricsets: - diskio diskio.include_devices: ["sda", "sda1","vda","vda1"] - module: system metricsets: - process processes: ['.*'] process.include_cpu_ticks: true processors: - script: lang: javascript id: ProcessDataUpdater file: ProcessDataUpdater.js params: hostname: '${HOSTNAME}' - module: system period: 10s metricsets: - cpu processors: - script: lang: javascript id: CPUStatusUpdater.js file: CPUStatusUpdater.js ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 10, 2022, 4:48pm UTC](https://discuss.elastic.co/t/metricbeat-cant-make-visualization-due-to-null-value-in-index-fields-value/302996/9 "2022-06-10T16:48:24Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.