Saved "field" parameter is now invalid. Please select a new field. .... Visualize: "field" is a required parameter

Hi Stacey_Gammon -

Could you please help me in resolving this issue. Thanks !

Regards,
Prakash

Sorry about that, this slipped off my radar.

What do you get if you go into dev tools and run the following command:

POST met*/_search
{
  "size": 0,
  "aggs": {
    "1": {
      "avg": {
        "field": "system.load.norm.1"
      }
    },
    "2": {
      "avg": {
        "field": "system.load.norm.5"
      }
    },
    "3": {
      "avg": {
        "field": "system.load.norm.15"
      }
    }
  },
  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "query": "metricset.module: system AND metricset.name: load",
            "analyze_wildcard": true
          }
        }
      ]
    }
  }
}

Can you try this with the System Load visualization? I'm hoping it will confirm the invalid field(s).

Hi-
I ran the post request and here is the response below:

{
"took": 54,
"timed_out": false,
"_shards": {
"total": 40,
"successful": 40,
"failed": 0
},
"hits": {
"total": 0,
"max_score": 0,
"hits": []
},
"aggregations": {
"1": {
"value": null
},
"2": {
"value": null
},
"3": {
"value": null
}
}
}

Here is the System Load Visulization snapshot:
Thanks for your help !!

Can you set the fields in the left hand data to be system.load.norm.1, system.load.norm.5 and system.load.norm.15?

I took a second look at your field list and I believe the issue is that some of your fields aren't marked as searchable and aggregatable like they are in mine. Hence the first error message is that the original field choices were invalid, and the second error message is letting you know that that is a required field.

You used the import_dashboards script from metricbeat 5.1.1 right?

Can you run the following command in Dev Tools, then do a search for system.load.norm in the results and paste the outcome (see the screenshot).

POST .kibana/_search/
{
  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "analyze_wildcard": true,
            "query": "_type: index-pattern AND system.load.norm.15"
          }
        }
      ]
    }
  }
}

You can see how the system.load.norm fields in my results have searchable and aggregatable set to true.

Can you also run

GET metricbeat*/_mapping/metricsets/field/system.load.norm.15

and

GET /_field_stats?fields=system.load.norm.15

in dev tools and send me the output?

Refreshing the field list in management (the yellow button) should correct those fields to be marked as searchable and aggregatable, unless their is an issue with no data existing for those fields.

[quote="Stacey_Gammon, pos

t:15, topic:70034"]
system.load.norm.15
[/quote]

Hi -
Yes, Am using metricbeat-5.1.1 and I've run the query in the dev tools and attached the screenshot, I believe the values are set to be false on my set-up.

Hi -
Yep, I had executed those two requests and attached the output here:

GET metricbeat*/_mapping/metricsets/field/system.load.norm.15

{
"metricbeat-2016.12.29": {
"mappings": {
"metricsets": {
"system.load.norm.15": {
"full_name": "system.load.norm.15",
"mapping": {
"15": {
"type": "scaled_float",
"scaling_factor": 100
}
}
}
}
}
},
"metricbeat-2017.01.02": {
"mappings": {
"metricsets": {
"system.load.norm.15": {
"full_name": "system.load.norm.15",
"mapping": {
"15": {
"type": "scaled_float",
"scaling_factor": 100
}
}
}
}
}
},
"metricbeat-2017.01.03": {
"mappings": {
"metricsets": {
"system.load.norm.15": {
"full_name": "system.load.norm.15",
"mapping": {
"15": {
"type": "scaled_float",
"scaling_factor": 100
}
}
}
}
}
},
"metricbeat-2017.01.04": {
"mappings": {
"metricsets": {
"system.load.norm.15": {
"full_name": "system.load.norm.15",
"mapping": {
"15": {
"type": "scaled_float",
"scaling_factor": 100
}
}
}
}
}
},
"metricbeat-2016.12.31": {
"mappings": {
"metricsets": {
"system.load.norm.15": {
"full_name": "system.load.norm.15",
"mapping": {
"15": {
"type": "scaled_float",
"scaling_factor": 100
}
}
}
}
}
},
"metricbeat-2017.01.05": {
"mappings": {
"metricsets": {
"system.load.norm.15": {
"full_name": "system.load.norm.15",
"mapping": {
"15": {
"type": "scaled_float",
"scaling_factor": 100
}
}
}
}
}
},
"metricbeat": {
"mappings": {}
},
"metricbeat-2016.12.26": {
"mappings": {
"metricsets": {
"system.load.norm.15": {
"full_name": "system.load.norm.15",
"mapping": {
"15": {
"type": "scaled_float",
"scaling_factor": 100
}
}
}
}
}
},
"metricbeat-2016.12.27": {
"mappings": {
"metricsets": {
"system.load.norm.15": {
"full_name": "system.load.norm.15",
"mapping": {
"15": {
"type": "scaled_float",
"scaling_factor": 100
}
}
}
}
}
},
"metricbeat-2016.12.28": {
"mappings": {
"metricsets": {
"system.load.norm.15": {
"full_name": "system.load.norm.15",
"mapping": {
"15": {
"type": "scaled_float",
"scaling_factor": 100
}
}
}
}
}
}
}

GET /_field_stats?fields=system.load.norm.15

{
"_shards": {
"total": 144,
"successful": 144,
"failed": 0
},
"indices": {
"_all": {
"fields": {}
}
}
}

And also I had filtered with system.load in Index patterns (refreshed the field list) and I don't think those fields are changed to Aggregatable. And I've attached the screenshot here.

Ah, so I believe the problem is that there is no data in your index for those fields. When there is no data for that field, the field_stats api won't return anything, which means Kibana will mark it as not search/aggregatabable/etc and the visualization will throw an error.

I'm going to guess if you run this query:

GET metric*/metricsets/_search?q=system.load.norm.15:>0

you would not get any hits. Can you confirm this?

You might be able to manually insert a single data point for that field so the visualization doesn't throw an error, but that's pretty pointless if you are actually interested in those stats. The real question is why metricbeat isn't sending data for that field.

1 Like

Here are some relevant git issues about the reason the visualizations are failing when there is no data for the field:


1 Like

Okay, I may have a solution to fix your dashboards. Try re-running the metricbeat import script, and this time do not refresh the field list. This should bypass the field_stats api and hopefully preserve the correct information for the field.

1 Like

Hi @Stacey_Gammon-
Yep, you are right.
It doesn't have any hits when I run the query, Here is the response:

{
"took": 77,
"timed_out": false,
"_shards": {
"total": 55,
"successful": 55,
"failed": 0
},
"hits": {
"total": 0,
"max_score": null,
"hits": []
}
}

Hi @Stacey_Gammon-
I ran the import script and didn't refresh the fields, I DON'T see any error message on the dashboard now.
But, I believe, Metric beat is sending empty data, so no results found message is displayed.

Hi Stacey

I am running metricbeat and my LOAD/CPU visualisation has this error too, I need to find out why metricbeat is not reporting this information

Any help appreciated?

My stack is all 5.1.1 - metricbeat is running on all nodes, here is the main part of the config

metricbeat.modules:

  • module: system
    metricsets:
    • cpu
    • filesystem
    • memory
    • network
    • process
      enabled: true
      period: 300s
      processes: ['.*']
      cpu_ticks: false

Maybe I don't have everything I need configured

Thanks

Actually this isn't an issue any more for me, fixed with changes to the metricbeat config

Hi @matt.dunks-
What was the changes in the Config file you made? Please let me know. thanks !

It was simply a case of re-reading the full config and then enabling more options

metricbeat.modules:

  • module: system
    metricsets:
    • cpu
    • filesystem
    • memory
    • network
    • process
    • load
    • core
    • diskio
      enabled: true
      period: 300s
      processes: ['.*']
      cpu_ticks: true

There are a couple more options in the full config which I haven't used
When the new index started the data for the other graphs populated and now all graphs work with no errors

Hope that helps you

I've got to know that, still Beats is not supported on Windows for Load Stats, so I could't get the data on the dashboard. Am good with that. Thanks @Stacey_Gammon.

I had the same issue, spend a lot of time to get it work. At the end, it looks like that 'refresh the field list' doesn't work
with chromium but works with firefox. Some dashboards have issues with chromium as well. Please find the versions below:

chromium 56.0.2924.87
firefox 51.0.1

Regards,
Daniel

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.