We upgraded to 5.4.1 last night from 2.X. Most everything is working, but a few of our visualizations are not. They are receiving the error message "Saved "field" parameter is now invalid. Please select a new field."
I looked through a few of the other threads, however my issue looks to be slightly different.
The common symptom in those other threads is that there is no data in the particular field, so when Kibana runs the field_stats command it doesn't know how to handle it. Once you have data in the index, refreshing the index pattern fixes the issue. However in my case there is data and refreshing the index doesn't solve anything.
Here is what the field looks like in Kibana. Notice that the type is a number, but searchable and aggregatable is disabled. This stops me from be able to take averages of this field in Visualizations.
And I got very interesting results. The indices are daily.
My index is actually an alias for two separate indices, each with very similar data. However one index has the duration as a double, the other has it as a long. This isn't noticeable in Kibana since it just displays everything as a number.
This index alias and pattern has been around for a while, so I assume that 2.X didn't care about the slightly different data types.
Just for the fun of it I tried running the deprecated field_stats command. So field_stats says I have a conflict between an integer and a float. But field_caps says the conflict is between a double and a float.
Digging through the actual mappings confirms that one is a double while another is a float. I'm not sure where field_stats got Integer from.
If I take the query that Kibana wants to run and run it through DevTools everything works fine.
So Elasticsearch itself doesn't have a problem taking an average when the field is a long and a double, but Kibana doesn't seem to know what to do with it.
Has anyone come across anything similar? Have you found any work arounds besides re-indexing with an updated template? I have around 1 billion documents this has to go through so I would like to avoid reindexing if I can.