Okay, so I've ran into this "famous" issue where the index pattern cannot be located. This was a fresh install, although I was experimenting a bit with the index patterns. At one moment some visualizations in the Metricbeat System overview wasn't working (number of hosts and host histogram). In all this, the actual index pattern of course exists, so you are pulling your hairs.
Environment:
Ubuntu 18.04
Elasticsearch 7.4.0, installed using the ansible scripts
Kibana 7.4.0, install from repository
Cause:
I've installed the metricbeat index and dashboard using the --setup option. Everything works
Go to Kibana > management > kibana > index patterns
select the metricbeat-* pattern
delete it
Re-create the same index pattern
Go to the system overview dashboard and you will notice that several visualizations are broken.
Frustrations:
It's an absolute nightmare to figure out which visualizations are broken. There is no way to navigate to the visualization to figure out which one is broken. From a working visualization you can edit and click to go to that specific visualization. The name is missing, the edit visualization is missing. I will write up a reply with a way to figure out which visualization is broken.
Semi-solution
The problem lies in the fact that the old index ID is not there any more. So you need to update the object itself.
Figure out which object is broken
Go to management > kibana > saved objects > search for object
Click on inspect
In the references section you will find "id": "metricbeat-*",
Change this to the new ID of the index pattern, for example "id": "16844e00-0530-11ea-ab45-8b85a0a14acf",
Question/bugs
How can I update the cached value that is referenced in the references section of the object. Somewhere in elastic there is a link stored between the index pattern and the actual UUID. Updating this value should save me a lot of hunting individual objects and changing them one by one.
We need to have a better way of tracking down the objects from within the dashboard. To know that something is broken is fine, however, not having a method to quickly navigate to the broken object is kind of frustrating
now repeat the following steps until you have found the right object
open a new tab in your browser and open any random visualization
Copy the UUID, and paste it into the URL instead of the current UUID in your other tab.
You'll want to find objects with references. Especially those referencing metricbeat-* or any other named index. Change the UUID of the index and you are done. (That is, until we can find another way to change the cached UUID)
Just out of curiosity, why didn't you use the same Custom index pattern ID for your index pattern when you re-created it (like displayed at the image below)?
To be honest I didn't even think about creating an index pattern and using the same ID as it's alias. Primarily because I'm used to having object UUID's, so that's why. Using a wildcard in the alias also seemed incorrect.
At least that solves one of my problems The UI didn't make it really clear that it was searching for the ID "metricbeat-*" instead of the pattern itself, which is listed in the kibana index pattern overview. Now that I know what to look for, it seems clear.
So I guess only one problem remains. If a specific panel fails, then it should be possible to navigate to that specific object and see it's name. Or maybe a way to navigate to the saved object from the Options context menu when editing the dashboard.
Yeah, I agree, we're planning to improve Index Patterns and everything that relates to them for a long time already. There is a lot of confusion around them. If you're curious about the progress you can follow Rethinking index patterns and Proposal to rename Kibana "Index Patterns".
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.
