For some reason this call below will return all the index patterns but only some of them have any fields in them even though the index has fields. What's really interesting is that if I do anything with these indices (like look at them from the KIbana UI management page) then the next request will return this index pattern with its fields. What is going on here?
Did some more research and it looks like some code in Kibana assumes these fields are going to be in the saved object while others have a comment that say it's an invalid index.
Here is some places where the code assumes that the saved object will have the fields populated
IndexPatternSelect assumes fields not in the saved object must be an invalid index.
However it's not an invalid index pattern. It's just that the saved objects client is not returning all the data. It seems like the index patterns don't have fields in them until someone calls getFieldsForWildcard which does seem like what happens when you look at the index in the index management page.
So for instance I go to use Timelion and it doesn't work for an index. I hit the index pattern management page look at the index pattern and see that it has fields. Go back to Timelion and it now works. This happens because the saved object now has fields in it because going to the index pattern management page and clicking the index pattern calls that getFieldsForWildcard which then populates the saved object with its fields.
What I don't understand is why is some Kibana code assuming the fields are going to exist and others think that if they don't then the index pattern is invalid. Again it's not that the index pattern is invalid but that the saved object is not populated correctly. I also really don't understand why the index pattern saved object is not ALWAYS populated with the fields. Would really appreciate the Kibana team to look into this. I don't want to have to keep telling my users, "Oh go look at the index pattern from the management page and then go back to what you were trying to do. It will work then."
The saved objects API is returning the data as persisted. When adding an index pattern through the UI, clicking refresh on the index pattern page, or as you discovered any other time getFieldsForWildcard is called - the fields property is populated. We have for a while wanted to remove this field cache kibana#6498 entirely and recently discussed changing how index patterns work kibana#35481.
Are you programmatically creating these index patterns using the saved object API?