Kibana still fails to locate filebeat-* even if it exists

Elastic Stack Kibana
1

Deleting all Filebeat dashboards, visualizations, indices and index patterns - then rerunning 'filebeat setup -e', setup loads the dashboards, I can see them newly added to kibana, but then it still fails to find the filebeat-* index, or it errors on "Could not locate that index-pattern-field (id: source.geo.location)", or errors with "Failed to find the filebeat-* index pattern. Click here to recreate" - though it exists. And all visualization fields have error exclamation marks. This has been a bug for a while in Kibana as I've hit this on a few previous versions too.

I deleted the Filebeat dashboards the first time by mistake after I saw "1 of 4 shards failed" when I added a new apache module to a node running filebeat. I figured I'd just delete them and recreate them, but they seem to look for some old index pattern id that does not exist.

2

Hello @rgrg

It sounds like you're trying to wipe out any kibana content. Is that correct? If so, you could try deleting the .kibana index in elasticsearch.

I deleted the Filebeat dashboards the first time by mistake after I saw "1 of 4 shards failed"

Have you verified that your elasticsearch cluster is healthy?

What version of the elasticstack are you running?

3

Thanks for the reply -

My cluster health seems fine and I am receiving new data all the time on my normal indices -

"status":"green","timed_out":false,"number_of_nodes":3,"number_of_data_nodes":3,"active_primary_shards":149,"active_shards":298,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":100.0}

I don't want to wipe out all .kibana contents - I just want to bring in the Filebeat visualizations and dashboards fresh so they work again. They were working, after I enabled filebeat on one of my nodes, then I saw "1 of 4 shards failed" in the apache filebeat dashboard, and just about all of the graphs stopped loading and showed the ! error on them.

ES version is 7.3.1
CentOS 7.6.1810.
Filebeat: 7.3.1

4

Can you provide kibana log info or screenshots of the errors?

Thanks,
Matt

5

Hi Matt,

I've just had the same problem with the metricbeat index. I believe is has to something with a cached UUID which kibana cannot find. Please see my post about metricbeat for a partial solution.

Regards,
Patrick

6

@pdeelman

Thanks for your detailed post.

I'm in the planning stages to address the problem you've described. I'm hoping to find a way to prevent the problem and make it easier to resolve. I can reference your post when describing the problem to my coworkers.

7

I've ran into what I think is a similar problem a couple of times. The problem appears to be that if you import the dashboards before there is is an index, all visualizations fail.

What usually works is deleting the beats index and all visualizations etc., then start your beat, wait for Elastic to show an index and then load the dashboards, it will work. But if you follow the steps in the Docs it will always tell you to load up the dashboards before starting beats.

In general it would be great if visualizations could update the index ID so you don't have to recreate a visualization or edit the index ID if for whatever reason you have to delete all indices/index pattern.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.