Troubleshooting / cleaning up pending reports

Elastic Stack Kibana
1

I've been testing with Report generation, both manual and automatic through Watcher. Things were working initially but 3-4 days ago reports started failing.
Now none of the reports seem to work anymore, both manual and automatic. I see they all go into a "pending" state and they never seem to actually timeout or complete or error.

Is there any log or API I can use to troubleshoot this further?
Any report index I can cleanup perhaps to get reports working again?
I'm only reporting 1 Dashboard that still exists and does work fine when viewed through Kibana.

The two Watches I was using for automatic report generation show "Aborting due to maximum number of retries hit [40]".

any suggestions?

thanks,

H

2

You didn't say what version of Kibana you're on, but in the most recent 6.x versions there are 3 indices used for watches;

image

I think .triggered_watches is temporary. I don't have any docs in mine and I have had watches execute.

You can create an index pattern in Kibana for .watches* (slide the toggle to show system indices). Then you can go to Discover and see all your watches. This might be handy if you need to find the id of a particular watch to delete.

.watcher-history* has about 150 fields and is useful for seeing what happened when your watch triggered (or if it did).

There are 6 built-in watches for cluster status and things like that so I wouldn't wipe out the .watches index. You could delete some of the watches you created if they are causing a problem.

See if any of that helps. If you're still having problems reply back and include the version of Kibana you're on.

Thanks,
Lee

3

thank you Lee. I'm using 6.4.1, sorry for not having said that before.
I think I'm all set now, I'll use the information you provided but my immediate issue was also combined with the fact I was sending reports every minute, and they were "queuing up".

After disabling the watch it took an hour or so to eventually generate all the reports. All watches failed as they'd timed out, but then I had no more pending reports and new reports would complete on time (and so new watches started working).

thanks a lot for the information.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.