Ghost Web Crawlers

[mszal_ib (Michael)]

I am using Elastic, Kibana, and Enterprise Search 8.14 on Elastic Cloud. I was test crawling some domains with the "Elastic Web Crawler" with multiple custom crawl schedules (top right drop down Kibana). Later, I deleted these crawlers, but 3 new crawlers have appeared in the dashboard with the index name "--" which cannot be inspected or deleted as I cannot enter the name. They also have "Ingestion Status" set to incomplete. These are the only crawlers on the dashboard. Also, it lists I have 4 "running crawlers" listed on the dashboard. I cannot find these indices either.

Is there a method for me to stop/delete these indices/crawlers?

[Jedr_Blaszyk]

Hey @mszal_ib! I confirm I could reproduce an issue in 8.14 and it seems that we have a bug here. We will address it.

For now you can use following steps to clean up you crawler overview page

Step 1: list your crawlers

GET _connector?service_type=elastic-crawler

Take the id from the response use to delete crawlers without index names:

DELETE _connector/{id}

Let me know if it fixes your issue.

[mszal_ib (Michael)]

@Jedr_Blaszyk

DELETE _connector/{id}

I can confirm this was able to delete the crawlers & active crawlers. However, now I have many orphan crawls. Will these eventually clean up, or is there another method to delete these?

[mszal_ib (Michael)]

I was able to solve the orphaned crawler's issue by finding the sync jobs

GET _connector/_sync_job

and deleting them with

DELETE _connector/_sync_job/{id}

Thank you for the help @Jedr_Blaszyk.