# All web crawler executions failed

**URL:** <https://discuss.elastic.co/t/all-web-crawler-executions-failed/287738>\
**Category:** Elastic Search\
**Tags:** elastic-app-search\
**Created:** [October 26, 2021, 9:55pm UTC](https://discuss.elastic.co/t/all-web-crawler-executions-failed/287738 "2021-10-26T21:55:41Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![\_leo](https://avatars.discourse-cdn.com/v4/letter/_/87869e/32.png) [@\_leo](https://discuss.elastic.co/u/_leo)\
**Post date:** [October 26, 2021, 9:55pm UTC](https://discuss.elastic.co/t/all-web-crawler-executions-failed/287738/1 "2021-10-26T21:55:41Z")

</div>

I've run the web crawler 8 times and only the first run was successful. The other 7 failed.

I also configured Kibana logs according to the link [1] but there is no explanation in the log as to why the failure. Apparently, the web crawler works and fails abruptly.

How can I find the reason for the failure?

[1] [View web crawler events logs | Elastic App Search Documentation [7.15] | Elastic](https://www.elastic.co/guide/en/app-search/7.15/view-web-crawler-events-logs.html)

---

<div class="post-metadata">

**Author:** ![orhantoy](https://avatars.discourse-cdn.com/v4/letter/o/c5a1d2/32.png) [@orhantoy](https://discuss.elastic.co/u/orhantoy)\
**Post date:** [October 27, 2021, 1:31pm UTC](https://discuss.elastic.co/t/all-web-crawler-executions-failed/287738/2 "2021-10-27T13:31:05Z")

</div>

Do the logs indicate that a crawl gets started?

Also, have you checked `log/crawler.log`? Do you see something there?

---

<div class="post-metadata">

**Author:** ![\_leo](https://avatars.discourse-cdn.com/v4/letter/_/87869e/32.png) [@\_leo](https://discuss.elastic.co/u/_leo)\
**Post date:** [October 27, 2021, 2:39pm UTC](https://discuss.elastic.co/t/all-web-crawler-executions-failed/287738/3 "2021-10-27T14:39:29Z")

</div>

Apparently the crawl starts, as shown the attached image.  
It runs for a few hours, stops and in the "Recent crawl requests" table, it is displayed as failed.

How can I check `log/crawler.log` ? I am using the cloud version ([https://www.elastic.co/](https://www.elastic.co/)).

 ![screen_shot](https://us1.discourse-cdn.com/elastic/original/3X/9/d/9de7b7e4086d210f0526639bea9d3aafc7b8ffc3.png)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 24, 2021, 2:39pm UTC](https://discuss.elastic.co/t/all-web-crawler-executions-failed/287738/4 "2021-11-24T14:39:36Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.