# Web Crawler API endpoints for URLs and crawl rules

**URL:** <https://discuss.elastic.co/t/web-crawler-api-endpoints-for-urls-and-crawl-rules/269598>\
**Category:** Elastic Search\
**Tags:** elastic-app-search\
**Created:** [April 8, 2021, 1:16pm UTC](https://discuss.elastic.co/t/web-crawler-api-endpoints-for-urls-and-crawl-rules/269598 "2021-04-08T13:16:56Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Yuriy\_Stakhniak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuriy_stakhniak/32/79051_2.png) [@Yuriy\_Stakhniak](https://discuss.elastic.co/u/Yuriy_Stakhniak)\
**Post date:** [April 8, 2021, 1:16pm UTC](https://discuss.elastic.co/t/web-crawler-api-endpoints-for-urls-and-crawl-rules/269598/1 "2021-04-08T13:16:56Z")

</div>

Since [this](https://discuss.elastic.co/t/add-site-to-crawler-via-api/264257) topic is already closed, I was forced to create a new one.  
Please tell me if it is possible now to programmatically set URLs and crawl rules into the web crawler. This is not currently described in the API documentation.  
Thank you in advance.

---

<div class="post-metadata">

**Author:** ![ross.bell](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ross.bell/32/77559_2.png) [@ross.bell](https://discuss.elastic.co/u/ross.bell)\
**Post date:** [April 8, 2021, 3:23pm UTC](https://discuss.elastic.co/t/web-crawler-api-endpoints-for-urls-and-crawl-rules/269598/2 "2021-04-08T15:23:19Z")

</div>

Hey Yuriy!

API functionality to modify crawl urls/rules is coming soon, either with or before Crawler goes GA. For now, the unofficial workaround is as alluded to in the post you linked to, with the stated caution about it not being officially supported.

Ross

---

<div class="post-metadata">

**Author:** ![Yuriy\_Stakhniak](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yuriy_stakhniak/32/79051_2.png) [@Yuriy\_Stakhniak](https://discuss.elastic.co/u/Yuriy_Stakhniak)\
**Post date:** [April 9, 2021, 8:48am UTC](https://discuss.elastic.co/t/web-crawler-api-endpoints-for-urls-and-crawl-rules/269598/3 "2021-04-09T08:48:15Z")

</div>

Thanks a lot, Ross.  
Whether it is known when GA date is expected?

---

<div class="post-metadata">

**Author:** ![ross.bell](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ross.bell/32/77559_2.png) [@ross.bell](https://discuss.elastic.co/u/ross.bell)\
**Post date:** [April 9, 2021, 1:12pm UTC](https://discuss.elastic.co/t/web-crawler-api-endpoints-for-urls-and-crawl-rules/269598/4 "2021-04-09T13:12:18Z")

</div>

Hey Yuriy,

There's no official date known yet, but I'll probably be looking forward to crawling some of my own personal projects this way in the next few months. I would keep an eye on Enterprise Search release notes.

Ross

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 7, 2021, 1:13pm UTC](https://discuss.elastic.co/t/web-crawler-api-endpoints-for-urls-and-crawl-rules/269598/5 "2021-05-07T13:13:16Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.