# API Endpoint to initiate crawl of single page

**URL:** https://discuss.elastic.co/t/api-endpoint-to-initiate-crawl-of-single-page/276296
**Category:** Elastic Search
**Tags:** elastic-app-search
**Created:** [June 17, 2021, 5:17pm UTC](https://discuss.elastic.co/t/api-endpoint-to-initiate-crawl-of-single-page/276296 "2021-06-17T17:17:36Z")
**Posts on this page:** 6
**Page:** 1

<div class="post-metadata">

### Author: ![tchocky](https://avatars.discourse-cdn.com/v4/letter/t/b77776/32.png) [@tchocky](https://discuss.elastic.co/u/tchocky)
#### Post date: [June 17, 2021, 5:17pm UTC](https://discuss.elastic.co/t/api-endpoint-to-initiate-crawl-of-single-page/276296/1 "2021-06-17T17:17:36Z")

</div>

Hi all -

I am looking for a way to initiate the crawl of a single page on demand via an API call. While looking through the docs I was lead to [this page on the Swiftype site](https://swiftype.com/documentation/site-search/api-crawler-operations#crawl-url). This is exactly what I am looking for but it does not seem to work on App Search version 7.13.1.

The endpoint I am trying to call is:

`http://localhost:3002/api/as/v1/engines/[engine_name]/domains/[domain_id]/crawl_url.json`

I get the error :

`{"errors": "Routing Error. The path you have requested is invalid."}`.

Is there an Elastic specific version of this API endpoint?

Thanks as always.

---

<div class="post-metadata">

### Author: ![constancecchen](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/constancecchen/32/71762_2.png) [@constancecchen](https://discuss.elastic.co/u/constancecchen)
#### Post date: [June 17, 2021, 7:03pm UTC](https://discuss.elastic.co/t/api-endpoint-to-initiate-crawl-of-single-page/276296/2 "2021-06-17T19:03:48Z")

</div>

Hey @tchocky! Always super exciting to see folks using our beta crawler. It looks like the API URL is a little different in Elastic vs. Swiftype, but you should still be able to do what you want using the `/api/as/v0/engines/[engine_name]/crawler/crawl_requests` API endpoint:

> **[Web crawler (beta) API reference | Elastic App Search Documentation \[7.13\] |...](https://www.elastic.co/guide/en/app-search/7.13/web-crawler-api-reference.html#web-crawler-apis-post-crawler-crawl-requests)**

Hope that helps!  
Constance

---

<div class="post-metadata">

### Author: ![tchocky](https://avatars.discourse-cdn.com/v4/letter/t/b77776/32.png) [@tchocky](https://discuss.elastic.co/u/tchocky)
#### Post date: [June 17, 2021, 7:07pm UTC](https://discuss.elastic.co/t/api-endpoint-to-initiate-crawl-of-single-page/276296/3 "2021-06-17T19:07:46Z")

</div>

Hey @constancecchen - thanks for the reply. My assumption of that endpoint was that that it initiates a full crawl for an engine. Is there a way to pass in a single URL like the Swiftype version? That is the feature I am looking for.

Thanks again.

---

<div class="post-metadata">

### Author: ![Rich\_Kuzsma](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rich_kuzsma/32/66159_2.png) [@Rich\_Kuzsma](https://discuss.elastic.co/u/Rich_Kuzsma)
#### Post date: [June 17, 2021, 7:26pm UTC](https://discuss.elastic.co/t/api-endpoint-to-initiate-crawl-of-single-page/276296/4 "2021-06-17T19:26:36Z")

</div>

Hi @tchocky ,

When you crawl the new domain, do you want the results to be ingested into a new App Search engine, or added to an existing engine? Are you changing the domain each time, or just the entry point within a domain?

Say I have an App Search engine with a crawl already configured for it. The crawl has 1 domain, `example.com`, and 1 entry point, `/foo`. I can change the entry point from `/foo` to `/bar` by invoking:

```auto
GET /api/as/v0/engines/$ENGINE/crawler
{
  "domains": [
    {
      "id": "$DOMAIN_UUID",
      "name": "https://example.com",
      "document_count": 0,
      "created_at": "Thu, 17 Jun 2021 19:19:02 +0000",
      "last_visited_at": "Thu, 17 Jun 2021 19:19:15 +0000",
      "entry_points": [
        {
          "id": "$ENTRY_POINT_UUID",
          "value": "/foo",
          "created_at": "Thu, 17 Jun 2021 19:19:02 +0000"
        }
      ],
      "crawl_rules": [],
      "default_crawl_rule": {
        "id": "-",
        "order": 0,
        "policy": "allow",
        "rule": "regex",
        "pattern": ".*",
        "created_at": "Thu, 17 Jun 2021 19:20:54 +0000"
      },
      "sitemaps": []
    }
  ],
  "onboarding_completed": true
}

PUT /api/as/v0/engines/$ENGINE_NAME/crawler/domains/$DOMAIN_UUID/entry_points/$ENTRY_POINT_UUID
{"value":"/bar"}

```

Then I can invoke the API @constancecchen mentioned to start the crawl.

---

<div class="post-metadata">

### Author: ![tchocky](https://avatars.discourse-cdn.com/v4/letter/t/b77776/32.png) [@tchocky](https://discuss.elastic.co/u/tchocky)
#### Post date: [June 17, 2021, 7:49pm UTC](https://discuss.elastic.co/t/api-endpoint-to-initiate-crawl-of-single-page/276296/5 "2021-06-17T19:49:21Z")

</div>

Hi @Rich_Kuzsma -

Yes, it is one domain within one engine. When a new page is created in our CMS I am looking to crawl that page right away rather than waiting for a scheduled crawl. We have 10K+ pages to crawl so it might take a while for the new page to be indexed if I don't manually trigger it. Your solution seems like it might work and I appreciate your creative thinking.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 15, 2021, 7:49pm UTC](https://discuss.elastic.co/t/api-endpoint-to-initiate-crawl-of-single-page/276296/6 "2021-07-15T19:49:24Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
