# Does Elastic Web Crawler supports noindex and nofollow directive **URL:** \ **Category:** Elastic Search\ **Tags:** elastic-app-search\ **Created:** [October 11, 2023, 1:22pm UTC](https://discuss.elastic.co/t/does-elastic-web-crawler-supports-noindex-and-nofollow-directive/344821 "2023-10-11T13:22:00Z")\ **Posts on this page:** 1\ **Showing post:** 9
**Author:** ![sebastianboelling](https://avatars.discourse-cdn.com/v4/letter/s/258eb7/32.png) [@sebastianboelling](https://discuss.elastic.co/u/sebastianboelling)\ **Post date:** [November 10, 2023, 1:35pm UTC](https://discuss.elastic.co/t/does-elastic-web-crawler-supports-noindex-and-nofollow-directive/344821/9 "2023-11-10T13:35:41Z")
Hi @Sean_Story, no worries. We can investigate a bit deeper. But what is the expected behavior for the Elastic Web Crawler from your point of view? In which case pages are deleted? case 1: page with `index` meta tag is in index -\> page switches to `noindex` -\> **full** re-crawl -\> page deleted from index: _yes_ vs. _no_ case 2: page with `index` meta tag is in index -\> page switches to `noindex` -\> **partial** crawl -\> page deleted from index: _yes_ vs. _no_ case 3: page with `index` meta tag is in index -\> page is deleted -\> HTTP 404 -\> **full** re-crawl -\> page deleted from index: _yes_ vs. _no_ case 4: page with `index` meta tag is in index -\> page is deleted -\> HTTP 404 -\> **partial** crawl -\> page deleted from index: _yes_ vs. _no_ Other cases for deleting? We only need to understand and can the investigate and decide how to delete or force deletion of pages from our indices. Best regards Sebastian --- _[View the full topic](https://discuss.elastic.co/t/does-elastic-web-crawler-supports-noindex-and-nofollow-directive/344821)._