# Delete (Elastic Web crawler) crawled Web pages - maybe per Pipeline

**URL:** <https://discuss.elastic.co/t/delete-elastic-web-crawler-crawled-web-pages-maybe-per-pipeline/344974>\
**Category:** Elastic Search\
**Tags:** painless, elastic-app-search\
**Created:** [October 13, 2023, 8:36am UTC](https://discuss.elastic.co/t/delete-elastic-web-crawler-crawled-web-pages-maybe-per-pipeline/344974 "2023-10-13T08:36:01Z")\
**Posts on this page:** 1\
**Showing post:** 5

<div class="post-metadata">

**Author:** ![Sean\_Story](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/sean_story/32/69987_2.png) [@Sean\_Story](https://discuss.elastic.co/u/Sean_Story)\
**Post date:** [December 5, 2023, 2:30pm UTC](https://discuss.elastic.co/t/delete-elastic-web-crawler-crawled-web-pages-maybe-per-pipeline/344974/5 "2023-12-05T14:30:36Z")

</div>

@sebastianboelling if a full crawl is run, pages that change to `noindex` should be deleted. See our discussion on another forum post: [Does Elastic Web Crawler supports noindex and nofollow directive - #9 by sebastianboelling](https://discuss.elastic.co/t/does-elastic-web-crawler-supports-noindex-and-nofollow-directive/344821/9)

---

_[View the full topic](https://discuss.elastic.co/t/delete-elastic-web-crawler-crawled-web-pages-maybe-per-pipeline/344974)._