# \#open-crawler

**URL:** https://discuss.elastic.co/tag/open-crawler/152.md

[Latest](https://discuss.elastic.co/latest.md) · [Categories](https://discuss.elastic.co/categories.md) · [Tags](https://discuss.elastic.co/tags.md)

---

## [Extraction rules for meta tag without "elastic" using a XPath selector](https://discuss.elastic.co/t/extraction-rules-for-meta-tag-without-elastic-using-a-xpath-selector/386450)

<div class="topic-metadata">

**Author:** [@creekinouryard](https://discuss.elastic.co/u/creekinouryard)\
**Replies:** 1\
**Last updated:** [May 29, 2026, 5:42pm UTC](https://discuss.elastic.co/t/extraction-rules-for-meta-tag-without-elastic-using-a-xpath-selector/386450 "2026-05-29T17:42:50Z")

</div>

Using the latest version of Open Crawler and having trouble extracting meta tag values We have a HTML page as follows: \<html lang="en-ca" xmlns=" http://www.w3.org/1999/xhtml "\> \<head\> \<!--\<meta http-equiv=Content-…

---

## [Elastic Open Crawler 0.3.0 - /var/lib/docker/overlay filling up](https://discuss.elastic.co/t/elastic-open-crawler-0-3-0-var-lib-docker-overlay-filling-up/379635)

<div class="topic-metadata">

**Author:** [@alongaks](https://discuss.elastic.co/u/alongaks)\
**Replies:** 7\
**Last updated:** [July 1, 2025, 3:45pm UTC](https://discuss.elastic.co/t/elastic-open-crawler-0-3-0-var-lib-docker-overlay-filling-up/379635 "2025-07-01T15:45:14Z")

</div>

Hello, I have been working with the Open Crawler for a bit, trying to tune it to replace the Enterprise Search crawler at some point. While running crawls I began seeing the following errors in logs and the resultant d…

---

## [Open Crawler - Scheduled Crawl](https://discuss.elastic.co/t/open-crawler-scheduled-crawl/377455)

<div class="topic-metadata">

**Author:** [@alongaks](https://discuss.elastic.co/u/alongaks)\
**Replies:** 4\
**Last updated:** [April 28, 2025, 12:42pm UTC](https://discuss.elastic.co/t/open-crawler-scheduled-crawl/377455 "2025-04-28T12:42:30Z")

</div>

Hi, edit: version 0.2.2 crawler I tried to run a crawl based on a schedule, but I don't think the crawl is starting? It may be something Im missing but wanted to check if anyone else ran into this. If I run a normal, …

---

## [Crawl options for Elastic Open Web Crawler](https://discuss.elastic.co/t/crawl-options-for-elastic-open-web-crawler/376917)

<div class="topic-metadata">

**Author:** [@alongaks](https://discuss.elastic.co/u/alongaks)\
**Replies:** 2\
**Last updated:** [April 15, 2025, 5:20pm UTC](https://discuss.elastic.co/t/crawl-options-for-elastic-open-web-crawler/376917 "2025-04-15T17:20:21Z")

</div>

Hello, I am doing some initial work with the Open Web Crawler and have a few settings I'm looking if they exist yet. The current Enterprise Search Crawler allows for the following: connector.crawler.crawl.threads.limi…

---

## [Elastic Open Web Crawler](https://discuss.elastic.co/t/elastic-open-web-crawler/376742)

<div class="topic-metadata">

**Author:** [@alongaks](https://discuss.elastic.co/u/alongaks)\
**Replies:** 3\
**Last updated:** [April 4, 2025, 9:00pm UTC](https://discuss.elastic.co/t/elastic-open-web-crawler/376742 "2025-04-04T21:00:59Z")

</div>

Hello, I am trying out the Open Web Crawler in preperation of whenever it replaces the existing Enterprise Search crawler and associated functionality. My environment uses a single node Elasticsearch, Kibana and Enterp…

---

## [Dec 12th, 2024: \[EN\] Swing through web content like a superhero](https://discuss.elastic.co/t/dec-12th-2024-en-swing-through-web-content-like-a-superhero/371474)

<div class="topic-metadata">

**Author:** [@ppf2](https://discuss.elastic.co/u/ppf2)\
**Replies:** 0\
**Last updated:** [December 12, 2024, 8:00am UTC](https://discuss.elastic.co/t/dec-12th-2024-en-swing-through-web-content-like-a-superhero/371474 "2024-12-12T08:00:00Z")

</div>

Swing through web content like a superhero Web crawlers at Elastic have undergone multiple evolutions throughout the years to adapt to the rapidly changing landscape of data ingestion (e.g., recent advancements in gen…
