# Parallel crawler execution **URL:** \ **Category:** Elastic Search\ **Tags:** elastic-app-search\ **Created:** [October 27, 2021, 9:12pm UTC](https://discuss.elastic.co/t/parallel-crawler-execution/287856 "2021-10-27T21:12:45Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![\_leo](https://avatars.discourse-cdn.com/v4/letter/_/87869e/32.png) [@\_leo](https://discuss.elastic.co/u/_leo)\ **Post date:** [October 27, 2021, 9:12pm UTC](https://discuss.elastic.co/t/parallel-crawler-execution/287856/1 "2021-10-27T21:12:45Z")
I would like to create a crawler per domain. This would allow crawling each domain independently and running them at different times. However, it doesn't seem possible. If running the crawler consumes a lot of resources, is there a way to allocate more resources to it ? ---
**Author:** ![orhantoy](https://avatars.discourse-cdn.com/v4/letter/o/c5a1d2/32.png) [@orhantoy](https://discuss.elastic.co/u/orhantoy)\ **Post date:** [October 28, 2021, 10:30am UTC](https://discuss.elastic.co/t/parallel-crawler-execution/287856/2 "2021-10-28T10:30:24Z")
Have you tried the `crawler.workers.pool_size.limit` config? From the reference config file: ```auto # The number of parallel crawls allowed per instance of Enterprise Search. # By default, it is set to 2x the number of available logical CPU cores. # Note: On Intel CPUs, the default value is 4x the number of physical CPU cores # due to hyper-threading (https://en.wikipedia.org/wiki/Hyper-threading). # #crawler.workers.pool_size.limit: N ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 25, 2021, 10:30am UTC](https://discuss.elastic.co/t/parallel-crawler-execution/287856/3 "2021-11-25T10:30:58Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.