# Best Web crawler to index data to elasticsearch

**URL:** <https://discuss.elastic.co/t/best-web-crawler-to-index-data-to-elasticsearch/327759>\
**Category:** Elasticsearch\
**Created:** [March 15, 2023, 1:55pm UTC](https://discuss.elastic.co/t/best-web-crawler-to-index-data-to-elasticsearch/327759 "2023-03-15T13:55:05Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Disha\_Bodade](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/disha_bodade/32/84522_2.png) [@Disha\_Bodade](https://discuss.elastic.co/u/Disha_Bodade)\
**Post date:** [March 15, 2023, 1:55pm UTC](https://discuss.elastic.co/t/best-web-crawler-to-index-data-to-elasticsearch/327759/1 "2023-03-15T13:55:05Z")

</div>

Hi Team,  
As elastic appsearch supports to crawl websites which uses basic auth alone.  
Any suggestions which web crawler we can use with elasticsearch to crawl the SSO, SAML, NTLM protected websites?

Thanks,  
Disha

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 12, 2023, 1:55pm UTC](https://discuss.elastic.co/t/best-web-crawler-to-index-data-to-elasticsearch/327759/2 "2023-04-12T13:55:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.