FSCrawler over HTTPS


I am trying to use FSCrawler to crawl a remote site over https. Does anyone know if this is even possible? I've read the docs and tried doing it over SSH butI get the "Error while crawling https://myserver.pages.com/pages/pages: https://myserver.pages.com/pages/pages doesn't exists."

We are trying to crawl some pages that exist on a public github repository.

FSCrawler has not been designed to crawl a website. https is not ssh protocol.

You might want to explore other solutions:

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.