Crawler always fails when there are several domains

When I run the crawler with only one domain configured, it runs successfully, even this domain has thousands of pages.

After that, I delete the domain, add a new one and start the crawler again.

This works fine, but it's a very manual and unrealistic process.

So if I add multiple domains (the same ones that worked before), the crawler always fails. What is the reason? The crawler should work the same for one or multiple domains.

Did you checked the logs?
This Kibana Crawler Dashboard can help you.

Hi @mbil !

I've been able to setup crawling multiple domains. Is there any specific error you're retrieving from the application, either in the UI or logs?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.