Crawler always fails when there are several domains

Elastic Search
1

When I run the crawler with only one domain configured, it runs successfully, even this domain has thousands of pages.

After that, I delete the domain, add a new one and start the crawler again.

This works fine, but it's a very manual and unrealistic process.

So if I add multiple domains (the same ones that worked before), the crawler always fails. What is the reason? The crawler should work the same for one or multiple domains.

2

Did you checked the logs?
This Kibana Crawler Dashboard can help you.

3

Hi @mbil !

I've been able to setup crawling multiple domains. Is there any specific error you're retrieving from the application, either in the UI or logs?

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.