Hello everybody,

I need help with the web crawler. I want the web crawler to use google and index all pages on a certain topic. Does it work and if so, how?

Thanks for help.

Hi Cama,

Unfortunately, no - the web crawler is only intended to crawl sites that belong to you and that you have control over (for changing things like noindex headers/meta tags, robots.txt, and so forth). It is not intended to be used for 3rd party sites or services such as Google.

Hope that helped,

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.