Webcrawler

Elastic Search
1

Hello everybody,

I need help with the web crawler. I want the web crawler to use google and index all pages on a certain topic. Does it work and if so, how?

Thanks for help.

2

Hi Cama,

Unfortunately, no - the web crawler is only intended to crawl sites that belong to you and that you have control over (for changing things like noindex headers/meta tags, robots.txt, and so forth). It is not intended to be used for 3rd party sites or services such as Google.

Hope that helped,
Constance

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.