I can crawl external webs but when I tried to crawl a web that is only accessible from my internal net I couldn't do.
I far as I've read it seems feasible, right? I've search the web but I couldn't find any post/topic talking about this.
Thanks!
Hello,
By default, Enterprise Search crawler configuration prohibits access to any websites that use a private IP address (or a loopback IP). This is by design, because it opens up your deployment to a number of pretty serious attacks including SSRF. On Elastic Cloud, this configuration is impossible to change since we have to protect our internal services from SSRF attacks.
If you're running Enterprise Search in your own environment (self-managed) and you both completely trust all users with access to its management console and you own (control DNS configuration) the domains you're planning to crawl, you can disable this protection by setting the crawler.security.dns.allow_private_networks_access to true in your enterprise-search.yml and restarting the service. But, once again, I'd like to point out that it opens you up to a number of different attacks and you need to be very careful with what you crawl with this setting enabled.
I hope this helps.
Thank you very much, you've answered my question!