Suggestions for web crawler

Elastic Stack Elasticsearch
1

Hi,

I am using ES 5.6.3 with xpack and need suggestions for web crawler. Can I use ES site search for crawling and ingest in to my ES 5.6.3 index ? Saw suggestions on Apache storm. Does it have features like include/exclude uri’s ? Can it handle bigger ones ? Is it easy to customise?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.