Preventing/Detecting web crawlers with the ELK stack or XPack

Elastic Stack Elasticsearch
1

Hi,

I’m still familiarizing myself with the ELK stack, including the Xpack and Beats. My organization is finding ways in detecting and also preventing web crawling activities by competing organizations.

I went through the ELK stack, Xpack and Beats stack to see if there are any offerings to achieve my task. However, I can’t seem to locate any solution with the ELK, Beats stacks and Xpack to prevent or detect/rectify such issues.

Given that Elasticsearch itself is a crawler, current remedies appear to be implemented outside of the ELK, Beats and Xpack stacks like the ones described here:
https://blog.hartleybrody.com/prevent-scrapers/

Can anyone share their thoughts about this subject?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.