How do you block bots?

Elastic Search
1

We are starting to get a lot of bot hits in our Swiftype search and I was wondering if anyone else has had this issue and what you did to solve it?

I don't see any captcha feature available and the volume of this bot traffic is messing up our query limit as well as reporting.

2

Hey Shawn,

There are no specific features built-in to the Site Search Application itself, however it should be possible to block requests initiated by bots or even other search agents (like googlebot), by incorporating some degree of input validation from the front-end of your site.

You'll want to consider the manner in which the requests are being made from either your host logs or a dedicated analytics platform like GA to trace the session or point of origin for the searches. I've seen cases where the behavior is due to an errant crawler that is able to access the Search Page of a site and is then attempting to submit information into any available input. For the non-bad actors, you can typically add a directive to either the site's robots.txt file or to individual pages via robots meta tags.

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.