<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<div class="moz-cite-prefix">Thank you very much, Jane!</div>
<div class="moz-cite-prefix"><br>
</div>
<div class="moz-cite-prefix">We will certainly give fail2ban a try,
though - as we use Apache - some implementation details will
probably be a bit different :-).<br>
</div>
<p>Linda<br>
</p>
<div class="moz-cite-prefix">On 4/19/24 13:05, Jane Sandberg wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAH++r7EfCwvOqVV2MzYiMg2v63Vy-eRN7aFa9LpErxkV-hpdpw@mail.gmail.com">
<div dir="ltr">Hi Linda,
<div><br>
</div>
<div>It's not for Evergreen, but my colleague <a
href="https://github.com/pulibrary/princeton_ansible/commit/6f9009249a168442391d90e2b75028d40a8a9e91"
moz-do-not-send="true">recently blocked claudebot using
fail2ban on our load balancer</a>. Essentially, fail2ban is
configured to watch Nginx's access log, and if more than 10
claudebot requests appear within the past minute from a
particular IP, it automatically blocks all requests from that
IP for the next 24 hours. I would think that something
similar could work for Apache's access log.</div>
<div><br>
</div>
<div>Good luck with the bots!</div>
<div><br>
</div>
<div> -Jane</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">El vie, 19 abr 2024 a la(s)
3:42 a.m., Linda Jansová via Evergreen-general (<a
href="mailto:evergreen-general@list.evergreen-ils.org"
moz-do-not-send="true" class="moz-txt-link-freetext">evergreen-general@list.evergreen-ils.org</a>)
escribió:<br>
</div>
<blockquote class="gmail_quote">Dear all,<br>
<br>
Have any of you encountered an extensive crawling by
Bytespider and <br>
Bytedance (see e.g., <br>
<a
href="https://wordpress.org/support/topic/psa-bytedance-and-bytespider-bots-recommend-blocking/"
rel="noreferrer" target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">https://wordpress.org/support/topic/psa-bytedance-and-bytespider-bots-recommend-blocking/</a>),
<br>
Claudebot or other AI bots?<br>
<br>
If so, do you have any secret recipe how to disable the
crawler from <br>
accessing the site?<br>
<br>
Thank you very much for sharing your experience!<br>
<br>
Linda<br>
<br>
_______________________________________________<br>
Evergreen-general mailing list<br>
<a href="mailto:Evergreen-general@list.evergreen-ils.org"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">Evergreen-general@list.evergreen-ils.org</a><br>
<a
href="http://list.evergreen-ils.org/cgi-bin/mailman/listinfo/evergreen-general"
rel="noreferrer" target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">http://list.evergreen-ils.org/cgi-bin/mailman/listinfo/evergreen-general</a><br>
</blockquote>
</div>
</blockquote>
<p><br>
</p>
</body>
</html>