[OPEN-ILS-DEV] url checker

[Ben Ostrowsky]

On Tue, Jun 28, 2011 at 5:47 PM, Duimovich, George <
George.Duimovich at nrcan-rncan.gc.ca> wrote:

> **
> It works pretty well but there's lots of room for improvement.  A few
> sites (e.g. Sciencedirect.com) detect the url checker and don't like
> "robots" so will return a "bad request" instead of something more useful, so
> some of the error reporting can be ignored for confirmed sites of this kind
> of case.
>

Setting the User-Agent string can help with this:


 $ua->agent('Mozilla/5.0');

Better living through prevarication! :)

Ben
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://libmail.georgialibraries.org/pipermail/open-ils-dev/attachments/20110701/694bc16d/attachment.htm>