[OPEN-ILS-DEV] url checker

[Duimovich, George]

Hello Gislaine,
 
NRCan Library has one we submitted to the community a while back:
 
http://markmail.org/message/kgbpzgg25cm6fqcs
 
It will investigate 856 links and generate a CSV report that looks like this:
 
Record ID, Link, Error
8214000,http://www.ga.gov.au/minerals/research/national/geothermal/hot_rock_map.jsp,404 Not Found
8212793,http://geopub.nrcan.gc.ca/moreinfo_e.php?id=226091,500 read timeout
8212662,http://www.sciencedirect.com/science/journal/00456535,400 Bad Request

We usually import to Excel or HTML and send the report off to our tech services that way.
 
It works pretty well but there's lots of room for improvement.  A few sites (e.g. Sciencedirect.com) detect the url checker and don't like "robots" so will return a "bad request" instead of something more useful, so some of the error reporting can be ignored for confirmed sites of this kind of case. 
 
If anybody is interested in in working on improving it let us know, otherwise we'll try to find some time early this Fall to re-examine and improve the checker.
 
George
 
George Duimovich, MLIS 
NRCan Library / Bibliothèque de RNCan 
george.duimovich at nrcan.gc.ca 
tel: 613.996.2101 

________________________________

From: open-ils-dev-bounces at list.georgialibraries.org [mailto:open-ils-dev-bounces at list.georgialibraries.org] On Behalf Of Gislaine.Hamelin at statcan.gc.ca
Sent: June 28, 2011 17:24
To: open-ils-dev at list.georgialibraries.org
Subject: [OPEN-ILS-DEV] url checker



Is anyone currently or planning on developing a url checker for Evergreen?

 

Gislaine Hamelin
Integrated Library Systems Coordinator | Coordonnatrice de système intégré de la bibliothèque
Information Resources Management | Gestion des ressources d'information | Statistics Canada Library | Bibliothèque de Statistique Canada
R.H. Coats Building | Immeuble R.-H.-Coats / Floor | Étage 2 Q
Statistics Canada | 100 Tunney's Pasture Driveway, Ottawa ON K1A 0T6
Statistique Canada | 100, promenade Tunney's Pasture, Ottawa ON K1A 0T6
Gislaine.Hamelin at statcan.gc.ca
Telephone | Téléphone 613-951-4576
Facsimile | Télécopieur 613-951-0939
Government of Canada | Gouvernement du Canada

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://libmail.georgialibraries.org/pipermail/open-ils-dev/attachments/20110628/c8f0e352/attachment-0001.htm>