[OPEN-ILS-GENERAL] ***SPAM*** Re: Here we grow again! Link checker functionality in Evergreen

[Paul Hoffman]

I had written:
> *please* let's not develop an *integrated* link checker for Evergreen at all

On Fri, May 11, 2012 at 06:17:21PM -0400, Duimovich, George wrote:
> Disagree -- there's no reason why link checker code couldn't be 
> developed to be used (or easily adapted) to service both general 
> purpose roles as well as "integrated" Staff Client functionality. What 
> do you do for libraries who don't have direct server or easy database 
> access? How else can cataloguers be empowered to service link errors, 
> especially as we move towards increasing digital collections? 

It sounds like we actually agree -- I would like to see Evergreen use an 
external link checker, but the user shouldn't be able to tell that it's 
external.

> Advantages of integration could include, for example, exposing batch 
> editing and more effective reporting, etc.
> 
> I won't argue the point right now but "integration" needn't be 
> incompatible with dis-integration, say encouraging the ability to 
> generalize the service or allowing alternate solutions to the problem.

Exactly.

> There is currently some sys admin oriented link checker code out there 
> already for Evergreen that could be adapted (and improved) for general 
> purpose usage (esp. for any database generated link listings, etc.)  
> For some basic working code, see our earlier submissions here:
> 
> http://markmail.org/message/v4cyszy33sgocqab and here: 
> http://markmail.org/message/kgbpzgg25cm6fqcs  (with maybe some 
> additional enhancements from Liam's work to be submitted by us later). 

Thanks for the pointers.

Paul.

> -----Original Message-----
> From: open-ils-general-bounces at list.georgialibraries.org [mailto:open-ils-general-bounces at list.georgialibraries.org] On Behalf Of Paul Hoffman
> Sent: May 11, 2012 15:14
> To: open-ils-general at list.georgialibraries.org; open-ils-dev at list.georgialibraries.org
> Subject: [OPEN-ILS-GENERAL] ***SPAM*** Re: Here we grow again! Link checker functionality in Evergreen
> 
> On Fri, May 11, 2012 at 02:55:42PM -0400, Suzannah Lipscomb wrote:
> > Equinox Software, Inc. is excited to announce the development of link 
> > checker functionality in Evergreen.
> 
> Uh-oh!
> 
> > Evergreen currently has no built-in mechanism for verifying the 
> > validity of URLs stored in MARC records. The ability to verify URLs 
> > will be of particular benefit to locations with large electronic 
> > resource collections. The requirements for this project are being 
> > developed in partnership with NRCan Library and Statistics Canada 
> > Library. The technical specifications for this project will be shared 
> > with the Evergreen Community once they are ready. Equinox developers 
> > estimate that coding will be completed no later than the end of the 
> > third quarter of 2012.
> 
> As someone who has had to deal with poorly written link checkers and the havoc they wreak, I sincerely hope that one of the requirements will be
> *politeness* -- specifically, a throttling mechanism that keeps the link checker from hammering away at a server that happens to serve a large number of resources linked to from an Evergreen catalog.  (This is actually quite simple in most cases: just shuffle the list of URLs that you check, and don't check too fast.  If you have 860,400 links in a catalog you can check ten per second and still finish in 24 hours, but you'd better make sure that those ten per second aren't all requested from the same server!)
> 
> And -- though it's not clear if this is what's intended or not --
> *please* let's not develop an *integrated* link checker for Evergreen at all.  A link checker is properly a separate special-purpose tool with a simple, well-defined interface that allows it to be used by any number of applications; it shouldn't be all tangled up in an ILS.  There might even be a perfectly good link checker that fits the bill now -- I don't know, as I haven't looked into the matter very closely.
> 
> OK, I'll get down off my soap box now.  :-)
> 
> Paul.
> 
> --
> Paul Hoffman <paul at flo.org>
> Systems Librarian
> Fenway Libraries Online
> c/o Wentworth Institute of Technology
> 550 Huntington Ave.
> Boston, MA 02115
> (617) 445-2914
> (617) 442-2384 (FLO main number)

-- 
Paul Hoffman <paul at flo.org>
Systems Librarian
Fenway Libraries Online
c/o Wentworth Institute of Technology
550 Huntington Ave.
Boston, MA 02115
(617) 445-2914
(617) 442-2384 (FLO main number)