[OPEN-ILS-DEV] Evergreen search discussion

[liam.whalen at bc.libraries.coop]

Quoting Kathy Lussier <klussier at masslnc.org>:

> Hi all,
>
> I've added a topic to the hack-a-way agenda to discuss Evergreen  
> search. However, I wanted to raise the topic here on the list first  
> since the discussion may require some forethought and because I  
> imagine there are people interested in this discussion who won't be  
> attending the hack-a-way.
>
> In discussing our development priorities for the year, MassLNC  
> decided to focus on making improvements to search in Evergreen. We  
> view  search as one of the most important pieces of the ILS, if not  
> the most important. It's what allows our users to find those  
> resources we spend so much time cataloging so that they can then  
> place holds on them, check them out from the library, or access them  
> in some other way.
>
> There are some specific development projects we identified as  
> possibilities: Did you Mean? functionality, working auto-suggest,  
> improved speed, etc. However, rather than tacking these improvements  
> on to the existing search, we thought it might be a good time for  
> the community to step back, take a big-picture look at how we're  
> doing search, and determine if we should continue down this path, if  
> we need to make major underlying changes for our current path to be  
> more performant/effective, or if we should consider moving to  
> something else to handle Evergreen search.
>
> Would it be worthwhile to move to something like Solr or  
> Elasticsearch or something other thing to handle Evergreen searches?  
> If not, are there changes we should do to better utilize  
> improvements full-text search that have been made to recent versions  
> of PostgreSQL? I don't have the answers to these questions, but I  
> think it's worthwhile for the community to identify what we expect  
> of Evergreen search and to do a thorough analysis of available  
> options to determine what will best help us attain those goals.

I think it is worth while considering a move to Solr &  
VuFind/Blacklight.  I think writing an OAI-PMH repository for  
Evergreen and a harvester for Solr would allow us to maintain near  
real time catalogue details.  EG could be modified to trigger the Solr  
harvester on update/insert/delete.

 From some research I have done today, there is some work already done  
in indexing MARC in Solr.  The code indexes binary MARC, but we could  
modify it to index MARCXML.

It looks like most of the systems in place rely upon dumping MARC data  
to file and indexing it in Solr, which is not very responsive.  It  
would be better to have EG notify Solr that a record has changed and  
have Solr harvest the changes via OAI-PMH.

>
> Over the past few months, the folks at MassLNC have started a  
> discussion of what our overall goals for search are. From these  
> discussions, we have created a vision for what we would like to see  
> in Evergreen search - http://masslnc.org/search_vision .
>

Thank you for sharing your vision.

> From this search vision, we then identified specific areas of  
> improvements / new features that would help Evergreen reach this  
> vision. We also identified areas where we already are doing well and  
> will want to maintain - http://masslnc.org/node/3164.
>
> I'm sure there are some areas where others may disagree with our  
> ideas, but I'm guessing there are other areas where we'll get broad  
> community consensus around some of these search priorities.
>
> I don't think we're in a position where we can choose a direction at  
> the hack-a-way, but maybe we can do the following:
>
> * At the hack-a-way,  can we have a discussion to see if there is  
> interest in this project? We might also be able to identify some  
> viable options that could be explored at the hack-a-way.
> * After the hack-a-way, the community could work on setting and  
> prioritizing high-level goals for search in the Evergreen catalog.  
> Ideally, we would have these search goals ready by the end of the  
> calendar year. I would be willing to help facilitate this process.
> * After the goals are identified, we explore available options to  
> see which will the best to help us attain those goals. It would be  
> great if we had the ability to do some prototypes during this phase,  
> but this would depend on people having the time / resources to do  
> those prototypes.
> * Ideally, by the time we meet again at the conference hackfest in  
> April, we'll be in a position where we can set a direction for  
> search and then move forward with development.
>

I look forward to speaking about this at the Hack-a-way.

Liam

> I'm sure the process won't be as simple as what I outlined above,  
> and all of you may have better ideas on the best ways to evaluate  
> our options. But I'm hoping this email helps us kick off a  
> conversation that ultimately leads to fast and relevant search in  
> Evergreen.
>
> Thanks!
> Kathy
>
> -- 
> Kathy Lussier
> Project Coordinator
> Massachusetts Library Network Cooperative
> (508) 343-0128
> klussier at masslnc.org
> Twitter:http://www.twitter.com/kmlussier