[OPEN-ILS-DEV] Auto-suggestion sorting and highlighting issues with diacritics

[Yamil Suarez]

Hello,

We noticed an issue with auto-suggest and the use diacritics in TPAC on EG 2.4, and probably since 2.2. I wondered if anyone else has seen this issue or has a fix, before I put in a bug report for it.

Here is an example of what we are seeing in terms of sorting, when we are ultimately looking for someone named "Morgan, Jenée Morgan". 


- If we use a single search term that includes a diacritic (Jenée) or without (Jenee), either way we get the following suggestions…

Dixon, Jenee 

Fleener, Jenee

Fleenor, Jenee

Haggard, Jenessa

Jenei, Szilveszter

Jeney, Zoltán

Jenez, Zoltan

Marshall, Jenell

Williams, Jenevora

Force, Jenée Morgan


- What we don't like is that the first three suggestions have the term "Jenee" without the diacritic, but the suggestion that actually has the diacritic is the last of all the suggestions. This suggestion sorting makes sense for when we do not use the diacritic, but in our opinion it does not makes as much sense when we do use the diacritic. BTW, I suspect that "cover density" might be at play here since "Morgan, Jenée Morgan" has three words and the others have only two, and when either the term "Morgan" or "Force" is added to the search the sorting improves. 


- I guess I would prefer a sorting that was closer to this…

Dixon, Jenee 

Fleener, Jenee

Fleenor, Jenee

Force, Jenée Morgan

Haggard, Jenessa

Jenei, Szilveszter

Jeney, Zoltán

Jenez, Zoltan

Marshall, Jenell

Williams, Jenevora


- If we only searched Jenée, it would be nice if the matching suggestion would be at least the 4th one listed, though perhaps it should be even higher in order if the patron used the diacritic. 


Is this a bug or is there something that I can do to tweak the sort order?


Another issue that we have with auto-suggest has to do with what part of the suggestion is highlighted & bolded when it matches the patrons search term(s). For example, currently if a patron searches for "Jenee" (without the diacritic) all of the above suggestions that contain "Jenee" are highlighted automatically, but not the suggestion that contains "Jenée" (with the diacritic). 

If a patron searches for "Jenée" (with the diacritic) none of the above suggestions are highlighted, not even the suggestion that contains "Jenée" (with the diacritic). If I recall correctly, when auto-suggest was originally built for us we were warned that this would happen, but now I wonder how hard it would be to reverse this behavior. I suspect those working with i18n in Evergreen might have an interest in this behavior.

Thoughts?

Thanks for reading,
Yamil