[OPEN-ILS-DEV] Mobius Carousel - reducing number of requests?

Tue Oct 16 15:06:59 EDT 2018

Thanks Josh!

I will have to take a look at this when I get a chance. It sounds like 
you are doing some good stuff here. Hopefully those that have chosen to 
integrate the carousel can also benefit. I will merge the branch when I 
have the time to devote to it.

-Blake-
Conducting Magic
MOBIUS

On 10/16/2018 9:57 AM, Josh Stompro wrote:
>
> Blake, I found that duplicate records in the title bucket will break 
> the carousel with my changes.  I think that may be because 
> open-ils.search.biblio.record.mods_slim.retrieve must not return 
> duplicates.  Or maybe it is because I create a hash based off the 
> bibid to store the ordering?  That isn’t a deal breaker for us, since 
> I didn’t intend to have duplicates in the lists, they were just there 
> by mistake. But it probably shouldn’t break because of that.  So I’ll 
> look into that.
>
> I also was happy to find that there are options to allow wrapping of 
> the carousel, so once you get to the end it starts again at the 
> beginning, along with autoscrolling to new items get shown ever x seconds.
>
> Adding the following to the .jcarousel config enables those
>
> wrap: ‘both’,
>
> auto: 9,
>
> The next step to make it load even faster would be to cache the 
> open-ils.search.biblio.record.mods_slim.retrieve results somehow.  For 
> our new arrival list, it takes our system 3219ms to generate that for 
> 200 titles.
>
> Josh Stompro - LARL IT Director
>
> *From:*Open-ils-dev <open-ils-dev-bounces at list.georgialibraries.org> 
> *On Behalf Of *Josh Stompro
> *Sent:* Saturday, October 13, 2018 4:50 PM
> *To:* Evergreen Development Discussion List 
> <open-ils-dev at list.georgialibraries.org>
> *Subject:* Re: [OPEN-ILS-DEV] Mobius Carousel - reducing number of 
> requests?
>
> Blake, I sent you a pull request with the changes.
>
> https://github.com/mcoia/mobius_evergreen/pull/3/commits/9e6de7dd80f182488012b64fa8311162ed433e6b
>
> I did get the ordering to work, but I’m not sure I did it in the most 
> efficient manner.
>
> Josh
>
> *From:*Open-ils-dev <open-ils-dev-bounces at list.georgialibraries.org 
> <mailto:open-ils-dev-bounces at list.georgialibraries.org>> *On Behalf Of 
> *Blake Henderson
> *Sent:* Saturday, October 13, 2018 4:11 PM
> *To:* open-ils-dev at list.georgialibraries.org 
> <mailto:open-ils-dev at list.georgialibraries.org>
> *Subject:* Re: [OPEN-ILS-DEV] Mobius Carousel - reducing number of 
> requests?
>
> Josh,
>
> Great detective work! The culprit is likely the bbag.js code as you 
> have mentioned. It sounds like you have made a major improvement. 
> Really, the whole thing needs to be moved into action_trigger and 
> library settings and eventually baked into Evergreen. Share your 
> improvements as you make them! I could imagine a path forward coding a 
> custom JS carousel (instead of using jcarousel) using some of the 
> great Angular things. Coupled with a server side component in the form 
> of an additional gateway-v1 listener to provide the JSON data as 
> needed. Right now, the bbag.js code downloads the entire bookbag and 
> injects the ID's into a hidden DOM element. Perhaps it's time to open 
> that can of worms again? Is there a LP bug for this?
>
> -Blake-
> Conducting Magic
> MOBIUS
>
> On 10/13/2018 10:05 AM, Josh Stompro wrote:
>
>     Hello, I’m moving this over to the dev list.
>
>     Using the batch method to flesh the titles from the bookbags does
>     seem to offer significant performance improvements for us.
>     (Sending all bib id’s in one array to
>     open-ils.search.biblio.record.mods_slim.retrieve)
>
>     Measured in chrome 69.0.3497.100
>
>     Before: (hot cache) 19 second page load, 480 requests, 757kb
>     transferred.
>
>     After: (hot cache) 6 second page load, 76 requests, 434kb transferred.
>
>     Difference: 68% reduction in page load time, 84% reduction in
>     requests, 42% reduction in transfer amount.
>
>     I haven’t dealt with getting the correct order yet,
>     open-ils.search.biblio.record.mods_slim.retrieve returns results
>     in bibid order.
>
>     Josh
>
>     *From:* Open-ils-general
>     <open-ils-general-bounces at list.georgialibraries.org>
>     <mailto:open-ils-general-bounces at list.georgialibraries.org> *On
>     Behalf Of *Josh Stompro
>     *Sent:* Friday, October 12, 2018 2:31 PM
>     *To:* Evergreen Discussion Group
>     <open-ils-general at list.georgialibraries.org>
>     <mailto:open-ils-general at list.georgialibraries.org>
>     *Subject:* [OPEN-ILS-GENERAL] Mobius Carousel - reducing number of
>     requests?
>
>     Hello, We have been using the carousel from Mobius for a while
>     now.  And I just noticed how resource intensive it seems to be.
>
>     Here are the slides from the presentation that Bake created about it.
>
>     http://slides.mobiusconsortium.org/blake/bookcarousel/#/1
>
>     And the code
>     https://github.com/mcoia/mobius_evergreen/tree/master/bookbag_update
>
>     We have ~ 30 catalog machines that are set to reset every 5
>     minutes of idle time (since people don’t always log themselves
>     out.)  When I watch our catalog page load which currently has 3
>     sliders, I see 483 requests made from the browser.  ~450 of those
>     requests are from the carousel, since it request each book bag,
>     and then request info for each bookbag item.  And then request
>     images for the visible items.  So our 30 catalog machines generate
>      162000 requests an hour.
>
>     I’m wondering if anyone has tried pre-generating the bookbag json
>     data on a schedule so the web page can do just one request for the
>     titles?  It also seems like the only info that is used is the
>     title, while 1-2kb of data gets returned about each item.  So a
>     lot of unused data gets moved around.  The script that generates
>     the static data file could do things like randomize the order of
>     the items, so a static bookbag list would show different titles
>     whenever it is re-generated.
>
>     It looks like open-ils.search.biblio.record.mods_slim.retrieve can
>     accept an array of ID’s, so that seems like it would reduce the
>     number of requests, so that may be another way to reduce the
>     number of requests.
>
>     Thanks
>
>     Josh
>
>     Lake Agassiz Regional Library - Moorhead MN larl.org
>
>     Josh Stompro     | Office 218.233.3757 EXT-139
>
>     LARL IT Director | Cell 218.790.2110
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://libmail.georgialibraries.org/pipermail/open-ils-dev/attachments/20181016/7a7c046e/attachment-0001.html>