[OPEN-ILS-DEV] cstore vs storage

[James Fournie]

Thanks Mike, this is the answer I was looking for.  This really
clarifies this dual CRUD business.  I have seen the cstore.json_query
and it looks great for adding bits to the circ scripts.

~James

On Mon, Dec 1, 2008 at 6:15 PM, Mike Rylander <mrylander at gmail.com> wrote:
> On Mon, Dec 1, 2008 at 5:51 PM, James Fournie
> <jfournie at sitka.bclibraries.ca> wrote:
>> Hi there,
>>
>> I am just wondering the differences between open-ils.cstore and
>> open-ils.storage. I am just wondering if there are any guidelines for
>> using one or the other of these, and if there is a list of the
>> differences.
>>
>
> Some history will help, I think.
>
> open-ils.storage was the first ORM we built.  It's written in Perl (as
> you know) and it started life with two goals:  to provide simple CRUD
> and search methods for all object types backed by tables and views;
> and to provide a platform for complex queries that needed to be
> constructed by hand (for speed or abstraction, but of which the rest
> of the system should not need to know the details.
>
> Once upon a time it had about 25,000 generated methods for performing
> various different search operations against every field on every
> table.  All that's really left now is a simple search method for each
> table that accepts a hash of criteria, along with the afformentioned
> CRUD (and id-list) methods.  All of the CRUD methods now have analogs
> in open-ils.cstore now, but more on that later.
>
> The second (complex queries returning OpenSRF objects) part has lived
> longer, though it is constantly being pared down as well.  The bib
> search methods, hold targeter implementation, fine generator and fa
> few other complex methods see the majority of the use now.  There will
> always be a need for some biz-logic/database border methods that do
> something more than simple CRUD, and today that is where these border
> methods are built.  We want to keep the number of ways the database
> can be directly touched to a minimum, and since Perl is much easier
> than C to write quickly it will be around for a while.
>
> open-ils.cstore, on the other hand, is written in C.  It was built
> mainly in response to performance issues with open-ils.storage,
> particularly the CRUD interfaces.  Long story short, simple CRUD and
> search methods went from taking a minimum of 35ms for a full round
> trip (usually significantly more) to taking 3-5ms.  Thank You C!  For
> more complex methods, C is a bit too rigid to encode them directly,
> but in those cases the perl overhead is swamped by the query time
> anyway, so it's not a big deal in many cases.
>
> What we were lacking all along, though, was a way to construct
> query-language agnostic (meaning, flavors of SQL, really) arbitrary
> queries.  This is open-ils.cstore's secondary role, which is becoming
> fairly advanced in 1.0.1 and beyond.  It's possible now to build a
> data structure which mirrors the basic layout of an SQL query and have
> OpenSRF objects returned to the client.  This means that many of the
> more complex open-ils.storage methods can be built using
> open-ils.cstore, and things that were not possible before without
> hand-tooling SQL are now relatively easy, and faster.
>
> So, to answer your question, open-ils.storage is where methods that
> need to know about the both the database structure and the business
> logic requirements of Evergreen live, and while it does have a CRUD
> interface (and, in a couple specific cases, should be used for CRUD)
> you should generally go to open-ils.cstore for CRUD access.  Also,
> take a look at some of the more complex open-ils.cstore.json_query
> calls scattered throughout the Perl-based OpenSRF apps in the open-ils
> namespace for a glimpse of the cstore-tastic future.
>
> There are some /really/ outdated docs for open-ils.storage on the
> wiki, but here are some (more or less) up to date, if now incomplete
> for json_query, docs for open-ils.cstore:
>
> http://open-ils.org/dokuwiki/doku.php?id=backend-devel:open-ils_cstore_api
>
>
> --
> Mike Rylander
>  | VP, Research and Design
>  | Equinox Software, Inc. / The Evergreen Experts
>  | phone:  1-877-OPEN-ILS (673-6457)
>  | email:  miker at esilibrary.com
>  | web:  http://www.esilibrary.com
>