SPAM: Re: [OPEN-ILS-DEV] PATCH: osrf_json_object.c (miscellaneous)

[Mike Rylander]

On Dec 3, 2007 11:46 PM, Scott McKellar <mck9 at swbell.net> wrote:
>
> --- Mike Rylander <mrylander at gmail.com> wrote:
>
> > On Dec 3, 2007 9:53 PM, Mike Rylander <mrylander at gmail.com> wrote:
> > > On Dec 2, 2007 11:55 PM, Scott McKellar <mck9 at swbell.net> wrote:
> > > [snip]
> > > > 4. I introduced a linked list of jsonObjects that have been
> > allocated
> > > > from the heap but are not currently in use.
> > >
> > > I like this idea a lot, but I'd like to have a mitigation strategy
> > in
> > > mind if it turns out there is an issue with bloat.  There are some
> > > patterns in cstore that I could imagine causing a problem, but I
> > > haven't had time to test that.
>
> <snip>
>
> The problem is that a fancy scheme to fine tune the allocation
> strategy at run time will be largely self-defeating.  The
> instrumentation and analysis necessary to optimize things will
> incur overhead of its own, leaving us roughly back where we started.

Heh, I would hardly call that scheme fancy! :)

For the sake of argument, the worst case (other than the cache-blowing
"tear it all down" threshold, which I now think is needless) is one
int comparison, one division, one float comparison and an increment.
Division is far from free, but I bet it's cheaper than heap
allocation.  In any case, we /do/ have a way to know for sure.  Would
you mind sharing your test harness that shows the 3.5x speed-up?  I'll
knock it around and see how much improvement we lose with my scheme.

>
> Probably the most reasonable way to avoid memory bloat is the
> simplest.  In jsonObjectFree(), apply the following logic:
>
>     IF length_of_free_list >= free_max
>         return the jsonObject to the heap with free()
>     ELSE
>         sitck it on the free list
>         add 1 to length_of_free_list
>     END IF
>
> ...and likewise decrement length_of_free_list whenever we allocate
> from the free list.
>

I thought the same thing at first too.

> This logic imposes a limit on the amount of unused memory to be
> cached, with a minimum of additional overhead.
>
> The next question is: what value do we use for free_max?
>
> Within reasonable limits it probably doesn't much matter what the
> maximum is, as long as it's big enough to cover us most of the time.
> We could arbitrarily set a maximum of, say, 50.  You should have a
> better idea than I do of what's reasonable.
>

Except that this hurts (or, more correctly, fails to help) us most,
when we are churning through /lots/ of objects in a broad loop (create
20k pile, free that pile, rinse, repeat), which is a reasonably common
pattern (though not normally on the scale of 20k objects).  Saying
"allow a pool of 1k" might work right now, but an new application next
week may be able to make use of 5k or 10k, while most apps may never
need more than 10.

> It might be worthwhile to create a special instrumented version that
> logs allocations and deallocations, or does whatever other analysis
> might be useful.  Link the instrumented version to an application
> of interest, run some typical real-world type loads through it, and
> see what happens.

Absolutely.  That must happen before we decide on a scheme for
limiting the heap object pool no matter what, so I'll add that to the
current patched version and we'll poke at the logs on demo and dev
after a couple days.

>
> We might discover that the free list never gets terribly long under
> realistic conditions, and we can stop worrying about it.
>

I suspect that this may very well be the case for most existing uses,
but we don't know yet.

> It may turn out that different applications have dramatically
> different allocation profiles.  We could easily provide a function
> to set the value of free_max at run time, so that we can apply
> different limits to different applications, or even to different
> parts of the same application.  I'm not sure whether such a function
> should prune any excess off of the free list, or just let it
> disappear by attrition.
>

One thing I'd like to avoid -- no, that we need to avoid -- if pool
size control is needed and we can manage to find a sufficiently
general and non-painful algorithm, is adding another knob.  Tuning
then turns into an art form, and we have enough of that with ejabberd.
 ;)

Again, though, this is all academic right now.  We shall see what the logs say.

> We can dream up still more elaborations, but anything beyond the
> simplest measures would probably be wasted effort.
>
> -------------
>
> The idea of reusing cached chunks of memory is a very general one.
> For example, we could apply it to growing_buffers, or any other
> structure that sees a lot of turnover.  You know better than I
> what the likeliest candidates are.  If we're going to tweak the
> code to limit memory bloat, we should apply the same tweaks
> wherever we cache memory, for the sake of consistency.

I would go along with a whole generalized memory management overhaul
if we get beyond two or three pool types, for sure.

Thanks for the thoughtful consideration Scott.

-- 
Mike Rylander
 | VP, Research and Design
 | Equinox Software, Inc. / The Evergreen Experts
 | phone:  1-877-OPEN-ILS (673-6457)
 | email:  miker at esilibrary.com
 | web:  http://www.esilibrary.com