[OPEN-ILS-DEV] Correction to md5sum function

[Scott McKellar]

In trunk I just committed a bug fix for the md5sum function.  There is a small chance that this fix will lead to some weird problems, particularly with authentication, in somebody's testbed (but not in production).  At the risk of telling you more than you really want to know, I'm giving a heads-up.

Before the fix, the code in question used the osrf_clearbuf macro to initialize a buffer.  This macro compiles differently depending on whether the NDEBUG macro is defined.  For production builds, it turns into a call to memset() that fills the buffer with binary zeros.  For debugging builds, it fills the buffer with exclamation points, plus a terminal nul.

The purpose of the clear_buf macro is to make certain kinds of buffer abuse easier to spot.  In this case the macro does double duty: it not only makes it easier to spot the buffer abuse, but also commits the abuse in the first place.  The buffer needs to start out as an empty string.

In most cases I could quietly fix a bug and no one need know about it.  In this case the fix may, in some cases, create the appearance (but not the reality) of a problem.

Typically when you do an md5 sum, it's because you want to do it twice, and compare the results to be sure that both versions of the input data were the same.

If you do one of those md5 sums using the version that was compiled in debug mode, and the using the version that was compiled in production mode, the bug would guarantee that you wouldn't get the same answer.

(In fact that's how I stumbled across the problem.  My test version of srfsh couldn't log in, because the authentication server had been compiled in production mode.  It took me a while to figure out what was going wrong.)

Result: this new version of mdfsum() will be incompatible with the old version if the old version was compiled with debugging turned on.  It's unlikely that anyone else will encounter the same circumstance, but it's not impossible.  You have been warned.

Scott McKellar