[OPEN-ILS-GENERAL] Creating a test database from production data

[Brent Mills]

Josh,

Thanks! Good points:

> are you looking to reduce the size of your production database so the backup is smaller to restore for your test system?  Or to reduce the size of your test database instance after it has been restored?


I’m looking for the latter. To reduce the size of the test database instance after it’s been restored. I can restore the full database, but then I’d like to see what I can prune down to keep it as small as possible — and then keep a copy of the reduced-size database dump around.


> I’ve never tried to just delete half the titles and patrons to see if the deletes cascade correctly and leave a working system.  You would probably want to choose titles and patrons randomly also, so you don’t just get rid of old or new records.

This would be great too. We certainly don’t need as many patron records, and there’s really no use for deleted volumes/items/bibs for our situation. I’ll give this a shot as well.

-Brent

-----------------------------

Brent Mills
Systems Librarian | Sage Library System

email: brent at hoodriverlibrary.org
tickets: https://sagelib.org/support
phone: 541.610.8384

> On Sep 1, 2016, at 12:54 PM, Josh Stompro <stomproj at exchange.larl.org> wrote:
> 
> Hello Brent,  are you looking to reduce the size of your production database so the backup is smaller to restore for your test system?  Or to reduce the size of your test database instance after it has been restored?
>  
> You mentioned the auditor tables would be one spot you could trim, that is a big chunk of our DB currently. There are several full copies of the tracked tables of our system in there since they recorded our test loads before migration.  I should really clean that up.
>  
> Also, the actor.usr_activity can be quite large unless you have adjusted the transient field for the user activity types.
>  
> The action_trigger.event and action_trigger.event_output can grow large, if you are not going to be testing event generation you can prune those down.  On a live system you have to be careful about the action_trigger.event data since the data there is consulted when looking for new events to generate in some cases.  I’m not sure if the action_trigger.event_output info matters that much after the fact other than for troubleshooting.
>  
> Any aged circ or holds tables can probably be truncated also.
>  
> I’ve never tried to just delete half the titles and patrons to see if the deletes cascade correctly and leave a working system.  You would probably want to choose titles and patrons randomly also, so you don’t just get rid of old or new records.
>  
> I would go after rows with deleted=true for copies, volumes, titles and patrons and see if those cascade to action.circulation.  I’m going to try it….
>  
> Josh Stompro - LARL IT Director
>  
> From: Open-ils-general [mailto:open-ils-general-bounces at list.georgialibraries.org] On Behalf Of Brent Mills
> Sent: Thursday, September 01, 2016 2:09 PM
> To: Evergreen Discussion Group
> Subject: [OPEN-ILS-GENERAL] Creating a test database from production data
>  
> Hello,
>  
> I’m trying to build up a test server with production data, but we’re a bit tight on drive space at the moment, so decreasing the database’s size any way we can would be great. Plus, it would just be nice to have a smaller dump of some “real-ish” data that we can load into other test environments.
>  
> Any hints/pointers toward things that can be stripped away that might bring the database size down? Things that can be truncated without issue for a non-production environment? Auditor, etc? Just thought I’d see what the hive mind is doing.
>  
> Thanks!
>  
> -Brent
> -----------------------------
> 
> Brent Mills
> Systems Librarian | Sage Library System
> 
> email: brent at hoodriverlibrary.org <mailto:brent at hoodriverlibrary.org>
> tickets: https://sagelib.org/support <https://sagelib.org/support>
> phone: 541.610.8384

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://libmail.georgialibraries.org/pipermail/open-ils-general/attachments/20160901/60f8cf00/attachment.html>