[OPEN-ILS-DEV] Indexing problem?

[Bill Ott]

My test database has 360,000+ bib records, and only about 20,000 
holdings attached.  When performing a simple keyword search from the 
staff client I get hundreds or thousands of results in a couple of 
seconds.  If I limit those to items with holdings or those available, in 
either the staff client or opac, it takes about 60 seconds to receive 
any results, if it ever finishes.

I used the import scripts to bring in all bib records, but I rolled my 
own scripts to populate asset.call_number and asset.copy.  Perhaps I 
missed something?

Any thoughts on where to start looking?

Here's the log of a search.


[2008-5-5 11:25:55] -e [INFO:18631:Client.pm:275:] timed_read() read 810 
bytes of data
[2008-5-5 11:25:55] open-ils.search 
[INFO:18686:Application.pm:101:1209998914186690] CALL: 
open-ils.search.biblio.multiclass.query [HASH(0x27023b0), blue, 1]
[2008-5-5 11:25:55] open-ils.search 
[INFO:18686:Biblio.pm:613:1209998914186690] bib search for: 
["default_class","keyword","depth",0,"limit",5000,"offset",0,"org_unit",1,"searches",{"keyword":{"term":"blue"}},"visibility_limit",3000]
[2008-5-5 11:25:55] open-ils.search [INFO:18714:Client.pm:84:] Creating 
Client instance: blade9.grpl.org:5222, osrf, 
open-ils.search_drone_at_blade9.grpl.org
[2008-5-5 11:25:55] open-ils.search 
[INFO:18686:AppSession.pm:501:1209998914186690] AppSession sending 
REQUEST to router at blade9.grpl.org/open-ils.storage with threadTrace [1]
[2008-5-5 11:25:55] open-ils.search [INFO:18714:Client.pm:275:] 
timed_read() read 28 bytes of data
[2008-5-5 11:25:55] -e [INFO:18639:Client.pm:275:] timed_read() read 907 
bytes of data
[2008-5-5 11:25:55] open-ils.storage 
[INFO:18685:Application.pm:101:1209998914186690] CALL: 
open-ils.storage.biblio.multiclass.search_fts.atomic [default_class, 
keyword, limit, 5000, org_unit, 1, visibility_limit, 3000, depth, 0, 
searches, HASH(0x4f74aa0), offset, 0]
[2008-5-5 11:25:55] open-ils.storage [INFO:18715:Client.pm:84:] Creating 
Client instance: blade9.grpl.org:5222, osrf, 
open-ils.storage_drone_at_blade9.grpl.org
[2008-5-5 11:25:55] open-ils.storage [INFO:18715:Client.pm:275:] 
timed_read() read 28 bytes of data
[2008-5-5 11:25:55] open-ils.storage [INFO:18715:Pg.pm:64:] Attempting 
to connect to evergreen at egdb1
[2008-5-5 11:25:55] open-ils.storage [INFO:18715:Pg.pm:94:] Connected to 
MASTER db evergreen at egdb1
[2008-5-5 11:27:5] open-ils.search 
[INFO:18686:Biblio.pm:532:1209998914186690] compiled search is 
{"default_class":"keyword","depth":0,"limit":10,"offset":0,"org_unit":1,"searches":{"keyword":{"term":"blue"}},"visibility_limit":3000}
[2008-5-5 11:27:5] open-ils.search 
[INFO:18686:AppSession.pm:501:1209998914186690] AppSession sending 
RESULT to osrf at blade9.grpl.org/1209998914__1209998914.129312_18669 with 
threadTrace [1]
[2008-5-5 11:27:5] open-ils.search 
[INFO:18686:AppSession.pm:501:1209998914186690] AppSession sending 
STATUS to osrf at blade9.grpl.org/1209998914__1209998914.129312_18669 with 
threadTrace [1]