[open-ils-commits] r987 - in conifer/branches/rel_1_6_1/src/perlmods/OpenILS/Application: . Storage/Driver/Pg (dbs)

[svn at svn.open-ils.org]

Author: dbs
Date: 2010-09-13 10:30:10 -0400 (Mon, 13 Sep 2010)
New Revision: 987

Modified:
   conifer/branches/rel_1_6_1/src/perlmods/OpenILS/Application/Ingest.pm
   conifer/branches/rel_1_6_1/src/perlmods/OpenILS/Application/Storage/Driver/Pg/fts.pm
Log:
Add indexing normalization for Polish l (?\197?\130)


Modified: conifer/branches/rel_1_6_1/src/perlmods/OpenILS/Application/Ingest.pm
===================================================================
--- conifer/branches/rel_1_6_1/src/perlmods/OpenILS/Application/Ingest.pm	2010-09-13 14:27:59 UTC (rev 986)
+++ conifer/branches/rel_1_6_1/src/perlmods/OpenILS/Application/Ingest.pm	2010-09-13 14:30:10 UTC (rev 987)
@@ -900,6 +900,9 @@
             $value =~ s/\pC+//sgo;
             $value =~ s/\W+$//sgo;
 
+            $term =~ s/\x{142}/l/sgo; # Convert Polish l (lowercase)
+            $term =~ s/\x{141}/l/sgo; # Convert Polish l (uppercase)
+
             # hack to normalize ratio-like strings
             while ($term =~ /\b\d{1}:[, ]?\d+(?:[ ,]\d+[^:])+/o) {
                 $term = $` . join ('', split(/[, ]/, $&)) . $';

Modified: conifer/branches/rel_1_6_1/src/perlmods/OpenILS/Application/Storage/Driver/Pg/fts.pm
===================================================================
--- conifer/branches/rel_1_6_1/src/perlmods/OpenILS/Application/Storage/Driver/Pg/fts.pm	2010-09-13 14:27:59 UTC (rev 986)
+++ conifer/branches/rel_1_6_1/src/perlmods/OpenILS/Application/Storage/Driver/Pg/fts.pm	2010-09-13 14:30:10 UTC (rev 987)
@@ -24,6 +24,9 @@
 		$term =~ s/(\pM+)//gos;
 		$term =~ s/(\b\.\b)//gos;
 
+		$term =~ s/\x{142}/l/sgo; # Convert Polish l (lowercase)
+		$term =~ s/\x{141}/l/sgo; # Convert Polish l (uppercase)
+
 		# hack to normalize ratio-like strings
 		while ($term =~ /\b\d{1}:[, ]?\d+(?:[ ,]\d+[^:])+/o) {
 			$term = $` . join ('', split(/[, ]/, $&)) . $';