[OPEN-ILS-DEV] Regarding Marc::Charset

[Mike Rylander]

On 6/21/07, Don Hamilton <dhamilton at wlu.ca> wrote:
>
>
> Thanks, Mike. I'm stuck at :
>
> root at tamarak:~/Desktop/evergreen/Open-ILS/src/extras/marcdumper#
> make
> gcc marcdumper.c -O2 -I /usr/include/libxml2/ -I /usr/include/yaz -lxml2
> -lyaz  -o marcdumper
> marcdumper.c:31:26: error: yaz/marcdisp.h: No such file or directory
> marcdumper.c:32:26: error: yaz/yaz-util.h: No such file or directory
> marcdumper.c:33:25: error: yaz/xmalloc.h: No such file or directory
> marcdumper.c:34:25: error: yaz/options.h: No such file or directory

[snip]

>
> so I assume I need to install yaz... which I'll have to leave for someone
> else (rene!) tomorrow...

libyaz2-dev, actually (assuming debian or ubuntu).

If you're feeling saucy, as root you can do

 # apt-get install libyaz2-dev

which will pull in libyaz2 and (I believe) libwrap0.  Then just make again.

>
> I will let you know what happens.
>

Thanks!

> don
>
> >>> mrylander at gmail.com 6/21/2007 3:49 PM >>>
>
> On 6/21/07, Don Hamilton <dhamilton at wlu.ca> wrote:
> > Hi Guys.
> >
> > I'm revisting an issue I looked at briefly a couple of weeks ago.
> > Marc::Charset is spewing out lots of "no mapping found for [0x??] at
> > position ?? in ???sometext??? / g0=ASCII_DEFAULT g1=EXTENDED_LATIN at
> > /usr/local/share/perl/5.8.8/MARC/Charset.pm line 210."
> > messages.
> >
> > I THINK this is because I already have Unicode Marc records (based on my
> > last Endeavor/ExLibris/Voyager upgrade) so I don't really want, or need,
> to
> > use Charset at all. I tried several variations in marc2bre of setting $enc
> > to 'utf8', and ignore errors to 0 or 1, but the conversion errors just
> keep
> > on a comin'...
> >
> > How might I stop this from happening?
>
> If your records are truly UTF-8 encoded (and properly marked as such)
> then you'll never see those errors.  So, assuming that the records are
> UTF-8 encoded, the problem is that the leader is incorrectly marked as
> MARC-8.
>
> So, lets see if we can convert your records to MARCXML with
> marcdumper, and then do the import dance (it should be much faster for
> the marc2bre part).
>
> In the current SVN trunk you'll find mardumper in
>
> Open-ILS/src/extras/marcdumper/
>
> This is a modified version of IndexData's utility.  We've extended it
> to accept a -r paramter, which when outputing XML allows you to strip
> out data using XPath.
>
> You'll need to run `make` in that directory to compile the binary.
> Once you have that, use the following command to convert your MARC21
> to MARCXML:
>
>   ./marcdumper -f UTF-8 -t UTF-8 -X /your/marc21/file.mrc >
> /tmp/marc-output.xml
>
>
> This will force the leader into the correct state.  Now you can give
> /tmp/marc-output.xml to marc2bre, along with --marctype=XML, and your
> records should come in fine.  If, however, you find your records
> bombing out either during the marc2bre or ingest phases, you'll
> probably want to run marcdumper again with a -f of MARC8.
>
> Let us know how that goes.
>
> TIA,
>
> --miker
>


-- 
Mike Rylander