The Perl Toolchain Summit needs more sponsors. If your company depends on Perl, please support this very important event.
 / 
Search-Tools-0.69
River stage two • 17 direct dependents • 34 total dependents
7 ++
/ Search::Tools::Transliterate

NAME

Search::Tools::Transliterate - transliterations of UTF-8 chars

SYNOPSIS

 my $tr = Search::Tools::Transliterate->new();
 print $tr->convert( 'some string of utf8 chars' );

DESCRIPTION

Search::Tools::Transliterate transliterates UTF-8 characters to single-byte equivalents. It is based on the transmap project by Markus Kuhn http://www.cl.cam.ac.uk/~mgk25/.

NOTE: All the is_* encoding check methods that existed in this class prior to version 0.05 were moved to Search::Tools::UTF8 and refactored as functions, many using XS for speed improvements.

METHODS

new

Create new instance. Takes the following optional parameters:

map

Customize the character mapping. Should be a hashref. See map() method.

ebit

Allow convert() to use full native 8bit characters for transliterating, rather than only 7bit ASCII. The default is true (1). Set to 0 to disable. NOTE: This must be set in new(). Changing via the accessor after new() will have no effect on map().

init

Called internally by new().

map

Access the transliteration character map. Example:

 use Search::Tools::Transliterate;
 my $tr = Search::Tools::Transliterate->new;
 $tr->map->{mychar} = 'my transliteration';
 print $tr->convert('mychar');  # prints 'my transliteration'

NOTE: The map() method is an accessor only. You can not pass in a new map.

convert( text )

Returns UTF-8 text converted with all single bytes, transliterated according to %Map. Will croak if text is not valid UTF-8, so if in doubt, check first with is_valid_utf8() in Search::Tools::UTF8.

convert1252( text )

Returns UTF-8 text converted to all single byte characters, transliterated with convert() and the Windows 1252 characters in the range 0x80 and 0x9f inclusive.

The 1252 codepoints are converted first to their UTF-8 counterparts per http://www.unicode.org/Public/MAPPINGS/VENDORS/MICSFT/WINDOWS/CP1252.TXT using Encoding::FixLatin::fix_latin() and then text is run through convert().

Note that text is checked with the looks_like_cp1252() function from Search::Tools::UTF8 before calling fix_latin().

BUGS

You might consider the whole attempt as a bug. It's really an attempt to accomodate applications that don't support Unicode. Perhaps we shouldn't even try. But for things like curly quotes and other 'smart' punctuation, it's often helpful to render the UTF-8 character as something rather than just letting a character without a direct translation slip into the ether.

That said, if a character has no mapping (and there are plenty that do not) a single space will be used.

AUTHOR

Peter Karman <karman@cpan.org>

Originally based on the HTML::HiLiter regular expression building code, by the same author, copyright 2004 by Cray Inc.

Thanks to Atomic Learning www.atomiclearning.com for sponsoring the development of some of these modules.

BUGS

Please report any bugs or feature requests to bug-search-tools at rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Search-Tools. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.

SUPPORT

You can find documentation for this module with the perldoc command.

    perldoc Search::Tools

You can also look for information at:

COPYRIGHT

Copyright 2006-2010 by Peter Karman.

This package is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

SEE ALSO

Search::Tools::UTF8, Unicode::Map, Encode, Test::utf8, Encoding::FixLatin