Char::Eutf2 - Run-time routines for Char/


  use Char::Eutf2;


  # "no Char::Eutf2;" not supported


This module is a run-time routines of the Char/ Because the Char/ automatically uses this module, you need not use directly.


I have tested and verified this software using the best of my ability. However, a software containing much regular expression is bound to contain some bugs. Thus, if you happen to find a bug that's in Char::UTF2 software and not your own program, you can try to reduce it to a minimal test case and then report it to the following author's address. If you have an idea that could make this a more useful tool, please let everyone share it.


This Char::Eutf2 module first appeared in ActivePerl Build 522 Built under MSWin32 Compiled at Nov 2 1999 09:52:28


INABA Hitoshi <>

This project was originated by INABA Hitoshi. For any questions, use <> so we can share this file.


This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself. See perlartistic.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


Split string
  @split = Char::Eutf2::split(/pattern/,$string,$limit);
  @split = Char::Eutf2::split(/pattern/,$string);
  @split = Char::Eutf2::split(/pattern/);
  @split = Char::Eutf2::split('',$string,$limit);
  @split = Char::Eutf2::split('',$string);
  @split = Char::Eutf2::split('');
  @split = Char::Eutf2::split();
  @split = Char::Eutf2::split;

  This subroutine scans a string given by $string for separators, and splits the
  string into a list of substring, returning the resulting list value in list
  context or the count of substring in scalar context. Scalar context also causes
  split to write its result to @_, but this usage is deprecated. The separators
  are determined by repeated pattern matching, using the regular expression given
  in /pattern/, so the separators may be of any size and need not be the same
  string on every match. (The separators are not ordinarily returned; exceptions
  are discussed later in this section.) If the /pattern/ doesn't match the string
  at all, Char::Eutf2::split returns the original string as a single substring, If it
  matches once, you get two substrings, and so on. You may supply regular
  expression modifiers to the /pattern/, like /pattern/i, /pattern/x, etc. The
  //m modifier is assumed when you split on the pattern /^/.

  If $limit is specified and positive, the subroutine splits into no more than that
  many fields (though it may split into fewer if it runs out of separators). If
  $limit is negative, it is treated as if an arbitrarily large $limit has been
  specified If $limit is omitted or zero, trailing null fields are stripped from
  the result (which potential users of pop would do wel to remember). If $string
  is omitted, the subroutine splits the $_ string. If /pattern/ is also omitted or
  is the literal space, " ", the subroutine split on whitespace, /\s+/, after
  skipping any leading whitespace.

  A /pattern/ of /^/ is secretly treated if it it were /^/m, since it isn't much
  use otherwise.

  String of any length can be split:

  @chars  = Char::Eutf2::split(//,  $word);
  @fields = Char::Eutf2::split(/:/, $line);
  @words  = Char::Eutf2::split(" ", $paragraph);
  @lines  = Char::Eutf2::split(/^/, $buffer);

  A pattern capable of matching either the null string or something longer than
  the null string (for instance, a pattern consisting of any single character
  modified by a * or ?) will split the value of $string into separate characters
  wherever it matches the null string between characters; nonnull matches will
  skip over the matched separator characters in the usual fashion. (In other words,
  a pattern won't match in one spot more than once, even if it matched with a zero
  width.) For example:

  print join(":" => Char::Eutf2::split(/ */, "hi there"));

  produces the output "h:i:t:h:e:r:e". The space disappers because it matches
  as part of the separator. As a trivial case, the null pattern // simply splits
  into separate characters, and spaces do not disappear. (For normal pattern
  matches, a // pattern would repeat the last successfully matched pattern, but
  Char::Eutf2::split's pattern is exempt from that wrinkle.)

  The $limit parameter splits only part of a string:

  my ($login, $passwd, $remainder) = Char::Eutf2::split(/:/, $_, 3);

  We encourage you to split to lists of names like this to make your code
  self-documenting. (For purposes of error checking, note that $remainder would
  be undefined if there were fewer than three fields.) When assigning to a list,
  if $limit is omitted, Perl supplies a $limit one larger than the number of
  variables in the list, to avoid unneccessary work. For the split above, $limit
  would have been 4 by default, and $remainder would have received only the third
  field, not all the rest of the fields. In time-critical applications, it behooves
  you not to split into more fields than you really need. (The trouble with
  powerful languages it that they let you be powerfully stupid at times.)

  We said earlier that the separators are not returned, but if the /pattern/
  contains parentheses, then the substring matched by each pair of parentheses is
  included in the resulting list, interspersed with the fields that are ordinarily
  returned. Here's a simple example:

  Char::Eutf2::split(/([-,])/, "1-10,20");

  which produces the list value:

  (1, "-", 10, ",", 20)

  With more parentheses, a field is returned for each pair, even if some pairs
  don't match, in which case undefined values are returned in those positions. So
  if you say:

  Char::Eutf2::split(/(-)|(,)/, "1-10,20");

  you get the value:

  (1, "-", undef, 10, undef, ",", 20)

  The /pattern/ argument may be replaced with an expression to specify patterns
  that vary at runtime. As with ordinary patterns, to do run-time compilation only
  once, use /$variable/o.

  As a special case, if the expression is a single space (" "), the subroutine
  splits on whitespace just as Char::Eutf2::split with no arguments does. Thus,
  Char::Eutf2::split(" ") can be used to emulate awk's default behavior. In contrast,
  Char::Eutf2::split(/ /) will give you as many null initial fields as there are
  leading spaces. (Other than this special case, if you supply a string instead
  of a regular expression, it'll be interpreted as a regular expression anyway.)
  You can use this property to remove leading and trailing whitespace from a
  string and to collapse intervaning stretches of whitespace into a single

  $string = join(" ", Char::Eutf2::split(" ", $string));

  The following example splits an RFC822 message header into a hash containing
  $head{'Date'}, $head{'Subject'}, and so on. It uses the trick of assigning a
  list of pairs to a hash, because separators altinate with separated fields, It
  users parentheses to return part of each separator as part of the returned list
  value. Since the split pattern is guaranteed to return things in pairs by virtue
  of containing one set of parentheses, the hash assignment is guaranteed to
  receive a list consisting of key/value pairs, where each key is the name of a
  header field. (Unfortunately, this technique loses information for multiple lines
  with the same key field, such as Received-By lines. Ah well)

  $header =~ s/\n\s+/ /g; # Merge continuation lines.
  %head = ("FRONTSTUFF", Char::Eutf2::split(/^(\S*?):\s*/m, $header));

  The following example processes the entries in a Unix passwd(5) file. You could
  leave out the chomp, in which case $shell would have a newline on the end of it.

  open(PASSWD, "/etc/passwd");
  while (<PASSWD>) {
      chomp; # remove trailing newline.
      ($login, $passwd, $uid, $gid, $gcos, $home, $shell) =

  Here's how process each word of each line of each file of input to create a
  word-frequency hash.

  while (<>) {
      for my $word (Char::Eutf2::split()) {

  The inverse of Char::Eutf2::split is join, except that join can only join with the
  same separator between all fields. To break apart a string with fixed-position
  fields, use unpack.

  Processing long $string (over 32766 octets) requires Perl 5.010001 or later.
  $tr = Char::Eutf2::tr($variable,$bind_operator,$searchlist,$replacementlist,$modifier);
  $tr = Char::Eutf2::tr($variable,$bind_operator,$searchlist,$replacementlist);

  This is the transliteration (sometimes erroneously called translation) operator,
  which is like the y/// operator in the Unix sed program, only better, in
  everybody's humble opinion.

  This subroutine scans a UTF-8 string character by character and replaces all
  occurrences of the characters found in $searchlist with the corresponding character
  in $replacementlist. It returns the number of characters replaced or deleted.
  If no UTF-8 string is specified via =~ operator, the $_ variable is translated.
  $modifier are:

  Modifier   Meaning
  c          Complement $searchlist.
  d          Delete found but unreplaced characters.
  s          Squash duplicate replaced characters.
  r          Return transliteration and leave the original string untouched.

  To use with a read-only value without raising an exception, use the /r modifier.

  print Char::Eutf2::tr('bookkeeper','=~','boep','peob','r'); # prints 'peekkoobor'
Chop string
  $chop = Char::Eutf2::chop(@list);
  $chop = Char::Eutf2::chop();
  $chop = Char::Eutf2::chop;

  This subroutine chops off the last character of a string variable and returns the
  character chopped. The Char::Eutf2::chop subroutine is used primary to remove the newline
  from the end of an input recoed, and it is more efficient than using a
  substitution. If that's all you're doing, then it would be safer to use chomp,
  since Char::Eutf2::chop always shortens the string no matter what's there, and chomp
  is more selective. If no argument is given, the subroutine chops the $_ variable.

  You cannot Char::Eutf2::chop a literal, only a variable. If you Char::Eutf2::chop a list of
  variables, each string in the list is chopped:

  @lines = `cat myfile`;

  You can Char::Eutf2::chop anything that is an lvalue, including an assignment:

  Char::Eutf2::chop($cwd = `pwd`);
  Char::Eutf2::chop($answer = <STDIN>);

  This is different from:

  $answer = Char::Eutf2::chop($tmp = <STDIN>); # WRONG

  which puts a newline into $answer because Char::Eutf2::chop returns the character
  chopped, not the remaining string (which is in $tmp). One way to get the result
  intended here is with substr:

  $answer = substr <STDIN>, 0, -1;

  But this is more commonly written as:

  Char::Eutf2::chop($answer = <STDIN>);

  In the most general case, Char::Eutf2::chop can be expressed using substr:

  $last_code = Char::Eutf2::chop($var);
  $last_code = substr($var, -1, 1, ""); # same thing

  Once you understand this equivalence, you can use it to do bigger chops. To
  Char::Eutf2::chop more than one character, use substr as an lvalue, assigning a null
  string. The following removes the last five characters of $caravan:

  substr($caravan, -5) = '';

  The negative subscript causes substr to count from the end of the string instead
  of the beginning. To save the removed characters, you could use the four-argument
  form of substr, creating something of a quintuple Char::Eutf2::chop;

  $tail = substr($caravan, -5, 5, '');

  This is all dangerous business dealing with characters instead of graphemes. Perl
  doesn't really have a grapheme mode, so you have to deal with them yourself.
Index string
  $byte_pos = Char::Eutf2::index($string,$substr,$byte_offset);
  $byte_pos = Char::Eutf2::index($string,$substr);

  This subroutine searches for one string within another. It returns the byte position
  of the first occurrence of $substring in $string. The $byte_offset, if specified,
  says how many bytes from the start to skip before beginning to look. Positions are
  based at 0. If the substring is not found, the subroutine returns one less than the
  base, ordinarily -1. To work your way through a string, you might say:

  $byte_pos = -1;
  while (($byte_pos = Char::Eutf2::index($string, $lookfor, $byte_pos)) > -1) {
      print "Found at $byte_pos\n";
Reverse index string
  $byte_pos = Char::Eutf2::rindex($string,$substr,$byte_offset);
  $byte_pos = Char::Eutf2::rindex($string,$substr);

  This subroutine works just like Char::Eutf2::index except that it returns the byte
  position of the last occurrence of $substring in $string (a reverse Char::Eutf2::index).
  The subroutine returns -1 if $substring is not found. $byte_offset, if specified,
  is the rightmost byte position that may be returned. To work your way through a
  string backward, say:

  $byte_pos = length($string);
  while (($byte_pos = Char::UTF2::rindex($string, $lookfor, $byte_pos)) >= 0) {
      print "Found at $byte_pos\n";
Lower case string
  $lc = Char::Eutf2::lc($string);
  $lc = Char::Eutf2::lc_;

  This subroutine returns a lowercased version of UTF-8 $string (or $_, if
  $string is omitted). This is the internal subroutine implementing the \L escape
  in double-quoted strings.

  You can use the Char::Eutf2::fc subroutine for case-insensitive comparisons via Char::UTF2
Lower case first character of string
  $lcfirst = Char::Eutf2::lcfirst($string);
  $lcfirst = Char::Eutf2::lcfirst_;

  This subroutine returns a version of UTF-8 $string with the first character
  lowercased (or $_, if $string is omitted). This is the internal subroutine
  implementing the \l escape in double-quoted strings.
Upper case string
  $uc = Char::Eutf2::uc($string);
  $uc = Char::Eutf2::uc_;

  This subroutine returns an uppercased version of UTF-8 $string (or $_, if
  $string is omitted). This is the internal subroutine implementing the \U escape
  in interpolated strings. For titlecase, use Char::Eutf2::ucfirst instead.

  You can use the Char::Eutf2::fc subroutine for case-insensitive comparisons via Char::UTF2
Upper case first character of string
  $ucfirst = Char::Eutf2::ucfirst($string);
  $ucfirst = Char::Eutf2::ucfirst_;

  This subroutine returns a version of UTF-8 $string with the first character
  titlecased and other characters left alone (or $_, if $string is omitted).
  Titlecase is "Camel" for an initial capital that has (or expects to have)
  lowercase characters following it, not uppercase ones. Exsamples are the first
  letter of a sentence, of a person's name, of a newspaper headline, or of most
  words in a title. Characters with no titlecase mapping return the uppercase
  mapping instead. This is the internal subroutine implementing the \u escape in
  double-quoted strings.

  To capitalize a string by mapping its first character to titlecase and the rest
  to lowercase, use:

  $titlecase = Char::Eutf2::ucfirst(substr($word,0,1)) . Char::Eutf2::lc(substr($word,1));


  $string =~ s/(\w)(\w*)/\u$1\L$2/g;

  Do not use:

  $do_not_use = Char::Eutf2::ucfirst(Char::Eutf2::lc($word));

  or "\u\L$word", because that can produce a different and incorrect answer with
  certain characters. The titlecase of something that's been lowercased doesn't
  always produce the same thing titlecasing the original produces.

  Because titlecasing only makes sense at the start of a string that's followed
  by lowercase characters, we can't think of any reason you might want to titlecase
  every character in a string.

  See also P.287 A Case of Mistaken Identity
  in Chapter 6: Unicode
  of ISBN 978-0-596-00492-7 Programming Perl 4th Edition.
Fold case string
  P.860 fc
  in Chapter 27: Functions
  of ISBN 978-0-596-00492-7 Programming Perl 4th Edition.

  $fc = Char::Eutf2::fc($string);
  $fc = Char::Eutf2::fc_;

  New to Char::UTF2 software, this subroutine returns the full Unicode-like casefold of
  UTF-8 $string (or $_, if omitted). This is the internal subroutine implementing
  the \F escape in double-quoted strings.

  Just as title-case is based on uppercase but different, foldcase is based on
  lowercase but different. In ASCII there is a one-to-one mapping between only
  two cases, but in other encoding there is a one-to-many mapping and between three
  cases. Because that's too many combinations to check manually each time, a fourth
  casemap called foldcase was invented as a common intermediary for the other three.
  It is not a case itself, but it is a casemap.

  To compare whether two strings are the same without regard to case, do this:

  Char::Eutf2::fc($a) eq Char::Eutf2::fc($b)

  The reliable way to compare string case-insensitively was with the /i pattern
  modifier, because Char::UTF2 software has always used casefolding semantics for
  case-insensitive pattern matches. Knowing this, you can emulate equality
  comparisons like this:

  sub fc_eq ($$) {
      my($a,$b) = @_;
      return $a =~ /\A\Q$b\E\z/i;
Make ignore case string
  @ignorecase = Char::Eutf2::ignorecase(@string);

  This subroutine is internal use to m/ /i, s/ / /i, split / /i, and qr/ /i.
Make capture number
  $capturenumber = Char::Eutf2::capture($string);

  This subroutine is internal use to m/ /, s/ / /, split / /, and qr/ /.
Make character
  $chr = Char::Eutf2::chr($code);
  $chr = Char::Eutf2::chr_;

  This subroutine returns a programmer-visible character, character represented by
  that $code in the character set. For example, Char::Eutf2::chr(65) is "A" in either
  ASCII or UTF-8, not Unicode. For the reverse of Char::Eutf2::chr, use Char::UTF2::ord.
Filename expansion (globbing)
  @glob = Char::Eutf2::glob($string);
  @glob = Char::Eutf2::glob_;

  This subroutine returns the value of $string with filename expansions the way a
  DOS-like shell would expand them, returning the next successive name on each
  call. If $string is omitted, $_ is globbed instead. This is the internal
  subroutine implementing the <*> and glob operator.
  This subroutine function when the pathname ends with chr(0x5C) on MSWin32.

  For ease of use, the algorithm matches the DOS-like shell's style of expansion,
  not the UNIX-like shell's. An asterisk ("*") matches any sequence of any
  character (including none). A question mark ("?") matches any one character or
  none. A tilde ("~") expands to a home directory, as in "~/.*rc" for all the
  current user's "rc" files, or "~jane/Mail/*" for all of Jane's mail files.

  Note that all path components are case-insensitive, and that backslashes and
  forward slashes are both accepted, and preserved. You may have to double the
  backslashes if you are putting them in literally, due to double-quotish parsing
  of the pattern by perl.

  The Char::Eutf2::glob subroutine grandfathers the use of whitespace to separate multiple
  patterns such as <*.c *.h>. If you want to glob filenames that might contain
  whitespace, you'll have to use extra quotes around the spacy filename to protect
  it. For example, to glob filenames that have an "e" followed by a space followed
  by an "f", use either of:

  @spacies = <"*e f*">;
  @spacies = Char::Eutf2::glob('"*e f*"');
  @spacies = Char::Eutf2::glob(q("*e f*"));

  If you had to get a variable through, you could do this:

  @spacies = Char::Eutf2::glob("'*${var}e f*'");
  @spacies = Char::Eutf2::glob(qq("*${var}e f*"));

  Another way on MSWin32

  # relative path
  @relpath_file = split(/\n/,`dir /b wildcard\\here*.txt 2>NUL`);

  # absolute path
  @abspath_file = split(/\n/,`dir /s /b wildcard\\here*.txt 2>NUL`);

  @relpath_file = split(/\n/,`dir /b wildcard\\here*.txt`);
  @abspath_file = split(/\n/,`dir /s /b wildcard\\here*.txt`);

1 POD Error

The following errors were encountered while parsing the POD:

Around line 4181:

=over without closing =back