++ed by:
1 non-PAUSE user
Author image INABA Hitoshi


EUCJP - Source code filter to escape EUC-JP script

Install and Usage

There are two steps there:

  • You'll have to download EUCJP.pm and Eeucjp.pm and put it in your perl lib directory.

  • You'll need to write "use EUCJP;" at head of the script.


  use EUCJP;
  use EUCJP ver.sion;             --- require minimum version
  use EUCJP ver.sion.0;           --- expects version (match or die)
  use EUCJP qw(ord reverse getc); --- demand enhanced feature of ord, reverse, and getc
  use EUCJP ver.sion qw(ord reverse getc);
  use EUCJP ver.sion.0 qw(ord reverse getc);

  # "no EUCJP;" not supported


  $ perl EUCJP.pm EUC-JP_script.pl > Escaped_script.pl.e


  $ perl Escaped_script.pl.e

  EUC-JP_script.pl  --- script written in EUC-JP
  Escaped_script.pl.e --- escaped script

  dummy functions:


EUCJP software is "middleware" between perl interpreter and your Perl script written in EUC-JP.

Perl is optimized for problems which are about 90% working with text and about 10% everything else. Even if this "text" doesn't contain EUC-JP, Perl3 or later can treat EUC-JP as binary data.

By "use EUCJP;", it automatically interpret your script as EUC-JP. The various functions of perl including a regular expression can treat EUC-JP now. The function length treats length per byte. This software does not use UTF8 flag.

Yet Another Future Of

JPerl is very useful software. -- Oops, note, this "JPerl" means "Japanized Perl" or "Japanese Perl". Therefore, it is unrelated to JPerl of the following.

 JPerl is an implementation of Perl written in Java.
 jPerl - Perl on the JVM
 Jamie's PERL scripts for bioinformatics
 jperl (Jonathan Perl)

Now, the last version of JPerl is 5.005_04 and is not maintained now.

Japanization modifier WATANABE Hirofumi said,

  "Because WATANABE am tired I give over maintaing JPerl."

at Slide #15: "The future of JPerl" of


in The Perl Confernce Japan 1998.

When I heard it, I thought that someone excluding me would maintain JPerl. And I slept every night hanging a sock. Night and day, I kept having hope. After 10 years, I noticed that white beard exists in the sock :-)

This software is a source code filter to escape Perl script encoded by EUC-JP given from STDIN or command line parameter. The character code is never converted by escaping the script. Neither the value of the character nor the length of the character string change even if it escapes.

I learned the following things from the successful software.

  • Upper Compatibility like Perl4 to Perl5

  • Maximum Portability like jcode.pl

  • Remains One Language Handling Raw EUC-JP, Doesn't Use UTF8 flag like JPerl

  • Remains One Interpreter like Encode module

  • Code Set Independent like Ruby

  • Monolithic Script like cpanminus

  • There's more than one way to do it like Perl itself

I am excited about this software and Perl's future --- I hope you are too.

JRE: JPerl Runtime Environment

  |        JPerl Application Script       | Your Script
  |  Source Code Filter, Runtime Routine  | ex. EUCJP.pm, Eeucjp.pm
  |          PVM 5.00503 or later         | ex. perl 5.00503

A Perl Virtual Machine (PVM) enables a set of computer software programs and data structures to use a virtual machine model for the execution of other computer programs and scripts. The model used by a PVM accepts a form of computer intermediate language commonly referred to as Perl byteorientedcode. This language conceptually represents the instruction set of a byte-oriented, capability architecture.

Basic Idea of Source Code Filter

I discovered this mail again recently.

[Tokyo.pm] jus Benkyoukai


save as: SJIS.pm

  package SJIS;
  use Filter::Util::Call;
  sub multibyte_filter {
      my $status;
      if (($status = filter_read()) > 0 ) {
  sub import {

I am glad that I could confirm my idea is not so wrong.

Command-line Wildcard Expansion on DOS-like Systems

The default command shells on DOS-like systems (COMMAND.COM or cmd.exe or Win95Cmd.exe) do not expand wildcard arguments supplied to programs. Instead, import of Eeucjp.pm works well.

   in Eeucjp.pm
   # @ARGV wildcard globbing
   sub import {

       if ($^O =~ /\A (?: MSWin32 | NetWare | symbian | dos ) \z/oxms) {
           my @argv = ();
           for (@ARGV) {

               # has space
               if (/\A (?:$q_char)*? [ ] /oxms) {
                   if (my @glob = Eeucjp::glob(qq{"$_"})) {
                       push @argv, @glob;
                   else {
                       push @argv, $_;

               # has wildcard metachar
               elsif (/\A (?:$q_char)*? [*?] /oxms) {
                   if (my @glob = Eeucjp::glob($_)) {
                       push @argv, @glob;
                   else {
                       push @argv, $_;

               # no wildcard globbing
               else {
                   push @argv, $_;
           @ARGV = @argv;

Software Composition

   EUCJP.pm               --- source code filter to escape EUC-JP
   Eeucjp.pm              --- run-time routines for EUCJP.pm

Upper Compatibility by Escaping

This software adds the function by 'Escaping' it always, and nothing of the past is broken. Therefore, 'Possible job' never becomes 'Impossible job'. This approach is effective in the field where the retreat is never permitted. It means incompatible upgrade of Perl should be rewound.

Escaping Your Script (You do)

You need write 'use EUCJP;' in your script.

  You do
  use EUCJP;

Multiple-Octet Anchoring of Regular Expression (EUCJP software provides)

EUCJP software applies multiple-octet anchoring at beginning of regular expression.

  Before                  After
  m/regexp/               m/${Eeucjp::anchor}(?:regexp).../

Multiple-Octet Character Regular Expression (EUCJP software provides)

EUCJP software clusters multiple-octet character with quantifier, makes cluster from multiple-octet custom character classes. And makes multiple-octet version metasymbol from classic Perl character class shortcuts and POSIX-style character classes.

  Before                  After
  m/...MULTIOCT+.../      m/...(?:MULTIOCT)+.../
  m/...[AN-EM].../        m/...(?:A[N-Z]|[B-D][A-Z]|E[A-M]).../
  m/...\D.../             m/...(?:${Eeucjp::eD}).../
  m/...[[:^digit:]].../   m/...(?:${Eeucjp::not_digit}).../

Calling 'Eeucjp::ignorecase()' (EUCJP software provides)

EUCJP software applies calling 'Eeucjp::ignorecase()' instead of /i modifier.

  Before                  After
  m/...$var.../i          m/...@{[Eeucjp::ignorecase($var)]}.../

Character-Oriented Regular Expression

Regular expression works as character-oriented that has no /b modifier.

  Before                  After
  /regexp/                /ditto$Eeucjp::matched/
  m/regexp/               m/ditto$Eeucjp::matched/
  ?regexp?                m?ditto$Eeucjp::matched?
  m?regexp?               m?ditto$Eeucjp::matched?
  $_ =~                   ($_ =~ m/ditto$Eeucjp::matched/) ?
  s/regexp/replacement/   CORE::eval{ Eeucjp::s_matched(); local $^W=0; my $__r=qq/replacement/; $_="${1}$__r$'"; 1 } :
  $_ !~                   ($_ !~ m/ditto$Eeucjp::matched/) ?
  s/regexp/replacement/   1 :
                          CORE::eval{ Eeucjp::s_matched(); local $^W=0; my $__r=qq/replacement/; $_="${1}$__r$'"; undef }
  split(/regexp/)         Eeucjp::split(qr/regexp/)
  split(m/regexp/)        Eeucjp::split(qr/regexp/)
  split(qr/regexp/)       Eeucjp::split(qr/regexp/)
  qr/regexp/              qr/ditto$Eeucjp::matched/

Byte-Oriented Regular Expression

Regular expression works as byte-oriented that has /b modifier.

  Before                  After
  /regexp/b               /(?:regexp)$Eeucjp::matched/
  m/regexp/b              m/(?:regexp)$Eeucjp::matched/
  ?regexp?b               m?regexp$Eeucjp::matched?
  m?regexp?b              m?regexp$Eeucjp::matched?
  $_ =~                   ($_ =~ m/(\G[\x00-\xFF]*?)(?:regexp)$Eeucjp::matched/) ?
  s/regexp/replacement/b  CORE::eval{ Eeucjp::s_matched(); local $^W=0; my $__r=qq/replacement/; $_="${1}$__r$'"; 1 } :
  $_ !~                   ($_ !~ m/(\G[\x00-\xFF]*?)(?:regexp)$Eeucjp::matched/) ?
  s/regexp/replacement/b  1 :
                          CORE::eval{ Eeucjp::s_matched(); local $^W=0; my $__r=qq/replacement/; $_="${1}$__r$'"; undef }
  split(/regexp/b)        split(qr/regexp/)
  split(m/regexp/b)       split(qr/regexp/)
  split(qr/regexp/b)      split(qr/regexp/)
  qr/regexp/b             qr/(?:regexp)$Eeucjp::matched/

Escaping Character Classes (Eeucjp.pm provides)

The character classes are redefined as follows to backward compatibility.

  Before        After
   .            ${Eeucjp::dot}
                ${Eeucjp::dot_s}    (/s modifier)
  \d            [0-9]              (universally)
  \s            \s
  \w            [0-9A-Z_a-z]       (universally)
  \D            ${Eeucjp::eD}
  \S            ${Eeucjp::eS}
  \W            ${Eeucjp::eW}
  \h            [\x09\x20]
  \v            [\x0A\x0B\x0C\x0D]
  \H            ${Eeucjp::eH}
  \V            ${Eeucjp::eV}
  \C            [\x00-\xFF]
  \X            X                  (so, just 'X')
  \R            ${Eeucjp::eR}
  \N            ${Eeucjp::eN}

Also POSIX-style character classes.

  Before        After
  [:alnum:]     [\x30-\x39\x41-\x5A\x61-\x7A]
  [:alpha:]     [\x41-\x5A\x61-\x7A]
  [:ascii:]     [\x00-\x7F]
  [:blank:]     [\x09\x20]
  [:cntrl:]     [\x00-\x1F\x7F]
  [:digit:]     [\x30-\x39]
  [:graph:]     [\x21-\x7F]
  [:lower:]     [\x61-\x7A]
                [\x41-\x5A\x61-\x7A]     (/i modifier)
  [:print:]     [\x20-\x7F]
  [:punct:]     [\x21-\x2F\x3A-\x3F\x40\x5B-\x5F\x60\x7B-\x7E]
  [:space:]     [\s\x0B]
  [:upper:]     [\x41-\x5A]
                [\x41-\x5A\x61-\x7A]     (/i modifier)
  [:word:]      [\x30-\x39\x41-\x5A\x5F\x61-\x7A]
  [:xdigit:]    [\x30-\x39\x41-\x46\x61-\x66]
  [:^alnum:]    ${Eeucjp::not_alnum}
  [:^alpha:]    ${Eeucjp::not_alpha}
  [:^ascii:]    ${Eeucjp::not_ascii}
  [:^blank:]    ${Eeucjp::not_blank}
  [:^cntrl:]    ${Eeucjp::not_cntrl}
  [:^digit:]    ${Eeucjp::not_digit}
  [:^graph:]    ${Eeucjp::not_graph}
  [:^lower:]    ${Eeucjp::not_lower}
                ${Eeucjp::not_lower_i}    (/i modifier)
  [:^print:]    ${Eeucjp::not_print}
  [:^punct:]    ${Eeucjp::not_punct}
  [:^space:]    ${Eeucjp::not_space}
  [:^upper:]    ${Eeucjp::not_upper}
                ${Eeucjp::not_upper_i}    (/i modifier)
  [:^word:]     ${Eeucjp::not_word}
  [:^xdigit:]   ${Eeucjp::not_xdigit}

\b and \B are redefined as follows to backward compatibility.

  Before      After
  \b          ${Eeucjp::eb}
  \B          ${Eeucjp::eB}

Definitions in Eeucjp.pm.

  After                    Definition
  ${Eeucjp::anchor}         qr{\G(?>[^\x8E\x8F\xA1-\xFE]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])*?}
                           for over 32766 octets string on ActivePerl5.6 and Perl5.10 or later
                           for over 32766 octets string on ActivePerl5.6 and Perl5.10 or later
  ${Eeucjp::dot}            qr{(?>[^\x8E\x8F\xA1-\xFE\x0A]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};
  ${Eeucjp::dot_s}          qr{(?>[^\x8E\x8F\xA1-\xFE]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};
  ${Eeucjp::eD}             qr{(?>[^\x8E\x8F\xA1-\xFE0-9]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};
  ${Eeucjp::eS}             qr{(?>[^\x8E\x8F\xA1-\xFE\s]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};
  ${Eeucjp::eW}             qr{(?>[^\x8E\x8F\xA1-\xFE0-9A-Z_a-z]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};
  ${Eeucjp::eH}             qr{(?>[^\x8E\x8F\xA1-\xFE\x09\x20]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};
  ${Eeucjp::eV}             qr{(?>[^\x8E\x8F\xA1-\xFE\x0A\x0B\x0C\x0D]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};
  ${Eeucjp::eR}             qr{(?>\x0D\x0A|[\x0A\x0D])};
  ${Eeucjp::eN}             qr{(?>[^\x8E\x8F\xA1-\xFE\x0A]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};
  ${Eeucjp::not_alnum}      qr{(?>[^\x8E\x8F\xA1-\xFE\x30-\x39\x41-\x5A\x61-\x7A]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};
  ${Eeucjp::not_alpha}      qr{(?>[^\x8E\x8F\xA1-\xFE\x41-\x5A\x61-\x7A]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};
  ${Eeucjp::not_ascii}      qr{(?>[^\x8E\x8F\xA1-\xFE\x00-\x7F]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};
  ${Eeucjp::not_blank}      qr{(?>[^\x8E\x8F\xA1-\xFE\x09\x20]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};
  ${Eeucjp::not_cntrl}      qr{(?>[^\x8E\x8F\xA1-\xFE\x00-\x1F\x7F]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};
  ${Eeucjp::not_digit}      qr{(?>[^\x8E\x8F\xA1-\xFE\x30-\x39]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};
  ${Eeucjp::not_graph}      qr{(?>[^\x8E\x8F\xA1-\xFE\x21-\x7F]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};
  ${Eeucjp::not_lower}      qr{(?>[^\x8E\x8F\xA1-\xFE\x61-\x7A]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};
  ${Eeucjp::not_lower_i}    qr{(?>[^\x8E\x8F\xA1-\xFE\x41-\x5A\x61-\x7A]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])}; # Perl 5.16 compatible
# ${Eeucjp::not_lower_i}    qr{(?>[^\x8E\x8F\xA1-\xFE]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};                   # older Perl compatible
  ${Eeucjp::not_print}      qr{(?>[^\x8E\x8F\xA1-\xFE\x20-\x7F]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};
  ${Eeucjp::not_punct}      qr{(?>[^\x8E\x8F\xA1-\xFE\x21-\x2F\x3A-\x3F\x40\x5B-\x5F\x60\x7B-\x7E]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};
  ${Eeucjp::not_space}      qr{(?>[^\x8E\x8F\xA1-\xFE\s\x0B]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};
  ${Eeucjp::not_upper}      qr{(?>[^\x8E\x8F\xA1-\xFE\x41-\x5A]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};
  ${Eeucjp::not_upper_i}    qr{(?>[^\x8E\x8F\xA1-\xFE\x41-\x5A\x61-\x7A]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])}; # Perl 5.16 compatible
# ${Eeucjp::not_upper_i}    qr{(?>[^\x8E\x8F\xA1-\xFE]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};                   # older Perl compatible
  ${Eeucjp::not_word}       qr{(?>[^\x8E\x8F\xA1-\xFE\x30-\x39\x41-\x5A\x5F\x61-\x7A]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};
  ${Eeucjp::not_xdigit}     qr{(?>[^\x8E\x8F\xA1-\xFE\x30-\x39\x41-\x46\x61-\x66]|[\x8E\xA1-\xFE][\xA1-\xFE]|\x8F[\xA1-\xFE][\x00-\xFF])};
  # This solution is not perfect. I beg better solution from you who are reading this.
  ${Eeucjp::eb}             qr{(?:\A(?=[0-9A-Z_a-z])|(?<=[\x00-\x2F\x40\x5B-\x5E\x60\x7B-\xFF])(?=[0-9A-Z_a-z])|(?<=[0-9A-Z_a-z])(?=[\x00-\x2F\x40\x5B-\x5E\x60\x7B-\xFF]|\z))};
  ${Eeucjp::eB}             qr{(?:(?<=[0-9A-Z_a-z])(?=[0-9A-Z_a-z])|(?<=[\x00-\x2F\x40\x5B-\x5E\x60\x7B-\xFF])(?=[\x00-\x2F\x40\x5B-\x5E\x60\x7B-\xFF]))};

Un-Escaping \ of \b{}, \B{}, \N{}, \p{}, \P{}, and \X (EUCJP software provides)

EUCJP software removes '\' at head of alphanumeric regexp metasymbols \b{}, \B{}, \N{}, \p{}, \P{} and \X. By this method, you can avoid the trap of the abstraction.

See also, Deprecate literal unescaped "{" in regexes. http://perl5.git.perl.org/perl.git/commit/2a53d3314d380af5ab5283758219417c6dfa36e9

  Before           After
  \b{...}          b\{...}
  \B{...}          B\{...}
  \p{L}            p\{L}
  \p{^L}           p\{^L}
  \p{\^L}          p\{\^L}
  \pL              pL
  \P{L}            P\{L}
  \P{^L}           P\{^L}
  \P{\^L}          P\{\^L}
  \PL              PL
  \X               X

Escaping Built-in Functions (EUCJP software provides)

Insert 'Eeucjp::' at head of function name. Eeucjp.pm provides your script Eeucjp::* subroutines.

  Before      After            Works as
  length      length           Byte
  substr      substr           Byte
  pos         pos              Byte
  split       Eeucjp::split     Character
  tr///       Eeucjp::tr        Character
  tr///b      tr///            Byte
  tr///B      tr///            Byte
  y///        Eeucjp::tr        Character
  y///b       tr///            Byte
  y///B       tr///            Byte
  chop        Eeucjp::chop      Character
  index       Eeucjp::index     Character
  rindex      Eeucjp::rindex    Character
  lc          Eeucjp::lc        Character
  lcfirst     Eeucjp::lcfirst   Character
  uc          Eeucjp::uc        Character
  ucfirst     Eeucjp::ucfirst   Character
  fc          Eeucjp::fc        Character
  chr         Eeucjp::chr       Character
  glob        Eeucjp::glob      Character

  Before                   After
  use Perl::Module;        BEGIN { require 'Perl/Module.pm'; Perl::Module->import() if Perl::Module->can('import'); }
  use Perl::Module @list;  BEGIN { require 'Perl/Module.pm'; Perl::Module->import(@list) if Perl::Module->can('import'); }
  use Perl::Module ();     BEGIN { require 'Perl/Module.pm'; }
  no Perl::Module;         BEGIN { require 'Perl/Module.pm'; Perl::Module->unimport() if Perl::Module->can('unimport'); }
  no Perl::Module @list;   BEGIN { require 'Perl/Module.pm'; Perl::Module->unimport(@list) if Perl::Module->can('unimport'); }
  no Perl::Module ();      BEGIN { require 'Perl/Module.pm'; }

Escaping Function Name (You do)

You need write 'EUCJP::' at head of function name when you want character- oriented subroutine. See 'Character-Oriented Subroutines'.

  Function   Character-Oriented   Description
  ord        EUCJP::ord
  reverse    EUCJP::reverse
  getc       EUCJP::getc
  length     EUCJP::length
  substr     EUCJP::substr
  index      EUCJP::index          See 'About Indexes'
  rindex     EUCJP::rindex         See 'About Rindexes'
  eval       EUCJP::eval

  About Indexes
  Function       Works as    Returns as   Description
  index          Character   Byte         JPerl semantics (most useful)
  (same as Eeucjp::index)
  EUCJP::index    Character   Character    Character-oriented semantics
  CORE::index    Byte        Byte         Byte-oriented semantics
  (nothing)      Byte        Character    (most useless)

  About Rindexes
  Function       Works as    Returns as   Description
  rindex         Character   Byte         JPerl semantics (most useful)
  (same as Eeucjp::rindex)
  EUCJP::rindex   Character   Character    Character-oriented semantics
  CORE::rindex   Byte        Byte         Byte-oriented semantics
  (nothing)      Byte        Character    (most useless)

Character-Oriented Subsroutines

  • Ordinal Value of Character

      $ord = EUCJP::ord($string);
      This subroutine returns the numeric value (ASCII or EUC-JP character) of the
      first character of $string, not Unicode. If $string is omitted, it uses $_.
      The return value is always unsigned.
      If you import ord "use EUCJP qw(ord);", ord of your script will be rewritten in
      EUCJP::ord. EUCJP::ord is not compatible with ord of JPerl.
  • Reverse List or String

      @reverse = EUCJP::reverse(@list);
      $reverse = EUCJP::reverse(@list);
      In list context, this subroutine returns a list value consisting of the elements
      of @list in the opposite order.
      In scalar context, the subroutine concatenates all the elements of @list and
      then returns the reverse of that resulting string, character by character.
      If you import reverse "use EUCJP qw(reverse);", reverse of your script will be
      rewritten in EUCJP::reverse. EUCJP::reverse is not compatible with reverse of
      Even if you do not know this subroutine, there is no problem. This subroutine
      can be created with
      $rev = join('', reverse(split(//, $jstring)));
      as before.
      P.558 JPerl (Japanese Perl)
      Appendix C Supplement the Japanese version
      ISBN 4-89052-384-7 PERL PUROGURAMINGU
  • Returns Next Character

      $getc = EUCJP::getc(FILEHANDLE);
      $getc = EUCJP::getc($filehandle);
      $getc = EUCJP::getc;
      This subroutine returns the next character from the input file attached to
      FILEHANDLE. It returns undef at end-of-file, or if an I/O error was encountered.
      If FILEHANDLE is omitted, the subroutine reads from STDIN.
      This subroutine is somewhat slow, but it's occasionally useful for
      single-character input from the keyboard -- provided you manage to get your
      keyboard input unbuffered. This subroutine requests unbuffered input from the
      standard I/O library. Unfortunately, the standard I/O library is not so standard
      as to provide a portable way to tell the underlying operating system to supply
      unbuffered keyboard input to the standard I/O system. To do that, you have to
      be slightly more clever, and in an operating-system-dependent fashion. Under
      Unix you might say this:
      if ($BSD_STYLE) {
          system "stty cbreak </dev/tty >/dev/tty 2>&1";
      else {
          system "stty", "-icanon", "eol", "\001";
      $key = EUCJP::getc;
      if ($BSD_STYLE) {
          system "stty -cbreak </dev/tty >/dev/tty 2>&1";
      else {
          system "stty", "icanon", "eol", "^@"; # ASCII NUL
      print "\n";
      This code puts the next character typed on the terminal in the string $key. If
      your stty program has options like cbreak, you'll need to use the code where
      $BSD_STYLE is true. Otherwise, you'll need to use the code where it is false.
      If you import getc "use EUCJP qw(getc);", getc of your script will be rewritten
      in EUCJP::getc. EUCJP::getc is not compatible with getc of JPerl.
  • Length by EUC-JP Character

      $length = EUCJP::length($string);
      $length = EUCJP::length();
      This subroutine returns the length in characters (programmer-visible characters)
      of the scalar value $string. If $string is omitted, it returns the EUCJP::length
      of $_.
      Do not try to use EUCJP::length to find the size of an array or hash. Use scalar
      @array for the size of an array, and scalar keys %hash for the number of key/value
      pairs in a hash. (The scalar is typically omitted when redundant.)
      To find the length of a string in bytes rather than characters, say simply:
      $bytes = length($string);
      Even if you do not know this subroutine, there is no problem. This subroutine
      can be created with
      $len = split(//, $jstring);
      as before.
      P.558 JPerl (Japanese Perl)
      Appendix C Supplement the Japanese version
      ISBN 4-89052-384-7 PERL PUROGURAMINGU
  • Substr by EUC-JP Character

      $substr = EUCJP::substr($string,$offset,$length,$replacement);
      $substr = EUCJP::substr($string,$offset,$length);
      $substr = EUCJP::substr($string,$offset);
      This subroutine extracts a substring out of the string given by $string and returns
      it. The substring is extracted starting at $offset characters from the front of
      the string. First character is at offset zero. If $offset is negative, starts that
      far back from the end of the string.
      If $length is omitted, returns everything through the end of the string. If $length
      is negative, leaves that many characters off the end of the string. Otherwise,
      $length indicates the length of the substring to extract, which is sort of what
      you'd expect.
      my $s = "The black cat climbed the green tree";
      my $color  = EUCJP::substr $s, 4, 5;      # black
      my $middle = EUCJP::substr $s, 4, -11;    # black cat climbed the
      my $end    = EUCJP::substr $s, 14;        # climbed the green tree
      my $tail   = EUCJP::substr $s, -4;        # tree
      my $z      = EUCJP::substr $s, -4, 2;     # tr
      If Perl version 5.14 or later, you can use the EUCJP::substr() subroutine as an
      lvalue. In its case $string must itself be an lvalue. If you assign something
      shorter than $length, the string will shrink, and if you assign something longer
      than $length, the string will grow to accommodate it. To keep the string the
      same length, you may need to pad or chop your value using sprintf.
      If $offset and $length specify a substring that is partly outside the string,
      only the part within the string is returned. If the substring is beyond either
      end of the string, EUCJP::substr() returns the undefined value and produces a
      warning. When used as an lvalue, specifying a substring that is entirely outside
      the string raises an exception. Here's an example showing the behavior for
      boundary cases:
      my $name = 'fred';
      EUCJP::substr($name, 4) = 'dy';         # $name is now 'freddy'
      my $null = EUCJP::substr $name, 6, 2;   # returns "" (no warning)
      my $oops = EUCJP::substr $name, 7;      # returns undef, with warning
      EUCJP::substr($name, 7) = 'gap';        # raises an exception
      An alternative to using EUCJP::substr() as an lvalue is to specify the replacement
      string as the 4th argument. This allows you to replace parts of the $string and
      return what was there before in one operation, just as you can with splice().
      my $s = "The black cat climbed the green tree";
      my $z = EUCJP::substr $s, 14, 7, "jumped from";    # climbed
      # $s is now "The black cat jumped from the green tree"
      Note that the lvalue returned by the three-argument version of EUCJP::substr() acts
      as a 'magic bullet'; each time it is assigned to, it remembers which part of the
      original string is being modified; for example:
      $x = '1234';
      for (EUCJP::substr($x,1,2)) {
          $_ = 'a';   print $x,"\n";    # prints 1a4
          $_ = 'xyz'; print $x,"\n";    # prints 1xyz4
          $x = '56789';
          $_ = 'pq';  print $x,"\n";    # prints 5pq9
      With negative offsets, it remembers its position from the end of the string when
      the target string is modified:
      $x = '1234';
      for (EUCJP::substr($x, -3, 2)) {
          $_ = 'a';   print $x,"\n";    # prints 1a4, as above
          $x = 'abcdefg';
          print $_,"\n";                # prints f
      Prior to Perl version 5.10, the result of using an lvalue multiple times was
      unspecified. Prior to 5.16, the result with negative offsets was unspecified.
  • Index by EUC-JP Character

      $index = EUCJP::index($string,$substring,$offset);
      $index = EUCJP::index($string,$substring);
      This subroutine searches for one string within another. It returns the character
      position of the first occurrence of $substring in $string. The $offset, if
      specified, says how many characters from the start to skip before beginning to
      look. Positions are based at 0. If the substring is not found, the subroutine
      returns one less than the base, ordinarily -1. To work your way through a string,
      you might say:
      $pos = -1;
      while (($pos = EUCJP::index($string, $lookfor, $pos)) > -1) {
          print "Found at $pos\n";
  • Rindex by EUC-JP Character

      $rindex = EUCJP::rindex($string,$substring,$offset);
      $rindex = EUCJP::rindex($string,$substring);
      This subroutine works just like EUCJP::index except that it returns the character
      position of the last occurrence of $substring in $string (a reverse EUCJP::index).
      The subroutine returns -1 if $substring is not found. $offset, if specified, is
      the rightmost character position that may be returned. To work your way through
      a string backward, say:
      $pos = EUCJP::length($string);
      while (($pos = EUCJP::rindex($string, $lookfor, $pos)) >= 0) {
          print "Found at $pos\n";
  • Eval EUC-JP Script

      $eval = EUCJP::eval { block };
      $eval = EUCJP::eval $expr;
      $eval = EUCJP::eval;
      The EUCJP::eval keyword serves two distinct but related purposes in JPerl.
      These purposes are represented by two forms of syntax, EUCJP::eval { block }
      and EUCJP::eval $expr. The first form traps runtime exceptions (errors)
      that would otherwise prove fatal, similar to the "try block" construct in
      C++ or Java. The second form compiles and executes little bits of code on
      the fly at runtime, and also (conveniently) traps any exceptions just like
      the first form. But the second form runs much slower than the first form,
      since it must parse the string every time. On the other hand, it is also
      more general. Whichever form you use, EUCJP::eval is the preferred way to do
      all exception handling in JPerl.
      For either form of EUCJP::eval, the value returned from an EUCJP::eval is
      the value of the last expression evaluated, just as with subroutines.
      Similarly, you may use the return operator to return a value from the
      middle of the eval. The expression providing the return value is evaluated
      in void, scalar, or list context, depending on the context of the
      EUCJP::eval itself. See wantarray for more on how the evaluation context
      can be determined.
      If there is a trappable error (including any produced by the die operator),
      EUCJP::eval returns undef and puts the error message (or object) in $@. If
      there is no error, $@ is guaranteed to be set to the null string, so you
      can test it reliably afterward for errors. A simple Boolean test suffices:
          EUCJP::eval { ... }; # trap runtime errors
          if ($@) { ... }     # handle error
      (Prior to Perl 5.16, a bug caused undef to be returned in list context for
      syntax errors, but not for runtime errors.)
      The EUCJP::eval { block } form is syntax checked and compiled at compile time,
      so it is just as efficient at runtime as any other block. (People familiar
      with the slow EUCJP::eval $expr form are occasionally confused on this issue.)
      Because the { block } is compiled when the surrounding code is, this form of
      EUCJP::eval cannot trap syntax errors.
      The EUCJP::eval $expr form can trap syntax errors because it parses the code
      at runtime. (If the parse is unsuccessful, it places the parse error in $@,
      as usual.) If $expr is omitted, evaluates $_ .
      Otherwise, it executes the value of $expr as though it were a little JPerl
      script. The code is executed in the context of the current of the current
      JPerl script, which means that it can see any enclosing lexicals from a
      surrounding scope, and that any nonlocal variable settings remain in effect
      after the EUCJP::eval is complete, as do any subroutine or format definitions.
      The code of the EUCJP::eval is treated as a block, so any locally scoped
      variables declared within the EUCJP::eval last only until the EUCJP::eval is
      done. (See my and local.) As with any code in a block, a final semicolon is
      not required.
      EUCJP::eval will be escaped as follows:
      Before                  After
      EUCJP::eval { block }    eval { block }
      EUCJP::eval $expr        eval EUCJP::escape $expr
      EUCJP::eval              eval EUCJP::escape
      To tell the truth, the subroutine EUCJP::eval does not exist. If it exists,
      you will troubled, when EUCJP::eval has a parameter that is single quoted
      string included my variables. EUCJP::escape is a subroutine that makes Perl
      script from JPerl script.
      Here is a simple JPerl shell. It prompts the user to enter a string of
      arbitrary JPerl code, compiles and executes that string, and prints whatever
      error occurred:
          # jperlshell.pl - simple JPerl shell
          use EUCJP;
          print "\nEnter some JPerl code: ";
          while (<STDIN>) {
              print $@;
              print "\nEnter some more JPerl code: ";
      Here is a rename.pl script to do a mass renaming of files using a JPerl
          # rename.pl - change filenames
          use EUCJP;
          $op = shift;
          for (@ARGV) {
              $was = $_;
              EUCJP::eval $op;
              die if $@;
              # next line calls the built-in function, not
              # the script by the same name
              if ($was ne $_) {
                  print STDERR "rename $was --> $_\n";
      You'd use that script like this:
          C:\WINDOWS> perl rename.pl 's/\.orig$//' *.orig
          C:\WINDOWS> perl rename.pl 'y/A-Z/a-z/ unless /^Make/' *
          C:\WINDOWS> perl rename.pl '$_ .= ".bad"' *.f
      Since EUCJP::eval traps errors that would otherwise prove fatal, it is useful
      for determining whether particular features (such as fork or symlink) are
      Because EUCJP::eval { block } is syntax checked at compile time, any syntax
      error is reported earlier. Therefore, if your code is invariant and both
      EUCJP::eval $expr and EUCJP::eval { block } will suit your purposes equally
      well, the { block } form is preferred. For example:
          # make divide-by-zero nonfatal
          EUCJP::eval { $answer = $a / $b; };
          warn $@ if $@;
          # same thing, but less efficient if run multiple times
          EUCJP::eval '$answer = $a / $b';
          warn $@ if $@;
          # a compile-time syntax error (not trapped)
          EUCJP::eval { $answer = }; # WRONG
          # a runtime syntax error
          EUCJP::eval '$answer =';   # sets $@
      Here, the code in the { block } has to be valid JPerl code to make it past
      the compile phase. The code in the $expr doesn't get examined until runtime,
      so it doesn't cause an error until runtime.
      Using the EUCJP::eval { block } form as an exception trap in libraries does
      have some issues. Due to the current arguably broken state of __DIE__ hooks,
      you may wish not to trigger any __DIE__ hooks that user code may have
      installed. You can use the local $SIG{__DIE__} construct for this purpose,
      as this example shows:
          # a private exception trap for divide-by-zero
          EUCJP::eval { local $SIG{'__DIE__'}; $answer = $a / $b; };
          warn $@ if $@;
      This is especially significant, given that __DIE__ hooks can call die again,
      which has the effect of changing their error messages:
          # __DIE__ hooks may modify error messages
              local $SIG{'__DIE__'} =
                  sub { (my $x = $_[0]) =~ s/foo/bar/g; die $x };
              EUCJP::eval { die "foo lives here" };
              print $@ if $@;                # prints "bar lives here"
      Because this promotes action at a distance, this counterintuitive behavior
      may be fixed in a future release.
      With an EUCJP::eval, you should be especially careful to remember what's being
      looked at when:
          EUCJP::eval $x;        # CASE 1
          EUCJP::eval "$x";      # CASE 2
          EUCJP::eval '$x';      # CASE 3
          EUCJP::eval { $x };    # CASE 4
          EUCJP::eval "\$$x++";  # CASE 5
          $$x++;                # CASE 6
      CASEs 1 and 2 above behave identically: they run the code contained in the
      variable $x. (Although CASE 2 has misleading double quotes making the reader
      wonder what else might be happening (nothing is).) CASEs 3 and 4 likewise
      behave in the same way: they run the code '$x' , which does nothing but return
      the value of $x. (CASE 4 is preferred for purely visual reasons, but it also
      has the advantage of compiling at compile-time instead of at run-time.) CASE 5
      is a place where normally you would like to use double quotes, except that in
      this particular situation, you can just use symbolic references instead, as
      in CASE 6.
      Before Perl 5.14, the assignment to $@ occurred before restoration of
      localized variables, which means that for your code to run on older versions,
      a temporary is required if you want to mask some but not all errors:
          # alter $@ on nefarious repugnancy only
              my $e;
                  local $@; # protect existing $@
                  EUCJP::eval { test_repugnancy() };
                  # $@ =~ /nefarious/ and die $@; # Perl 5.14 and higher only
                  $@ =~ /nefarious/ and $e = $@;
              die $e if defined $e
      The block of EUCJP::eval { block } does not count as a loop, so the loop
      control statements next, last, or redo cannot be used to leave or restart the
  • Filename Globbing

      @glob = glob($expr);
      $glob = glob($expr);
      @glob = glob;
      $glob = glob;
      @glob = <*>;
      $glob = <*>;
      Performs filename expansion (globbing) on $expr, returning the next successive
      name on each call. If $expr is omitted, $_ is globbed instead.
      This operator is implemented via the Eeucjp::glob() subroutine. See Eeucjp::glob
      of Eeucjp.pm for details.

Byte-Oriented Functions

  • Chop Byte String

      $byte = CORE::chop($string);
      $byte = CORE::chop(@list);
      $byte = CORE::chop;
      This function chops off the last byte of a string variable and returns the
      byte chopped. The CORE::chop operator is used primarily to remove the newline
      from the end of an input record, and is more efficient than using a
      substitution (s/\n$//). If that's all you're doing, then it would be safer to
      use chomp, since CORE::chop always shortens the string no matter what's there,
      and chomp is more selective.
      You cannot CORE::chop a literal, only a variable.
      If you CORE::chop a @list of variables, each string in the list is chopped:
      @lines = `cat myfile`;
      CORE::chop @lines;
      You can CORE::chop anything that is an lvalue, including an assignment:
      CORE::chop($cwd = `pwd`);
      CORE::chop($answer = <STDIN>);
      This is different from:
      $answer = CORE::chop($temp = <STDIN>); # WRONG
      which puts a newline into $answer because CORE::chop returns the byte chopped,
      not the remaining string (which is in $tmp). One way to get the result
      intended here is with substr:
      $answer = substr <STDIN>, 0, -1;
      But this is more commonly written as:
      CORE::chop($answer = <STDIN>);
      In the most general case, CORE::chop can be expressed in terms of substr:
      $last_byte = CORE::chop($var);
      $last_byte = substr($var, -1, 1, ""); # same thing
      Once you understand this equivalence, you can use it to do bigger chops. To
      CORE::chop more than one byte, use substr as an lvalue, assigning a null
      string. The following removes the last five bytes of $caravan:
      substr($caravan, -5) = "";
      The negative subscript causes substr to count from the end of the string
      instead of the beginning. If you wanted to save the bytes so removed, you
      could use the four-argument form of substr, creating something of a quintuple
      $tail = substr($caravan, -5, 5, "");
      If no argument is given, the function chops the $_ variable.
  • Ordinal Value of Byte

      $ord = CORE::ord($expr);
      This function returns the numeric value of the first byte of $expr, regardless
      of "use EUCJP qw(ord);" exists or not. If $expr is omitted, it uses $_.
      The return value is always unsigned.
      If you want a signed value, use unpack('c',$expr). If you want all the bytes of
      the string converted to a list of numbers, use unpack('C*',$expr) instead.
  • Reverse List or Byte String

      @reverse = CORE::reverse(@list);
      $reverse = CORE::reverse(@list);
      In list context, this function returns a list value consisting of the elements
      of @list in the opposite order.
      In scalar context, the function concatenates all the elements of @list and then
      returns the reverse of that resulting string, byte by byte, regardless of
      "use EUCJP qw(reverse);" exists or not.
  • Returns Next Byte

      $getc = CORE::getc(FILEHANDLE);
      $getc = CORE::getc($filehandle);
      $getc = CORE::getc;
      This function returns the next byte from the input file attached to FILEHANDLE.
      It returns undef at end-of-file, or if an I/O error was encountered. If
      FILEHANDLE is omitted, the function reads from STDIN.
      This function is somewhat slow, but it's occasionally useful for single-byte
      input from the keyboard -- provided you manage to get your keyboard input
      unbuffered. This function requests unbuffered input from the standard I/O library.
      Unfortunately, the standard I/O library is not so standard as to provide a portable
      way to tell the underlying operating system to supply unbuffered keyboard input to
      the standard I/O system. To do that, you have to be slightly more clever, and in
      an operating-system-dependent fashion. Under Unix you might say this:
      if ($BSD_STYLE) {
          system "stty cbreak </dev/tty >/dev/tty 2>&1";
      else {
          system "stty", "-icanon", "eol", "\001";
      $key = CORE::getc;
      if ($BSD_STYLE) {
          system "stty -cbreak </dev/tty >/dev/tty 2>&1";
      else {
          system "stty", "icanon", "eol", "^@"; # ASCII NUL
      print "\n";
      This code puts the next single-byte typed on the terminal in the string $key.
      If your stty program has options like cbreak, you'll need to use the code where
      $BSD_STYLE is true. Otherwise, you'll need to use the code where it is false.
  • Index by Byte String

      $index = CORE::index($string,$substring,$offset);
      $index = CORE::index($string,$substring);
      This function searches for one byte string within another. It returns the position
      of the first occurrence of $substring in $string. The $offset, if specified, says
      how many bytes from the start to skip before beginning to look. Positions are based
      at 0. If the substring is not found, the function returns one less than the base,
      ordinarily -1. To work your way through a string, you might say:
      $pos = -1;
      while (($pos = CORE::index($string, $lookfor, $pos)) > -1) {
          print "Found at $pos\n";
  • Rindex by Byte String

      $rindex = CORE::rindex($string,$substring,$offset);
      $rindex = CORE::rindex($string,$substring);
      This function works just like CORE::index except that it returns the position of
      the last occurrence of $substring in $string (a reverse CORE::index). The function
      returns -1 if not $substring is found. $offset, if specified, is the rightmost
      position that may be returned. To work your way through a string backward, say:
      $pos = CORE::length($string);
      while (($pos = CORE::rindex($string, $lookfor, $pos)) >= 0) {
          print "Found at $pos\n";

Yada Yada Operator (EUCJP software provides)

  The yada yada operator (noted ...) is a placeholder for code. Perl parses it
  without error, but when you try to execute a yada yada, it throws an exception
  with the text Unimplemented:

  sub unimplemented { ... }
  eval { unimplemented() };
  if ( $@ eq 'Unimplemented' ) {
      print "I found the yada yada!\n";

  You can only use the yada yada to stand in for a complete statement. These
  examples of the yada yada work:

  { ... }
  sub foo { ... }
  eval { ... };
  sub foo {
      my( $self ) = shift;
  do { my $n; ...; print 'Hurrah!' };

  The yada yada cannot stand in for an expression that is part of a larger statement
  since the ... is also the three-dot version of the range operator
  (see "Range Operators"). These examples of the yada yada are still syntax errors:

  print ...;
  open my($fh), '>', '/dev/passwd' or ...;
  if ( $condition && ... ) { print "Hello\n" };

  There are some cases where Perl can't immediately tell the difference between an
  expression and a statement. For instance, the syntax for a block and an anonymous
  hash reference constructor look the same unless there's something in the braces that
  give Perl a hint. The yada yada is a syntax error if Perl doesn't guess that the
  { ... } is a block. In that case, it doesn't think the ... is the yada yada because
  it's expecting an expression instead of a statement:

  my @transformed = map { ... } @input;  # syntax error

  You can use a ; inside your block to denote that the { ... } is a block and not a
  hash reference constructor. Now the yada yada works:

  my @transformed = map {; ... } @input; # ; disambiguates
  my @transformed = map { ...; } @input; # ; disambiguates

Un-Escaping bytes::* Subroutines (EUCJP software provides)

EUCJP software removes 'bytes::' at head of subroutine name.

  Before           After     Works as
  bytes::chr       chr       Byte
  bytes::index     index     Byte
  bytes::length    length    Byte
  bytes::ord       ord       Byte
  bytes::rindex    rindex    Byte
  bytes::substr    substr    Byte

Ignore Pragmas and Modules

  Before                    After
  use strict;               use strict; no strict qw(refs);
  use 5.12.0;               use 5.12.0; no strict qw(refs);
  require utf8;             # require utf8;
  require bytes;            # require bytes;
  require charnames;        # require charnames;
  require I18N::Japanese;   # require I18N::Japanese;
  require I18N::Collate;    # require I18N::Collate;
  require I18N::JExt;       # require I18N::JExt;
  require File::DosGlob;    # require File::DosGlob;
  require Wild;             # require Wild;
  require Wildcard;         # require Wildcard;
  require Japanese;         # require Japanese;
  use utf8;                 # use utf8;
  use bytes;                # use bytes;
  use charnames;            # use charnames;
  use I18N::Japanese;       # use I18N::Japanese;
  use I18N::Collate;        # use I18N::Collate;
  use I18N::JExt;           # use I18N::JExt;
  use File::DosGlob;        # use File::DosGlob;
  use Wild;                 # use Wild;
  use Wildcard;             # use Wildcard;
  use Japanese;             # use Japanese;
  no utf8;                  # no utf8;
  no bytes;                 # no bytes;
  no charnames;             # no charnames;
  no I18N::Japanese;        # no I18N::Japanese;
  no I18N::Collate;         # no I18N::Collate;
  no I18N::JExt;            # no I18N::JExt;
  no File::DosGlob;         # no File::DosGlob;
  no Wild;                  # no Wild;
  no Wildcard;              # no Wildcard;
  no Japanese;              # no Japanese;

  Comment out pragma to ignore utf8 environment, and Eeucjp.pm provides these
  • Dummy utf8::upgrade

      $num_octets = utf8::upgrade($string);
      Returns the number of octets necessary to represent the string.
  • Dummy utf8::downgrade

      $success = utf8::downgrade($string[, FAIL_OK]);
      Returns true always.
  • Dummy utf8::encode

      Returns nothing.
  • Dummy utf8::decode

      $success = utf8::decode($string);
      Returns true always.
  • Dummy utf8::is_utf8

      $flag = utf8::is_utf8(STRING);
      Returns false always.
  • Dummy utf8::valid

      $flag = utf8::valid(STRING);
      Returns true always.
  • Dummy bytes::chr

      This subroutine is same as chr.
  • Dummy bytes::index

      This subroutine is same as index.
  • Dummy bytes::length

      This subroutine is same as length.
  • Dummy bytes::ord

      This subroutine is same as ord.
  • Dummy bytes::rindex

      This subroutine is same as rindex.
  • Dummy bytes::substr

      This subroutine is same as substr.

Environment Variable

 This software uses the flock function for exclusive control. The execution of the
 program is blocked until it becomes possible to read or write the file.
 You can have it not block in the flock function by defining environment variable
 (The value '1' doesn't have the meaning)


I have tested and verified this software using the best of my ability. However, a software containing much regular expression is bound to contain some bugs. Thus, if you happen to find a bug that's in EUCJP software and not your own program, you can try to reduce it to a minimal test case and then report it to the following author's address. If you have an idea that could make this a more useful tool, please let everyone share it.

  • (dummy item to avoid Test::Pod error)

  • format

    Function "format" can't handle multiple-octet code same as original Perl.

  • cloister of regular expression

    The cloister (?s) and (?i) of a regular expression will not be implemented for the time being. Cloister (?s) can be substituted with the .(dot) and \N on /s modifier. Cloister (?i) can be substituted with \F...\E.

  • EUCJP::substr as Lvalue

    If Perl version is older than 5.14, EUCJP::substr differs from CORE::substr, and cannot be used as a lvalue. To change part of a string, you need use the optional fourth argument which is the replacement string.

    EUCJP::substr($string, 13, 4, "JPerl");

  • Special Variables $` and $& need /( Capture All )/

      Because $` and $& use $1.
      Before          After                Works as
      $`              Eeucjp::PREMATCH()    CORE::substr($&,0,CORE::length($&)-CORE::length($1))
      ${`}            Eeucjp::PREMATCH()    CORE::substr($&,0,CORE::length($&)-CORE::length($1))
      $PREMATCH       Eeucjp::PREMATCH()    CORE::substr($&,0,CORE::length($&)-CORE::length($1))
      ${^PREMATCH}    Eeucjp::PREMATCH()    CORE::substr($&,0,CORE::length($&)-CORE::length($1))
      $&              Eeucjp::MATCH()       $1
      ${&}            Eeucjp::MATCH()       $1
      $MATCH          Eeucjp::MATCH()       $1
      ${^MATCH}       Eeucjp::MATCH()       $1
      $'              $'                   $'
      ${'}            ${'}                 $'
      $POSTMATCH      Eeucjp::POSTMATCH()   $'
      ${^POSTMATCH}   Eeucjp::POSTMATCH()   $'
  • Limitation of Regular Expression

    This software has limitation from \G in multibyte anchoring. Only the following Perl can treat the character string which exceeds 32766 octets with a regular expression.

    perl 5.6 or later --- ActivePerl on MSWin32

    perl 5.10.1 or later --- other Perl

      see also,
      In 5.10.0, the * quantifier in patterns was sometimes treated as {0,32767}
      [perl #116379] \G can't treat over 32767 octet
      perlre - Perl regular expressions
      perlre length limit
      Japanese Document
  • Empty Variable in Regular Expression

    Unlike literal null string, an interpolated variable evaluated to the empty string can't use the most recent pattern from a previous successful regular expression.

  • Limitation of ?? and m??

    Multibyte character needs ( ) which is before {n,m}, {n,}, {n}, *, and + in ?? or m??. As a result, you need to rewrite a script about $1,$2,$3,... You cannot use (?: ) ?, {n,m}?, {n,}?, and {n}? in ?? and m??, because delimiter of m?? is '?'.

  • Modifier /a /d /l and /u of Regular Expression

    The concept of this software is not to use two or more encoding methods as literal string and literal of regexp in one Perl script. Therefore, modifier /a, /d, /l, and /u are not supported. \d means [0-9] universally.

  • Named Character

    A named character, such \N{GREEK SMALL LETTER EPSILON}, \N{greek:epsilon}, or \N{epsilon} is not supported.

  • Unicode Properties (aka Character Properties) of Regular Expression

    Unicode properties (aka character properties) of regexp are not available. Also (?[]) in regexp of Perl 5.18 is not available. There is no plans to currently support these.

  • Delimiter of String and Regexp

    qq//, q//, qw//, qx//, qr//, m//, s///, tr///, and y/// can't use a wide character as the delimiter.

  • \b{...} Boundaries in Regular Expressions

    Following \b{...} available starting in v5.22 are not supported.

      \b{gcb} or \b{g}   Unicode "Grapheme Cluster Boundary"
      \b{sb}             Unicode "Sentence Boundary"
      \b{wb}             Unicode "Word Boundary"
      \B{gcb} or \B{g}   Unicode "Grapheme Cluster Boundary" doesn't match
      \B{sb}             Unicode "Sentence Boundary" doesn't match
      \B{wb}             Unicode "Word Boundary" doesn't match


INABA Hitoshi <ina@cpan.org>

This project was originated by INABA Hitoshi.


This software is free software; you can redistribute it and/or modify it under the same terms as Perl itself. See perlartistic.

This software is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

My Goal

P.401 See chapter 15: Unicode of ISBN 0-596-00027-8 Programming Perl Third Edition.

Before the introduction of Unicode support in perl, The eq operator just compared the byte-strings represented by two scalars. Beginning with perl 5.8, eq compares two byte-strings with simultaneous consideration of the UTF8 flag.

/* You are not expected to understand this */

  Information processing model beginning with perl 5.8
    |     Text strings     |                     |
    +----------+-----------|    Binary strings   |
    |  UTF-8   |  Latin-1  |                     |
    | UTF8     |            Not UTF8             |
    | Flagged  |            Flagged              |

  Confusion of Perl string model is made from double meanings of
  "Binary string."
  Meanings of "Binary string"
  1. Non-Text string
  2. Digital octet string

  Let's draw again using those term.
    |     Text strings     |                     |
    +----------+-----------|   Non-Text strings  |
    |  UTF-8   |  Latin-1  |                     |
    | UTF8     |            Not UTF8             |
    | Flagged  |            Flagged              |
    |            Digital octet string            |

There are people who don't agree to change in the character string processing model of Perl 5.8. It is impossible to get to agree it to majority of Perl user who hardly ever use Perl. How to solve it by returning to a original method, let's drag out page 402 of the old dusty Programming Perl, 3rd ed. again.

  Information processing model beginning with perl3 or this software
  of UNIX/C-ism.

    |    Text string as Digital octet string     |
    |    Digital octet string as Text string     |
    |       Not UTF8 Flagged, No Mojibake        |

  In UNIX Everything is a File
  - In UNIX everything is a stream of bytes
  - In UNIX the filesystem is used as a universal name space

  Native Encoding Scripting
  - native encoding of file contents
  - native encoding of file name on filesystem
  - native encoding of command line
  - native encoding of environment variable
  - native encoding of API
  - native encoding of network packet
  - native encoding of database

Ideally, I'd like to achieve these five Goals:

  • Goal #1:

    Old byte-oriented programs should not spontaneously break on the old byte-oriented data they used to work on.

    This goal has been achieved by that this software is additional code for perl like utf8 pragma. Perl should work same as past Perl if added nothing.

  • Goal #2:

    Old byte-oriented programs should magically start working on the new character-oriented data when appropriate.

    Still now, 1 octet is counted with 1 by built-in functions length, substr, index, rindex, and pos that handle length and position of string. In this part, there is no change. The length of 1 character of 2 octet code is 2.

    On the other hand, the regular expression in the script is added the multibyte anchoring processing with this software, instead of you.

    figure of Goal #1 and Goal #2.

                                   GOAL#1  GOAL#2
                            (a)     (b)     (c)     (d)     (e)
          | data         |  Old  |  Old  |  New  |  Old  |  New  |
          | script       |  Old  |      Old      |      New      |
          | interpreter  |  Old  |              New              |
          Old --- Old byte-oriented
          New --- New character-oriented

    There is a combination from (a) to (e) in data, script, and interpreter of old and new. Let's add the Encode module and this software did not exist at time of be written this document and JPerl did exist.

                            (a)     (b)     (c)     (d)     (e)
                                          JPerl,japerl    Encode,EUCJP
          | data         |  Old  |  Old  |  New  |  Old  |  New  |
          | script       |  Old  |      Old      |      New      |
          | interpreter  |  Old  |              New              |
          Old --- Old byte-oriented
          New --- New character-oriented

    The reason why JPerl is very excellent is that it is at the position of (c). That is, it is not necessary to do a special description to the script to process new character-oriented string. (May the japerl take over JPerl!)

  • Goal #3:

    Programs should run just as fast in the new character-oriented mode as in the old byte-oriented mode.

    It is impossible. Because the following time is necessary.

    (1) Time of escape script for old byte-oriented perl.

    (2) Time of processing regular expression by escaped script while multibyte anchoring.

    Someday, I want to ask Larry Wall about this goal in the elevator.

  • Goal #4:

    Perl should remain one language, rather than forking into a byte-oriented Perl and a character-oriented Perl.

    JPerl remains one Perl language by forking to two interpreters. However, the Perl core team did not desire fork of the interpreter. As a result, Perl language forked contrary to goal #4.

    A character-oriented perl is not necessary to make it specially, because a byte-oriented perl can already treat the binary data. This software is only an application program of byte-oriented Perl, a filter program.

    And you will get support from the Perl community, when you solve the problem by the Perl script.

    EUCJP software remains one language and one interpreter.

  • Goal #5:

    JPerl users will be able to maintain JPerl by Perl.

    May the JPerl be with you, always.

Back when Programming Perl, 3rd ed. was written, UTF8 flag was not born and Perl is designed to make the easy jobs easy. This software provides programming environment like at that time.

Perl's motto

   Some computer scientists (the reductionists, in particular) would
  like to deny it, but people have funny-shaped minds. Mental geography
  is not linear, and cannot be mapped onto a flat surface without
  severe distortion. But for the last score years or so, computer
  reductionists have been first bowing down at the Temple of Orthogonality,
  then rising up to preach their ideas of ascetic rectitude to any who
  would listen.
   Their fervent but misguided desire was simply to squash your mind to
  fit their mindset, to smush your patterns of thought into some sort of
  Hyperdimensional Flatland. It's a joyless existence, being smushed.
  --- Learning Perl on Win32 Systems

  If you think this is a big headache, you're right. No one likes
  this situation, but Perl does the best it can with the input and
  encodings it has to deal with. If only we could reset history and
  not make so many mistakes next time.
  --- Learning Perl 6th Edition

   The most important thing for most people to know about handling
  Unicode data in Perl, however, is that if you don't ever use any Uni-
  code data -- if none of your files are marked as UTF-8 and you don't
  use UTF-8 locales -- then you can happily pretend that you're back in
  Perl 5.005_03 land; the Unicode features will in no way interfere with
  your code unless you're explicitly using them. Sometimes the twin
  goals of embracing Unicode but not disturbing old-style byte-oriented
  scripts has led to compromise and confusion, but it's the Perl way to
  silently do the right thing, which is what Perl ends up doing.
  --- Advanced Perl Programming, 2nd Edition


 Larry Wall, Randal L.Schwartz, Yoshiyuki Kondo
 December 1997
 ISBN 4-89052-384-7

 Programming Perl, Second Edition
 By Larry Wall, Tom Christiansen, Randal L. Schwartz
 October 1996
 Pages: 670
 ISBN 10: 1-56592-149-6 | ISBN 13: 9781565921498

 Programming Perl, Third Edition
 By Larry Wall, Tom Christiansen, Jon Orwant
 Third Edition  July 2000
 Pages: 1104
 ISBN 10: 0-596-00027-8 | ISBN 13: 9780596000271

 The Perl Language Reference Manual (for Perl version 5.12.1)
 by Larry Wall and others
 Paperback (6"x9"), 724 pages
 Retail Price: $39.95 (pound 29.95 in UK)
 ISBN-13: 978-1-906966-02-7

 Perl Pocket Reference, 5th Edition
 By Johan Vromans
 Publisher: O'Reilly Media
 Released: July 2011
 Pages: 102

 Programming Perl, 4th Edition
 By: Tom Christiansen, brian d foy, Larry Wall, Jon Orwant
 Publisher: O'Reilly Media
 Formats: Print, Ebook, Safari Books Online
 Released: March 2012
 Pages: 1130
 Print ISBN: 978-0-596-00492-7 | ISBN 10: 0-596-00492-3
 Ebook ISBN: 978-1-4493-9890-3 | ISBN 10: 1-4493-9890-1

 Perl Cookbook
 By Tom Christiansen, Nathan Torkington
 August 1998
 Pages: 800
 ISBN 10: 1-56592-243-3 | ISBN 13: 978-1-56592-243-3

 Perl Cookbook, Second Edition
 By Tom Christiansen, Nathan Torkington
 Second Edition  August 2003
 Pages: 964
 ISBN 10: 0-596-00313-7 | ISBN 13: 9780596003135

 Perl in a Nutshell, Second Edition
 By Stephen Spainhour, Ellen Siever, Nathan Patwardhan
 Second Edition  June 2002
 Pages: 760
 Series: In a Nutshell
 ISBN 10: 0-596-00241-6 | ISBN 13: 9780596002411

 Learning Perl on Win32 Systems
 By Randal L. Schwartz, Erik Olson, Tom Christiansen
 August 1997
 Pages: 306
 ISBN 10: 1-56592-324-3 | ISBN 13: 9781565923249

 Learning Perl, Fifth Edition
 By Randal L. Schwartz, Tom Phoenix, brian d foy
 June 2008
 Pages: 352
 Print ISBN:978-0-596-52010-6 | ISBN 10: 0-596-52010-7
 Ebook ISBN:978-0-596-10316-3 | ISBN 10: 0-596-10316-6

 Learning Perl, 6th Edition
 By Randal L. Schwartz, brian d foy, Tom Phoenix
 June 2011
 Pages: 390
 ISBN-10: 1449303587 | ISBN-13: 978-1449303587

 Advanced Perl Programming, 2nd Edition
 By Simon Cozens
 June 2005
 Pages: 300
 ISBN-10: 0-596-00456-7 | ISBN-13: 978-0-596-00456-9

 Futato, Irving, Jepson, Patwardhan, Siever
 ISBN 10: 1-56592-370-7

 Perl Resource Kit -- Win32 Edition
 Erik Olson, Brian Jepson, David Futato, Dick Hardt
 ISBN 10:1-56592-409-6

 By Daisuke Maki
 Pages: 344
 ISBN 10: 4798119172 | ISBN 13: 978-4798119175

 Understanding Japanese Information Processing
 By Ken Lunde
 January 1900
 Pages: 470
 ISBN 10: 1-56592-043-0 | ISBN 13: 9781565920439

 CJKV Information Processing
 Chinese, Japanese, Korean & Vietnamese Computing
 By Ken Lunde
 First Edition  January 1999
 Pages: 1128
 ISBN 10: 1-56592-224-7 | ISBN 13: 9781565922242

 Mastering Regular Expressions, Second Edition
 By Jeffrey E. F. Friedl
 Second Edition  July 2002
 Pages: 484
 ISBN 10: 0-596-00289-0 | ISBN 13: 9780596002893

 Mastering Regular Expressions, Third Edition
 By Jeffrey E. F. Friedl
 Third Edition  August 2006
 Pages: 542
 ISBN 10: 0-596-52812-4 | ISBN 13:9780596528126

 Regular Expressions Cookbook
 By Jan Goyvaerts, Steven Levithan
 May 2009
 Pages: 512
 ISBN 10:0-596-52068-9 | ISBN 13: 978-0-596-52068-7

 Regular Expressions Cookbook, 2nd Edition
 By Jan Goyvaerts, Steven Levithan
 Final Release Date: August 2012
 Pages: 612
 ISBN: 978-1-4493-1943-4 | ISBN 10:1-4493-1943-2

 By Kouji Shibano
 Pages: 1456
 ISBN 4-542-20129-5

 1993 Aug
 Pages: 172
 T1008901080816 ZASSHI 08901-8

 By YAMAGATA Hiroo, Stephen J. Turnbull, Craig Oda, Robert J. Bickel
 June, 2000
 Pages: 376
 ISBN 4-87311-016-5

 MacPerl Power and Ease
 By Vicki Brown, Chris Nandor
 April 1998
 Pages: 350
 ISBN 10: 1881957322 | ISBN 13: 978-1881957324

 Windows NT Shell Scripting
 By Timothy Hill
 April 27, 1998
 Pages: 400
 ISBN 10: 1578700477 | ISBN 13: 9781578700479

 Windows(R) Command-Line Administrators Pocket Consultant, 2nd Edition
 By William R. Stanek
 February 2009
 Pages: 594
 ISBN 10: 0-7356-2262-0 | ISBN 13: 978-0-7356-2262-3

 Kaoru Maeda, Perl's history Perl 1,2,3,4

 nurse, What is "string"

 NISHIO Hirokazu, What's meant "string as a sequence of characters"?

 nurse, History of Japanese EUC 22:00

 Mike Whitaker, Perl And Unicode

 Ricardo Signes, Perl 5.14 for Pragmatists

 Ricardo Signes, What's New in Perl? v5.10 - v5.16 #'

 YAP(achimon)C::Asia Hachioji 2016 mid in Shinagawa
 Kenichi Ishigaki (@charsbar) July 3, 2016 YAP(achimon)C::Asia Hachioji 2016mid

 CPAN Directory INABA Hitoshi


 Recent Perl packages by "INABA Hitoshi"


This software was made referring to software and the document that the following hackers or persons had made. I am thankful to all persons.

 Rick Yamashita, Shift_JIS
 ttp://furukawablog.spaces.live.com/Blog/cns!1pmWgsL289nm7Shn7cS0jHzA!2225.entry (dead link)
 (add 'h' at head)

 Larry Wall, Perl

 Kazumasa Utashiro, jcode.pl

 Jeffrey E. F. Friedl, Mastering Regular Expressions

 SADAHIRO Tomoyuki, The right way of using Shift_JIS

 Yukihiro "Matz" Matsumoto, YAPC::Asia2006 Ruby on Perl(s)

 jscripter, For jperl users

 Bruce., Unicode in Perl

 Hiroaki Izumi, Perl5.8/Perl5.10 is not useful on the Windows.

 TSUKAMOTO Makio, Perl memo/file path of Windows

 chaichanPaPa, Matching Shift_JIS file name

 SUZUKI Norio, Jperl

 WATANABE Hirofumi, Jperl

 Chuck Houpt, Michiko Nozu, MacJPerl

 Kenichi Ishigaki, Pod-PerldocJp, Welcome to modern Perl world

 Fuji, Goro (gfx), Perl Hackers Hub No.16

 Dan Kogai, Encode module
 http://www.archive.org/details/YAPCAsia2006TokyoPerl58andUnicodeMythsFactsandChanges (video)
 http://yapc.g.hatena.ne.jp/jkondo/ (audio)

 Takahashi Masatuyo, JPerl Wiki

 Juerd, Perl Unicode Advice

 daily dayflower, 2008-06-25 perluniadvice

 Unicode issues in Perl

 Jesse Vincent, Compatibility is a virtue

 Tokyo-pm archive

 Error: Runtime exception on jperl 5.005_03


 Object-oriented with Perl