CHANGES Changelog for the Ngram Statistics Package (Text-NSP)

    Revision history for Perl module Text::NSP

        Released March 5, 2010 all changes by TDP and YL

        *   Replaced with a more efficient version that counts
            large number of bigrams by creating multiple files, sorting, and
            merging them. The sorting and merging are carried out by
            huge-sort and Note that the previous versions of
   and associated utilities can be found in
            /Text-NSP/bin/utils/deprecated and will remain there for at
            least one more release. They will not however be installed
            automatically. (YL)

        *   Added --uremove and --ufrequency options to This allow
            for frequency cutoffs based on ngrams occuring more than a given
            number of times (rather than just less than, which is what
            --remove and --frequency enable). This is a long standing item
            on the NSP Todo list that has finally been checked off! (YL)

        *   Introduced /bin/utils/contributed to allow for the distribution
            of user contributed programs that might be useful to other
            users. These programs do not get installed automatically with
            NSP, and are not included in our standard testing streams, but
            could still prove very useful to users. Please let us know if
            you have code you might like to include here. (TDP)

        *   Added nsp-stoplist.regex to distribution (in
            /Text-NSP/bin/utils), to serve as a default stoplist. (TDP)

             Reported here : <L>

            This was not added in 1.11 due to failure to rebuilt MANIFEST.

        *   Added support for 4-d log-likelihood
            (Text::NSP::Measures::4D::MI:ll). (TDP)

        Released Nov 5, 2009 all changes by TDP

        *   Fixed bug in which caused long form of pmi
            (Text::NSP::Measures::3D::MI::pmi) not to be handled correctly
            on the command line, and that caused pmi_exp not to be properly
            initialize when using the long form of pmi.

              Reported here : <L>

        *   Added nsp-stoplist.regex to distribution (in
            /Text-NSP/bin/utils), to serve as a default stoplist.

              Reported here : <L>

        *   Fixed link to class diagram in FAQ.pod.

              Reported here : <L>

        *   Fixed documentation Text::NSP::Measures::3D::MI::pmi to
            correctly show how we are computing expected values.

              Reported here : <L>

        *   Fixed a few broken links in README.pod that were discovered
            while preparng this release

        Released March 26, 2008 all changes by TDP

        *   Spell checked the modules

        *   Relaxed test cases 27, 29 for ll, 20 for x2, and 13 for phi, due
            to arithmetic differences on 64 bit architectures

        *   Modified Makefile.PL to go back to more standard methods of
            testing and installation.

        *   Modified structure of /t directory for 'make test'. It appears
            that the use of subdirectories in /t with test cases might have
            been causing problems for Windows testing, so we have moved all
            test files to the top level of /t, and also removed the TEST
            program so that things are called in a more standard or generic

        Released March 24, 2008 all changes by TDP

        *   Updated Makefile.PL to no longer require 5.8.5 - have dropped
            back to 5.6

        *   Updated FAQ with some explanation of

        *   Renamed /docs as /doc to be consistent with other packages

        *   Added descriptive labels in POD in NAME field of .pl programs to
            provide that info on CPAN display

        *   Fixed duplicate Copyright message bug in documentation of

        *   Removed "help" messages from Makefile.PL execution so as to
            (hopefully) avoid problems with installations on Windows.

        *   Corrected error in INSTALL instructions - csh ./
            must be performed after 'make install'

        Released March 20, 2008 all changes by TDP

        *   Fixed problem with file Testing/statistic/t2 would appear
            (mysteriously) but not be in the MANIFEST. This file was left
            behind during /Testing/statistic/ and is now being

        *   Fixed problem in /Testing where .sh files are sometimes not
            executable. Those files are now invoked via 'csh' rather
            than './', meaning that they no longer need to be

        *   Fixed ticket number 24061 from regarding incorrect
            version information coming from

        *   Archived all old ChangeLogs to doc/ChangeLogs directory. Began
            to use pod in CHANGES directory instead

        *   Added doc/ to automatically refresh top level read
            only documentation including README, CHANGES, TODO and INSTALL

        *   Fixed Makefile.PL to avoid problems during Windows install. This
            problem and fix was reported by Richard Churchill to the ngram
            mailing list. This may also address ticket #20371 from

        *   Modified Makefile.PL to allow for use of 'make dist' and also
            creation of META.yml

     Ted Pedersen, University of Minnesota, Duluth
     tpederse at

    This document last modified by : $Id: CHANGES.pod,v 1.17 2010/03/04
    04:08:16 tpederse Exp $


    Copyright (c) 2004-2010 Ted Pedersen

    Permission is granted to copy, distribute and/or modify this document
    under the terms of the GNU Free Documentation License, Version 1.2 or
    any later version published by the Free Software Foundation; with no
    Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts.

    Note: a copy of the GNU Free Documentation License is available on the
    web at <> and is included in this
    distribution as FDL.txt.