balance.pl - Create a balanced Senseval-2 data file that has the same number of instances for each possible sense.
balance.pl [OPTIONS] DATA N > Balanced-DATA
This is the original distribution of senses in a Senseval-2 data file:
frequency.pl begin-v.test.xml
Output =>
<sense id="begin%2:30:00::" percent="64.31"/> <sense id="begin%2:30:01::" percent="14.51"/> <sense id="begin%2:42:04::" percent="21.18"/> Total Instances = 255 Total Distinct Senses=3 Distribution={64.31,21.18,14.51} % of Majority Sense = 64.31
Here they are balanced with 20 instances per sense.
balance.pl begin.v-test.xml 20 > bal-output frequency.pl bal-output
<sense id="begin%2:30:00::" percent="33.33"/> <sense id="begin%2:30:01::" percent="33.33"/> <sense id="begin%2:42:04::" percent="33.33"/> Total Instances = 60 Total Distinct Senses=3 Distribution={33.33,33.33,33.33} % of Majority Sense = 33.33
Here they are balanced with 50 instances per sense. Please note that any sense with less than 50 instances is removed from the data.
balance.pl begin.v-test.xml 50 > bal-output frequency.pl bal-output
<sense id="begin%2:30:00::" percent="50.00"/> <sense id="begin%2:42:04::" percent="50.00"/> Total Instances = 100 Total Distinct Senses=2 Distribution={50.00,50.00} % of Majority Sense = 50.00
You can find begin-v.test.xml in samples/Data
Type balance.pl --help for a quick summary of options
balance.pl --help
This program will choose exactly the same number of instances for each sense found in a given Senseval-2 file. Unless a value is specified, it will choose the number of instances present in the least frequent sense. Output is to STDOUT, so the original input data is unchanged.
balance.pl accepts a Senseval-2 data file.
Specifies the number of instances to be selected from each sense.
--count COUNT
Balances the COUNT file created by SenseTool's preprocess.pl along with the DATA file. COUNT file is balanced such that it stays consistent with the new balanced DATA file and contains only those instances left after balancing, in the same order as they appear in the output.
Balanced COUNT is written to file COUNT.balanced and every ith line in COUNT.balanced is instance data within <context> and </context> tags for the ith instance in the output of balance.
Displays this message.
Displays the version information.
Output is a sense balanced Senseval-2 file and is displayed to stdout. Output will show exactly N instances of each sense that has at least N instances. All senses in the output Senseval-2 will have equal number of instances meaning the senses will be equally distributed.
The output of balance.pl will have un-balanced distribution of senses when some of the instances have multiple sense tags in the given DATA file.
Ted Pedersen, University of Minnesota, Duluth tpederse at d.umn.edu Amruta Purandare, University of Pittsburgh
Copyright (c) 2002-2008, Amruta Purandare and Ted Pedersen
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to
The Free Software Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
To install Text::SenseClusters, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Text::SenseClusters
CPAN shell
perl -MCPAN -e shell install Text::SenseClusters
For more information on module installation, please visit the detailed CPAN module installation guide.