Treex::Tool::Parser::MSTperl::Config
version 0.08055
Handles the configuration of the parser.
Fields describing fields used with nodes, such as form, pos, lemma...
Field names (for conversion of field index to field name)
1 for each field name to easily check if a field name exists
Index of each field name in field_names (for conversion of field name to field index)
Most of the settings are set by a config file in YAML format. However, you do not have to understand YAML to be able to change the settings provided that you keep things like formating of the file unchanged (some whitespaces are significant etc.). Actually only a subset of all all that YAML provides is used.
Contents of a line from the # character till the end of the line are comments and are ignored (if you need to actually use the # sign, you can quote it - eg. '#empty#' is interpreted as #empty#). Lines that contain only whitespace chars or are empty are ignored as well.
'#empty#'
#empty#
Some of the settings are ignored when in parsing mode (i.e. not training). These are use_edge_features_cache (turned off) and number_of_iterations (irrelevant).
These are settings which are acquired from the configuration file (see also its contents, the options are also richly commented there):
Lowercase names of fields in the input file (the data fields are to be separated by tabs in the input file). Use [a-z0-9_] only, using always at least one letter. Use unique names, i.e. devise some names even for unused fields.
Field values to set for the (technical) root node.
Name of field containing ord of the parent of the node (also called "head" or "governing node").
How many times the trainer (Tagger::MSTperl::Trainer) should go through all the training data (default is 10).
10
Turns on and off using the edge_features_cache. Default is 0.
edge_features_cache
0
Using cache should be turned on (1) if training with a lot of RAM or on small training data, as it uses a lot of memory but speeds up the training greatly (approx. by 30% to 50%). If you need to save RAM, turn it off (0).
1
Features codes to use in the unlabelled/labelled parser. See Treex::Tool::Parser::MSTperl::FeaturesControl for details.
These fields cannot be set by the config file, their default values are hard-coded at beginning of the source code and they can be set on creating the Config object, eg.:
my $config = Treex::Tool::Parser::MSTperl::Config->new( config_file => 'file.config', VITERBI_STATES_NUM_THRESHOLD => '5', EM_EPSILON => '0.00000001', )
If there is a need, they might be changed to fully config-file-configurable settings in future.
Algorithm used for Viterbi labelling as well as for training. Several possibilities are being tried out; if one of them finally significantly outscores the other variants, this will become obsolete and get deleted.
An integer specifying how much debug information you will be getting while running the program, ranging from 0 (no debug info) through 1 (progress messages - this is the default setting) through 2, 3 and 4 to 5 (more and more debug info).
If you set this value to something higher than 1, you should always redirect the output to a file as printing it to the console is very very slow (and there is so much info that you wouldn't be able to read anything anyway).
This the only read-write field in this section, it is therefore possible to change its value not only on creating a new Config instance but also while running the program (eg, if you only want to debug only a particular part).
This is only a technical thing; a label must be assigned to the (basically virtual) boundary of a sequence, different from any label used in the data. The default value is '###', so if you use this exact label as a valid label in your data, change the setting to something else. If nothing goes wrong, you should never see this label in the output; however, it is contained in the model and used for "transition scores" to score the "transition" between the sequence boundary and the first/last node (i.e. it determines the scores of labels used as the first or last label in the sequence where no actual transition takes place and the transition scores would otherwise get ignored).
Number of states to keep when pruning. The pruning takes place after each Viterbi step (i.e. after each computation of possible labels and their scores for one edge). For more details see the prune subroutine.
prune
Stopping criterion of EM algorithm which is used to compute smoothing parameters for linear combination smoothing of transition probabilities in some variants of the Labeller. (when the sum of change of smoothing parameters is lower than the epsilon, the algorithm stops).
A number between 0 and 1 specifying where in training data do heldout data for EM algorithm start (eg. 0.75 means that first 75% of sentences are training data and the last 25% are heldout data).
The training/heldout data division only affects computation of transition probabilities by MLE, it does not affect MIRA training or MLE for emission probabilities.
If EM is not used for smoothing, all data are used as training data.
Provide access to things needed in more than one of the other packages.
Provides access to unlabelled features, especially enabling their computation. Intance of Treex::Tool::Parser::MSTperl::FeaturesControl.
Provides access to labeller features, especially enabling their computation. Intance of Treex::Tool::Parser::MSTperl::FeaturesControl.
The best source of information about all the possible settings is the configuration file itself (usually called config.txt), as it is richly commented and accompanied by real examples at the same time.
config.txt
Reads the configuration file (in YAML format) and applies the settings.
See file samples/sample.config.
samples/sample.config
Fields are referred to by names in the config files but by indexes in the code. Therefore this conversion function is necessary; the other direction of the conversion is ensured by the field_names field.
field_names
Rudolf Rosa <rosa@ufal.mff.cuni.cz>
Copyright © 2011 by Institute of Formal and Applied Linguistics, Charles University in Prague
This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
To install Treex::Tool::Parser::MSTperl::Node, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Treex::Tool::Parser::MSTperl::Node
CPAN shell
perl -MCPAN -e shell install Treex::Tool::Parser::MSTperl::Node
For more information on module installation, please visit the detailed CPAN module installation guide.