SYNOPSIS: touch timestamp.file treex --watch=timestamp.file my.scen & # or without & and open another terminal # after all documents are processed, treex is still running, watching timestamp.file # you can modify any modules/blocks and then touch timestamp.file # All modified modules will be reloaded (the number of reloaded modules is printed). # The document reader is restarted, so it starts reading the first file again. # To exit this "watching loop" either rm timestamp.file or press Ctrl^C.
BENEFITS: * much faster development cycles (e.g. most time of en-cs translation is spent on loading) * Now I have some non-deterministic problems with loading NER::Stanford - using --watch I get it loaded on all jobs once and then I don't have to reload it.
TODO: * modules are just reloaded, no constructors are called yet
Treex::Core::Run + treex - applying Treex blocks and/or scenarios on data
version 2.20210102
In bash:
> treex myscenario.scen -- data/*.treex > treex My::Block1 My::Block2 -- data/*.treex
In Perl:
use Treex::Core::Run q(treex); treex([qw(myscenario.scen -- data/*.treex)]); treex([qw(My::Block1 My::Block2 -- data/*.treex)]);
Treex::Core::Run allows to apply a block, a scenario, or their mixture on a set of data files. It is designed to be used primarily from bash command line, using a thin front-end script called treex. However, the same list of arguments can be passed by an array reference to the function treex() imported from Treex::Core::Run.
Treex::Core::Run
treex
treex()
Note that this module supports distributed processing (Linux-only!), simply by adding the switch -p. The treex method then creates a Treex::Core::Parallel::Head object, which extends Treex::Core::Run by providing parallel processing functionality.
-p
Treex::Core::Parallel::Head
Then there are two ways to process the data in a parallel fashion. By default, SGE cluster\'s qsub is expected to be available. If you have no cluster but want to make the computation parallelized at least on a multicore machine, add the --local switch.
qsub
--local
create new runner and runs scenario given in parameters
Can't locate Treex/Core/Run.pm in @INC (you may need to install the Treex::Core::Run module) (@INC contains: /home/martin/perl5/lib/perl5/5.26.1/x86_64-linux-gnu-thread-multi /home/martin/perl5/lib/perl5/5.26.1 /home/martin/perl5/lib/perl5/x86_64-linux-gnu-thread-multi /home/martin/perl5/lib/perl5 /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.26.1 /usr/local/share/perl/5.26.1 /usr/lib/x86_64-linux-gnu/perl5/5.26 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.26 /usr/share/perl/5.26 /home/martin/perl5/lib/perl5/5.26.0 /home/martin/perl5/lib/perl5/5.26.0/x86_64-linux-gnu-thread-multi /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at bin/treex line 5. BEGIN failed--compilation aborted at bin/treex line 5.
Zdeněk Žabokrtský <zabokrtsky@ufal.mff.cuni.cz>
Martin Popel <popel@ufal.mff.cuni.cz>
Martin Majliš
Ondřej Dušek <odusek@ufal.mff.cuni.cz>
Copyright © 2011-2014 by Institute of Formal and Applied Linguistics, Charles University in Prague
This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
To install Treex::Core, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Treex::Core
CPAN shell
perl -MCPAN -e shell install Treex::Core
For more information on module installation, please visit the detailed CPAN module installation guide.