NAME
Parallel::Dragons - Daemon are forever... Dragons lay eggs, grow fast and die in flames!
SYNOPSIS
use parent qw'Parallel::Dragons';
sub max_childs { 5 } # max 5 workers at a time
sub wait_before_restart { 10 }
sub main {
my $i = 0;
while ($i++ < 100) {
print "$$: counted to $i\n";
sleep 1;
}
print "counted to 100, going away\n";
}
main::->new()->run();DESCRIPTION
Parallel::Dragons is a framework to create long lived processes that run the same code all the time, in some cases multiple times in parallel.
It doesn't know, and doesn't care about what the code does. Even when used in queue driven mode, it doesn't care about the queue, leaving the implementation of the queue access to the application itself.
USE
Parallel::Dragon was written to be the parent of you daemon class.
The minimal Daemon you can create implements only the sub main, which does the job you need done. In this case max_childs will default to the number of CPUs in your server and a new child is created every 3 seconds until that number is reached.
The method new doesn't take any parameters, but the class handles @ARGS to accept several command line commands and options.
Most of the time, you want to call $your_class->new()->run() and let Parallel::Dragons take care of everything for you.
Unlike what happens with most libraries, you don't call methods in Parallel::Dragons - Parallel::Dragons calls methods you write for it.
Parallel::Dragons was created with the idea that by defining specific methods in your class you can change the way your Dragon behaves. We call those methods hooks, and that's what the next section is about.
Parallel::Dragons works by having a main process that starts childs to do all the work - the main process only stops when a small set of hooks fails with error.
HOOKS
The hooks as listed in the order they are usually called. Some of them are called in the main process, some of them are called in the child process - this list tells you all you need to know.
max_childs
MAIN: Called during the init phase of the main process - it's assumed to be a constant - see needs_childs for a more dynamic hook. Your dragon will never create more than <max_childs> at the same time.
Params: $self
wait_before_restart
MAIN: called during the init phase of the main process - it's assumed to be a constant and represents the number of seconds that the main process waits between starting two childs and between the end of a child and starting a new one when the max_childs is reached.
See restart_next for a more dynamic way to decide when to start the next child.
Params: $self
restart_after
MAIN: if your Dragon uses a <queue> based model, this is the number of time a child will ask for tasks before ending. This is called during the init phase of the main process and is assumed to be a constant.
Params: $self
max_memory_per_child
MAIN: this is called during the init phase of the main process and is assumed to be a constant. It's used only in Dragons that use the <queue> model - when a child ends executing main for a task, it checks whether it is using more than this amount of memory and ends if it is.
Params: $self
pre_start
MAIN: called before the Dragon is started.
Note: pre_start can prevent the Dragon from starting by calling $self->stop
Params: $self
need_childs
MAIN[in the loop]: it's called every time it's time to start a new child to decide if a child needs to be created.
Params: $self
pre_fork
MAIN[in the loop]: the decision to create a new child was made, this allow you to set any data you need to pass to the child.
Params: $self
post_child_start
MAIN[in the loop]: called in the main process imediately after a new child is created.
Params: $self, $child
restart_next
MAIN[in the loop]: called after a child is started and is expected to return an unix timestamp for when the next child should be started. The child will only be started if there are less than max_childs running.
Params: $self
init_child
CHILD: called when a new child is created.
Params: $self
data_consumer
CHILD: expected to return a Data::Consumer compatible object. If this method is used, the Dragon will run in <DC> mode. In this mode the method main will be called as a (modified) consume call back. See main.
Params: $self
claim_task
CHILD: expected to return a <task> that main will be able to process. If claim_task is used, the Dragon will run in <queue> mode. In this mode main will be called with the task return by claim_tasks.
If claim_task returns false, the child will exit - and a new one will be created by the MAIN process when time for that comes.
Params: $self
Note on data_consumer and claim_task
If neither of those methods exists, main will run once for each child and the child will end after that. We call that <single run> mode.
main
CHILD: this is the body of the child - all the other methods are intented to setup the work to be done in this method.
Depending on which mode the Dragon is running in, the parameters will differ, but a single Dragon will always recieve the same type of parameters - meaning, all the childs of a given Dragon will always run in the same mode, as the mode depends on the existence of the methods data_consumer and claim_task.
Params:
- DC mode: $self, $id
-
The $id is the $id claimed by Data::Consumer - your main method should be able to handle getting the task from that id.
- queue mode: $self, $task
-
the $task returned by claim_task
- single run more: $self
task_done
CHILD[queue mode]: called when the child finishes processing a task and the main method ended without errors. if the main method dies, task_failed will be called, instead of task_done.
task_done is only called in queue mode.
Params: $self, $task
task_failed
CHILD[queue mode]: called when the main method dies.
Params: $self, $task
post_child_exit
MAIN[in the loop]: when a child ends, post_child_exit is called. This allows you to clean up any data you may have in the main process specifically for that child.
Params: $self, $child
pre_exit
MAIN: Called just before the process ends - allows you to run any clean up you main want.
Params: $self
ADDITIONAL HOOKS
This two hooks are not related with the workflow of managing child processes. However, this hooks are still running in the loop of managing childs, so they should be as efficient as possible.
monitor
MAIN[in the loop]: monitor is called once a seconds in the main process.
Params: $self
idle
MAIN[in the loop]: idle is called in the main process when no child as ended, monitor didn't run and no child was created in that cycle.
Params: $self
COMMANDS
All the Dragons support several command line commands. You run a command by calling your dragon followed by that command. Example:
<dragon.pl> start
start
Starts the Dragon as a dameon - it detachs from the current terminal, create log files and socket file in /tmp. Before starting a new Dragon it checks if there is a Dragon running and if it is responsive.
A normal thing to do, if you want your Dragons to never stop running is to have a minutely cron starting then - if they are already running nothing will happen.
startfg
Runs the dragon in the foreground, as a single process. Useful to test and debug your process. It still calls all the relevant methods, except for post_child_start and post_child_exit.
stop
Stops a Dragon started with start. This uses the socket file to tell the dragon to stop. The stopping process is graceful, which means that this command tells the MAIN process of the running process that you want it to stop, and it tells all their childs to stop when they are done with their corrent task, which may take some time, depending on how long each of you tasks takes to finish.
ping
Send a ping command to the MAIN process of a running. If there is a Daemonized Dragon running, it will respond with pong.
info
Sends a info request to the MAIN process of a running Dragon and gets some information about it's status - this will probably be extended in the future.
ENVIRONMENT VARIABLES
This variables are useful when running start or startfg. The control the amount of information you will see from the Parallel::Dragons itself.
DRAGONS_DEBUG
Expected a values between 0 and 10. The relevant levels are:
DRAGONS_CRONJOB
Expected to be a true/false (1/0) value. It disables information about other or not a Dragon was already or a new one was started.
Useful for cronjobs - may be replaced in the future with a check of a TTY in STDIN/STDOUT
SEE ALSO
There are a lot of other perl packages to manage parallel execution of code - depending on your use case some of them may be better than Parallel::Dragons. This is just another option.
BUGS REPORTS and FEATURE REQUESTS
Please report any bugs , or request features in:
* github: https://github.com/themage/perl-parallel-dragons/ * Magick Source: http://magick-source.net/project/view/10/
AUTHOR
theMage <themage@magick-source.net>
COPYRIGHT AND LICENSE
Copyright (C) 2016 by theMage
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.22.1 or, at your option, any later version of Perl 5 you may have available.
Alternativally, you can also redistribute it and/or modify it under the terms of the GPL 2.0 licence (or any future version of it).