Text::Editor::Easy::Comm - Thread communication mecanism of "Text::Editor::Easy" module.
Version 0.3
There are 2 (or 3 if we include the "Text::Editor::Easy::File_manager" module) complex modules in the "Text::Editor::Easy" tree. This module and the "Text::Editor::Easy::Abstract" which handles graphics in an encapsulated way.
This module tries to make thread manipulation obvious with "Text::Editor::Easy" objects. Maybe this module could be adpated to be used with other objects to facilitate thread creation and use. This is not my goal : my goal is to write a generator of applications that can be modified dynamically (the Editor is the first part of that).
There are 2 main classes of threads : server and client. A client thread is, for instance, your program that runs sequentially and, from time to time, ask a server thread for a service. A server thread is a waiting thread that manages a particular service. From time to time, it's called by a client (which can be a real client thread or another server thread : the calling server thread can be seen here as a client for our responding server), responds to the client and then waits again. Of course, if the server is saturated with calls, it won't wait and will execute all the calls in the order they have been made. So, the clients (real or other servers) may have to wait for the response of the server... but not always. Here come asynchronous calls : in an asynchronous call, the client asks for something to the server (gets, if it wants, an identification of the call, the "call_id"), and can go on without waiting for the response. But asynchronous calls are not always possible. Often, you have to make things in a certain order and be sure they have been made before going on. So most calls to server threads (by client) will be synchronous and blocking.
Now that we have seen the 2 classes of threads let's talk more about server threads. There are mainly 3 types of server threads : owned by an instance (let's call it OWNED thread), shared by all the instances with separate data for all the instances (let's call it MULTIPLEXED thread), shared with all instances with no separate data (let's call it CLASS thread). All these types of threads haven't been invented for theorical beauty, but just because I needed them. The OWNED thread is the "File_manager" thread : each "Text::Editor::Easy" instance have a private one. The MULTIPLEXED thread is the graphic thread (number 0) : Tk is not multi-threaded, so I had to put private data in only one thread. All other threads that I use are CLASS threads : the thread model, number 1, that is only used to create new threads, the "Data" thread number 2, that shares common data such as "call_id" and asynchronous responses...
The thread system allows me to create all the types of threads defined previously (OWNED, MULTIPLEXED, and CLASS) but it allows me more. First, there is no real limit between the 3 types of threads (I can have a thread with a MULTIPLEXED and CLASS personnality... or any other combination). Second, I'm able to define dynamic methods and have access to the code of all the methods to enable dynamic modifications. The demo8 of version 0.01 gave me ideas about what I needed to increase my productivity in developping my Editor.
The "create_new_server" method can be called either with an instance, or with a class :
my $tid = $editor->create_new_server ( {...} );
or
my $tid = Text::Editor::Easy->create_new_server ( {...} );
For an OWNED or MULTIPLEXED type, use the instance call. For the CLASS type use the class call. "create_new_server" uses a hash reference for parameters, and returns the "tid" ("thread identification" in the interpreted thread perl mecanism), which is an integer. This interface may be changed : just given to see actual capabilities. Of course, the more I use this interface to create all my threads, and the more I will be reluctant to change the interface.
Here are the parameters of the given hash to the "create_new_server" method :
$editor->create_new_server ( { # optionnal, without this option, no new module is evaluated by the new created thread 'use' => 'Module_name', # optional, without this option, the "main" package is used for the following methods 'package' => 'Package_name', # mandatory : methods to be added to the instance or to the class. # Calls to these methods will be served by the new created thread. # The names of the methods must correspond to the name of the subs to be called. # This limitation may be suppressed later (by including a hash ref in the elements # of the array) 'methods' => [ 'method1', 'method2', 'method3', ... ], # optionnal but either 'object' or 'new' must be provided. # 'object' or 'new' specify the first parameter that the methods handled # by your thread will receive (the other parameters will be given by # the call itself : for instance "$editor->method1('param2', 'param3', ...)" 'object' => ... (any "dumpable" reference for instance [], or {} ...), # optionnal but either 'object' or 'new' must be provided. # Method that will be called first by the new thread : # - must return the object that the thread will use for the methods call (first parameter) # - the return value of the method can contain non "dumpable" data (sub reference, file descriptor, ...) # The first value of the tab reference is the sub name (including the package), # the other values are "dumpable" parameters to be sent to the sub returning the object. # The first parameter received by 'package::sub_name' will be 'param1', the second 'param2' and so on 'new' => [ 'package::sub_name', 'param1', 'param2', ... ], # optionnal, gives a sub that will initialize the object. # The sub does not need to return the object because the reference is given. # Be careful, the first parameter received by 'package::sub_name' is the reference # of the object that will be used when calling the newly defined methods (this is # what should be initialized) # AND THE SECOND PARAMETER is the "pseudo-reference" with which the "create_new_server" # has been called (either "a unique reference" for an instance call which is an integer that # uniquely identifies a "Text::Editor::Easy" object, or the class name) init => [ 'package::sub_name', 'param3', 'param4', ...], # optionnal (allows multiple OWNED threads to share code (one different tid for each instance) 'name' => 'thread_name', # optionnal, put the tid of the thread in the shared hash %get_tid_from_instance_method # even if a name is given (could be used for MULTIPLEXED thread) 'put_tid' => 1, # optionnal, indicates that the code of the methods won't be shared with other instances. # May be used for specific OWNED thread. # In short, some methods may be have the same name but different associated code according # to the instance calling the method 'specific' => 1, # optionnal, indicates that no thread will be created, the calling client (true client) # will have to manage itself the defined methods (the client thread will have to # use "anything_for_me" and "have_task_done" methods exported by "Text::Editor::Easy::Comm" # to respond to potential clients; it could also use "get_task_to_do" and "execute_this_task" # instead of "have_task_done" to have a better control over the client calls : see # 'Text::Editor::Easy::Abstract::examine_external_request' for that). # The tid returned by the "create_new_server" will be the tid of the calling client # This option and the desire to create interruptible methods are the only 2 reasons why you # should include "Text::Editor::Easy::Comm" in a private module 'do_not_create' => 1, } );
Once your thread is created, you can change a little it's behaviour with the following calls. Again, you can use instance call or class call.
Text::Editor::Easy->ask_thread('add_thread_method', 'tid', { ... } ); $editor2->ask_thread('add_thread_object', 'tid', { ... } ); $editor2->ask_thread('any_function_you_want', 'tid', 'param3', 'param4', 'param5' );
The difficult task achieved by "Text::Editor::Easy::Comm" module is to ask the good server thread for the "Text::Editor::Easy" method-call that you've made (either instance or class call). As long as you provide the "tid" of the thread you want to ask for something when using "ask_thread", you can specify anything for the method, even if it hasn't been declared as a method : still the fully qualified method should be known by the thread (in a package contained by 'main' or by any other module that has already been evaluated by the thread : either with "create_new_server" and 'use' option or by the "add_thread_method" and the 'use' option).
The 'add_thread_method' allows you to define a new method for the thread and not necessarily for the same reference that was initially used for the thread creation. This "reference change" can modify slightly the "personnality" of your thread. The possible options for the hash are :
'use' (evaluation of a new module for this method) 'package' 'method' 'sub' (if the sub of the package has a different name of the method) 'memory' (for dynamic designing : the code of the method to be executed is given and not on a file)
The 'add_thread_object' method allows you to add new objects in a MULTIPLEXED thread. The possible options for the hash are :
'object' (see "create_new_server") 'new' (see "create_new_server")
You don't add a new thread, but you ask an existing thread to handle a new instance with the same default methods that have been defined by "create_new_server" (and possibly added by "add_thread_method" if the reference was the same as the first initially used with "create_new_server").
You may have a look at the tests included with "Text::Editor::Easy" if you want to understand by practise these explanations (for instance, "multiplexed_without_thread_creation.t" is a good example for asynchronous calls and a client that acts as a server).
There are only 2 reasons to include this module in a private module of yours. Either you've created a pseudo-server with the "do_not_create" option during "create_new_server" call, or (a little more interesting), you want to create a server with a "lazy behaviour" : that is to say, which implements interruptible tasks.
As communication between threads uses the "Thread::Queue" mecanism, you can know if there is something waiting for you in the "Queue". I encapsulate the "pending call" to the queue object in the "anything_for_me" function which does not accept any parameter. "anything_for_me" returns true if there is another task for you, false otherwise. If you look at my code, you'll see, in lots of my lazy graphical methods, the
return if anything_for_me;
This little line is magical. Before beginning a new heavy instruction, you check if something could invalidate your processing. Then, you may see another reason to create thread : if you can separate the functions that lead to give up other ones, you can put these functions in a single thread and write them in "lazy mode". No matter the memory used ! It's now cheap. But take into consideration the time of your user : it's the most precious thing.
Another useful function exported is "have_task_done". Used with "anything_for_me", this allows you to implement "interruptible long task". For me, what I have in mind is "counting the lines of a file that may be huge", or "parsing a file for contextual syntax highlighting". I really needed that to go further in the development of my Editor. "have_task_done" does not accept any parameter. For the moment, it returns "true" only if the thread should be stopped ("stop_thread" method already called). So, to create an interruptible long task, you could write, from time to time, during your long process :
while ( anything_for_me ) { return if ( have_task_done ); }
The difference between "return if anything_for_me" is that your thread still is in your long interruptible task after having done a few more urgent tasks. More over, you don't lose any of the values of your variables before the call. Except the "main shared object" that some of your methods normally shares (it could have been modified by the urgent calls), nothing should have been changed.
You can make your interruptible tasks a little more complex using "get_task_to_do" and "execute_this_task" as a replacement of "have_task_done". If there is anything for you, you can get what is really for you with "get_task_to_do". The first parameter returned is the method called, the second the "call_id". An example of this is the "examine_external_request" sub of the "Text::Editor::Easy::Abstract" module. This allows my module to know if we're working responding to a graphical event or responding to a client call. This is used to test "event conditions" before calling user callback after an Editor event occurs. This mecanism has still to be improved (more events have to be added and tested...).
"anything_for_me" can tell one thread if there is something more to do for itself. With "anything_for" and the tid of the thread given, any thread can know about the queue of any other thread. This is used to propagate lazyness between threads : a running task can be stopped if the change in another thread invalidates it.
ask_thread requires to know the tid of a thread. But an arbitrary number is not what we'd like to use.
Return the reference (unique) of a Text::Editor::Easy instance.
This sub is called only in the graphic thread context (with the thread that has tid 0). It initializes new graphical data for the new "Text::Editor::Easy" object as well as reference this object for thread communication.
Quick mecanism to display or to hide debug information from a package / thread. Don't need to remove all the "print DBG" everywhere.
Every AUTOLOAD calls are traced. Still some important methods (instance creation, thread creation, ...) are not yet traced. To be done.
Sebastien Grommier, <sgrommier at free.fr>
<sgrommier at free.fr>
Besides the numerous bugs installed maybe for years in this code, there is one which I think about. It's deadlock. This, of course, is not a bug of my module, but a bug in the conception of the server thread calls. For the moment, I just print "DANGER client '$client' asking '$method' to server '$server', already pending : $thread_status" when a thread is called whereas it's already busy. See "Text::Editor::Easy::Data" for this warning. Well, in the future, I will handle deadlocks : when circular calls will be noticed, I will give up one of the call made to free other calls. Of course, the response of the given up call will be "undef" and not correct. But you're supposed to test it and it'll be always better than not to respond any more to any other requests... Still, this situation should never happened in a good thread organisation (I sometimes had to debug deadlocks for now, but not really much : I often use asynchronous calls to avoid that).
Copyright 2008 Sebastien Grommier, all rights reserved.
This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
To install Text::Editor::Easy, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Text::Editor::Easy
CPAN shell
perl -MCPAN -e shell install Text::Editor::Easy
For more information on module installation, please visit the detailed CPAN module installation guide.