FleetConf - An multi-agent, configuration management tool
perl Build.PL install_base=/master/install/dir/FleetConf ./Build ./Build test ./Build install vim /master/install/dir/FleetConf/etc/fleet.conf /master/install/dir/FleetConf/bin/fleetconf -v /usr/local/Fleetconf/bin/fleetconf
This is a highly configurable, Perl-based, multi-agent, configuration management tool. It was originally designed by the author to handle the account management tasks for a mixed platform environment. In order to create accounts, he needed to create directory users on a Microsoft Windows Server 2003, create home directories and establish quotas on a Linux file server, add the user to mailing lists on another Linux mail server, monitor the process, and notify the user when the operations were complete.
This system has been further generalized to perform nearly any common configuration task needed to be performed across multiple systems. It's meant to perform actions similar to (and is being used by the author to replace) cfengine.
The front-end for the tool is fleetconf. Though, the real work is all started within the Perl module named FleetConf and through another set of Perl modules used by FleetConf::Agent. (Therefore, if FleetConf doesn't quite do it right for your needs, it should be a simple matter to modify fleetconf or create a new similar script to perform the same purpose.) For details on how to use fleetconf see it's documentation.
Essentially, the FleetConf system starts by determining the root directory of FleetConf. Using fleetconf, this is determined by moving upward to the parent directory that script is being run from. Within this root, the system opens and reads the contents of etc/fleet.conf, which contains some initial settings (see "CONFIGURATION" below).
Then, one or more plugin directories are loaded. Each plugin is a Perl file with the suffix ".pl" or ".pm" and may do whatever it likes to modify the internals of the system. Usually, this involves defining new agent language commands or defining helper functions (in the FleetConf::Functions namespace) or helper commands (in the FleetConf::Commands namespace). However, these can be used to modify any aspect of the published API they want.
FleetConf::Functions
FleetConf::Commands
Following this, a series of "workflows" are loaded, as defined in the configuration file (and possibly plugins). Agents use the workflows to determine what to do and also to record their progress in doing it.
Next, one or more agent directories are loaded. All files ending in ".agent" are read using the agent parser. The agent parser compiles every agent script found. Each agent is defined in a fairly simple command-based language that can be extended using plugins. For a full definition of the language see FleetConf::Agent::Parser, and any plugins you have loaded (the included plugins are linked from FleetConf::Agent::Parser).
Finally, each agent is run. Each agent runs over each record found in the given workflow system and may perform an action permitted by the agent language. Each iteration involves running the agent in 5 phases (plus a phase before iteration begins). It is likely that future versions will feature an additional phase at the end and perhaps other phases. For a full description of running and the phases see FleetConf::Agent. For a description of which commands run in which phases, you'll need to see the FleetConf::Agent::Parser and command plugin documentation.
The first step, of course, is to install it. Then, you will need to configure it and write some useful agents. Then, you need to run it all over the place to copy it locally.
For information on installation, see "INSTALLATION". For information on configuration, see "CONFIGURATION". For information to get started with agents, see FleetConf::Agent.
The public API is documented to allow contributors to learn about the system, for users to create plugins that take full advantage of the API, and to keep the author from forgetting what something is supposed to do.
In generally, I try to keep the documentation up to date with the actual program, but there will be lapses. Please let me know about them by filing a bug at http://rt.cpan.org/ if you discover such a lapse.
The FleetConf object exists as a singleton created sometime after startup. (For those who may not know, a "singleton" is approximately what it sounds like: it is an object for which there can be only one for each Perl process. That means that there should only ever be one of these objects-per-run of fleetconf.) This singleton is accessed (and, during the first call, created) by calling the instance method of FleetConf.
instance
Prior to calling the instance method you will probably want to configure the global variables, which are used to determine certain parts of the configuration.
NOTE: Please be aware that these variables may change. These were an easy mechanism of communication between fleetconf and FleetConf originally, but aren't very good solutions. Future revisions may include accessors from the FleetConf singleton so that their effects can be modified after startup. Currently, if you set these after the instance method is called, that change may have no effect. However, it's likely that I will keep these around and use some devious tieing to allow them to be effective. So perhaps my warning doesn't matter anyway.
tie
This documentation shows the fully qualified name of each global variable and the initial/default value of that variable (if any).
The verbose setting is used to determine how much information to output while running. All output should be routed through the logger (see $FleetConf::log below), which is configured according to the value of this setting when initialize is called.
$FleetConf::log
initialize
When this setting is set to "0", the default log level of "warning" or "error" is used, meaning that anything below that threshold is not output. (See $FleetConf::no_warnings for more information.) If verbose is set to "1", then the log level is set to "notice". If verbose is set to "2", the log level is set to "info". If verbose is set to "3", the log level is set to "debug". Higher levels will result in greater amounts of debug information being displayed. At this time, I believe level "4" is the highest implemented debug level, but more might be used in the future.
$FleetConf::no_warnings
All plugins should be careful to check the value of this variable. If this value is set to a true (non-zero) value, then no changes should be made to the system, the workflow, or anything. Plugins and commands should merely report back as if they were doing that work (or may log messages stating that we're pretending to do something, whatever is most appropriate).
This flag allows the user to safely run and see what would happen without actually causing anything to happen.
Setting this value to will cause verbose to be set to at least a value of "1" (so that log level is set to "notice").
In some cases, it might be desireable to hide warnings from standard output. Setting this to a true (non-zero) value will mean that the log level will be set to "error" rather than "warning" as long as neither $FleetConf::verbose nor $FleetConf::pretend are non-zero.
$FleetConf::verbose
$FleetConf::pretend
This value is very important to set if you are writing an alternative to the fleetconf front-end. This tells FleetConf which directory to use as the basis for all relative file names and where to find the configuration file (i.e., etc/fleet.conf inside of this directory).
it defaults to the current working directory of the current Perl interpreter, but this is a poor default.
This log mechanism is deprecated. Please use FleetConf::Log instead. This is still used, but only internally.
Name the log file to log to. If none is given, then logging only goes to standard error. This file will be appended to rather than overwritten.
If this is set to a false value, then logging will not be printed to the terminal on standard error.
This global is setup initially from the "globals" configuration option. It may contain any information a plugin or agent wants to store here. It may be modified by any agent or plugin and those changes stay until changed by a later agent or plugin or when FleetConf quits. The contents of this variable are not saved between runs of FleetConf.
The most important method is the instance method, which returns the singleton FleetConf instance that is used to call most of the other methods.
This method returns the singleton reference to the FleetConf object. This is the starting point for all things FleetConf in the API. See the rest of the method documentation below to find out what you can do with it.
The first time this method is called, it will perform the first four phases of loading based upon the globals (see "GLOBAL VARIABLES" above).
Before doing anything it will determine the log level that should be used to output to the screen (via standard error). It then resets the logger to use this log level on the screen (prior to this call, the logger will output anything to the screen as it's rare that it should be used at all prior to this point).
It also performs a late loading of the FleetConf::Agent library (which shouldn't, in general, be loaded directly by any plugin or application). This late loading happens so that we can capture the output of the parser trace for debugging purposes.
Loading the configuration happens next. See "CONFIGURATION" for details on what is stored there.
We search all directories specified by the plugin search path for any file ending in ".pl" or ".pm" and we run that file as if it were Perl. We report as errors or warnings any plugin that fails to load correctly, but we otherwise ignore them.
Most of the time, FleetConf ignores errors and continues despite them because it's possible that unrelated parts of the system should run anyway. If an error should stop some other part of the system from running a plugin should make sure and fail cleanly itself and alter the state of the system somehow to prevent the other system from failing (the %FleetConf::globals would be a good place to do this).
%FleetConf::globals
Based upon the configuration in the "workflows" variable, we will load workflow instances into the system. Agents then request named workflow instances to determine how they proceed.
One workflow instance is ubiquitous and loaded regardless of any other configuration. This workflow is called "Null" and is defined as a basic instance of FleetConf::Workflow::Null. This module basically always returns a single empty workflow record and uses the $FleetConf::log logger to record changes. See FleetConf::Workflow::Null for more information.
Finally, we search all of the agent include directories for files ending with ".agent" and compile the agents there according to the load_file method of FleetConf::Agent. (See that module for more documentation on this process.)
load_file
At this point everything that will be loaded is loaded, but (unless a plugin does something) no action should have been taken yet. The returned object is ready to continue.
This method can be used to fetch the configuration file after it is read in. If this method is not called on a FleetConf instance, the instance method will be called to get one and the configuration method called on that (i.e., don't call this if you aren't ready to call instance).
For details on what this variable will contain, see "CONFIGURATION".
This method allows the user to specify which agents should be run. In some cases, it may be desireable to limit which agents are run externally (i.e., agents can internally refuse to run based on the situation as well). If only some of the agents should be run, then this method should be used to select the agents.
Each element of @selectors is a subroutine (code reference) that takes an FleetConf::Agent object reference as it's only argument. An agent will be used if any selector returns a true (non-zero) value when pased that agent.
@selectors
This call affects any call to run_agents following until this method is called again. Each call to this method replaces the entire list of selectors. Calling this method with no arguments will cause all agents to be selected if immediately followed by a call to run_agents.
run_agents
This method returns a list of agents that are currently selected (according to the last call to select_agents) or all agents if no selectors are set. See select_agents for information on how to select agents.
select_agents
Runs all the agents that have been selected (i.e., those that would be returned by agents). This basically means that each agent is loaded and the run_foreach method is called with the requested workflow passed.
agents
run_foreach
The configuration is stored as a YAML file. Please see the documentation of YAML for information on how YAML works. The top-level of the FleetConf configuration file, fleet.conf, is a hash containing keys pointing to the different configuration directives. At this time, there are very few directives.
For a basic and simple configuration file illustrating these directives, see the file etc/fleet.conf.sample included in the distribution.
Each are described here:
This is an array containing the name of the directories to search for agents.
For example:
agent_include: - agents - /etc/FleetConf/agents
Here, all agents found in the directory or subdirectories of agents in the FleetConf root directory will be searched first and then those found in /etc/FleetConf/agents will be loaded second.
This is an array containing the name of the directories to search for plugins.
plugin_include: - plugins - /etc/FleetConf/plugins
Here, all agents found in the directory or subdirectories of plugins in the FleetConf root directory will be searched first. Then, those found in /etc/FleetConf/plugins will be leaded second.
This option contains a hash of the all the workflows to load. Each hash key is the name of the workflow instance. Each of these should then be a hash containing two keys, "class" and "args". The "class" key is a scalar naming the Perl class that will be used to create the workflow instance (and should already be loaded as a plugin). The "args" will be set to whatever the new method of the given "class" expects to set it up.
new
workflows: new_jobs: class: FleetConf::Workflow::RT args: proxy: https://example.com/cgi-bin/rt-soap-server.pl query: Queue='Jobs' AND Status='new' ssl_client_certificate: /etc/ssl/client/rt-workflow.crt ssl_client_key: /etc/ssl/client/rt-workflow.key open_jobs: class: FleetConf::Workflow::RT args: proxy: https://example.com/cgi-bin/rt-soap-server.pl query: Queue='Jobs' AND Status='open' ssl_client_certificate: /etc/ssl/client/rt-workflow.crt ssl_client_key: /etc/ssl/client/rt-workflow.key
This will instantiate two workflow instances: one named "new_jobs" and another named "open_jobs". If a workflow requires more complicated work to instantiate it, that work may be done with a plugin (though, the way to do that isn't part of teh published API yet as this isn't necessarily safe if you upgrade in the future).
This option sets the initial value of %FleetConf::globals. Thus, it should, obviously, be a hash.
globals: master_path: /common/admin/FleetConf local_path: /usr/local
This would set $FleetConf::globals{master_path} to "/common/admin/FleetConf" and $FleetConf::globals{local_path} to "/usr/local".
$FleetConf::globals{master_path}
$FleetConf::globals{local_path}
Installation follows the typical Module::Build template with one important caveat. FleetConf is meant to operate in a more controlled environment, so install_base is automatically set to /usr/local/FleetConf. That is, by doing the typical (such as installing with CPAN):
install_base
perl Build.PL ./Build ./Build test ./Build install
The files will all be dropped within /usr/local/FleetConf and subdirectories within that folder. It is strongly recommended that you install FleetConf into it's very own directory, which is why you will receive a complaint if you try to install into a directory not named "FleetConf", just to try and make sure you're aware of this if you have not yet read the docs.
Thus, the more typical installation will look like this:
perl Build.PL install_base=/some/master/root/FleetConf ./Build ./Build test ./Build install
Just in case you hate using capitals, no complaint will occur if you install using all lower case, e.g., /usr/local/fleetconf.
After installation, you will need to create a file named etc/fleet.conf inside the installation directory and add any agents you like.
At this point, one agent is installed automatically, update.agent, which will update a local installation from a global one. Delete this if you don't want it. Future versions might include this as an update.agent.sample or something similar. You will need to check the agent folder and make sure you have all the agents you want there. You may also want to add plugins.
The rest is up to you. At this point, FleetConf has no standard policies on how things should be done. This will likely remain the case until either I find something useful, or a user community starts choosing those policies. I have no idea what would be best in general or what tricks might be handy.
FleetConf::Agent, FleetConf::Agent::Parser, FleetConf::Commands, FleetConf::Functions, FleetConf::Workflow::Null, FleetCOnf::Workflow::RT, Log::Dispatch, Module::Build, CPAN
Andrew Sterling Hanenkamp, <hanenkamp@users.sourceforge.net>
Copyright 2005 Andrew Sterling Hanenkamp. All Rights Reserved.
FleetConf is licensed and distributed under the same terms as Perl itself.
To install FleetConf, copy and paste the appropriate command in to your terminal.
cpanm
cpanm FleetConf
CPAN shell
perl -MCPAN -e shell install FleetConf
For more information on module installation, please visit the detailed CPAN module installation guide.