NAME

ponfishd - The Ponfish command daemon

USAGE

  ponfishd [-p]
           [-queue_size NUM_COMMANDS]
           [-check_time SECONDS]
           [-connections NUM_THREADS]
           [-l] [-log_file LOG_FILE]
           [-kill_lock]

OVERVIEW

Ponfishd is the Usenet daemon for ponfishr. It's job is to read and execute commands issued by ponfishr.

Ponfishd creates multiple threads and dispatches commands to these threads, allowing multiple commands to be executed at the same time.

Options

-p

Preview mode - only execute "sync" and "preview" commands. This allows the user to more easily preview files.

Note that sync and preview command filenames all begin with "00000000", the highest priority. While these would normally be executed first, if the daemon is busy executing normal commands, the normal command(s) would need to complete before a thread could execute the higher priority one. This prevents normal commands from executing, thus allowing previews and syncs to execute quickly.

-connections NUM_THREADS

The number of threads to spawn - the threads all execute different commands simultaneously. Recommended value: 3 to 5. You might find higher numbers to be more efficient if you have a very fast network connection.

-check_time SECONDS

The number of seconds to wait between checking for new commands. Each command is created as a file on the disk. This file contains command information. For example, a "decode" command would create a file instructing ponfishd what to grab from the news server, what sequence to process the command in, and the target directory for the file decoded, among other things.

Checking for new commands can take time, depending on disk speed and the number of commands there are pending. I recommend 10 seconds (the default) as a minimum.

-queue_size NUM_COMMANDS

The number of commands to put into the queue at a time. The threads all take commands from the queue, and the queue is replenished every -check_time seconds. This option is here only for flexibility purposes.

One example may be that you have 100_000 commands pending, and you do not want to check the filesystem every 10 seconds, so you might do something like this:

> ponfishd -check_time 1000000 -queue_size 100000

This will load the queue up with every command, and then never again check for new commands.

Also, if you have a particularly fast network connection, it is possible that your threads can empty the queue before it can be replenished. To remedy this you can increase the queue_size. (or decrease the check_time, but it's not recommended)

The -connections, -check_time, and -queue_size options are all interrelated.

-kill_lock

If a lock exists, this will remove the lock file. It will also try to kill the PID that's in the lock file. The program will exit after removing the lock file, and you will have to run it again normally.

-l

Enable file logging.

-log_file FILENAME

Log to file FILENAME

The Anatomy of a Command File Filename

The filename consists of four parts separated by a space:

DATE ARTICLE_ID NEWSGROUP SERVER

DATE:

Perl's numeric equivalent of the date of the post. The commands are queued up for execution in order of the DATE, allowing the oldest ones to be decoded first. This ensures that we do not decode newer posts while the old posts potentially expire from the news server.

In the case of sync and preview commands, the DATE will be "00000000", forcing them to the top of the queue to be executed first.

ARTICLE_ID:

This is required to ensure each command has a unique filename. ARTICLE_ID is the first article in the post. Each post in a particular newsgroup has a unique sequentially assigned article id. Multiple posts may have the same post DATE

NEWSGROUP:

The name of the newsgroup.

SERVER:

The server name.

Ponfishd uses the SERVER field from the filename, and possibly the NEWSGROUP as well, so do not rename any part of the command file other than the DATE.

COMMAND FILE / DOWNLOAD MANAGEMENT

Decodes and saves are ordered by date such that the oldest posts get executed first. This ensures the posts in danger of expiring from the news server get downloaded first.

In actuality, the queue is ordered by command filename. The command filename begins with a numeric date, or 00000000 for high-priority commands, which ensures they go to the head of the queue.

THE ANATOMY OF A COMMAND FILE

The command file is simply a text file which contains a command and any arguments that the command might use.

Take, for example, a decode command. The command would be "decode", the arguments would be a list of one or more article IDs that comprise the post, an optional destination directory, and the subject line and newsgroup.

A sync command would only require a newsgroup and server name as arguments.

BUGS

None. (Just kidding)

TO DO

* The decoder converts some special characters to underscore, I should prevent this behavior.

* Would like to be able to switch to and from preview mode without restarting.

* Would like option of all threads tackling one download at a time.

* It could use better messaging - the output as the daemon runs is informational, but not nice or easy to look at.

SEE ALSO

The ponfishr man page

AUTHOR

Desmond Lee <gmail: desimus.prime>

cpanm Ponfish::Config

perl -MCPAN -e shell
install Ponfish::Config