Time::Progress::Stored - Report progress + store and retrieve the current status
This module helps if you have a long running process which reports progress, but you need to actually display the progress to the user in a different process.
Typically this is a long running web request in the web server or in a job queue worker, while the web browser periodically sends Ajax requests to get updated on the current progress status to show using a progress bar.
Time::Progress::Stored stores the progress report as the worker performs its job, and retrieves it from elsewhere where the report is needed.
The report is a hashref with the following details:
| id | "progress-1454518121172" | | max | 262 | | current | 2 | | activity | "Importing Users" | | elapsed_seconds | "1" | | elapsed_time | " 0:01" | | finish_time | "Wed Feb 3 16:50:54 2016" | | percent | 0.8 | | percent_string | " 0.8%" | | remaining_seconds | "130" | | remaining_time | " 2:10" | | is_done | 0 |
### In the worker process my $progess_id = ...; # Probably comes from the client my $max = @items * 2; # Let's say two passes over @items my $redis = Redis->new( ... ); my $progress = Time::Progress::Stored->new({ max => $max, progress_id => $progress_id, storage => Progress::Stored::Storage::Redis->new({ redis => $redis }), }); # Do the work for my $item (@items) { # Do stuff $progress->advance("Doing the first thing"); } for my $item (@items) { # Do other stuff $progress->advance("Doing other stuff with $item->{name}"); } $progress->done(); ### HTTP endpoint, e.g. a Mojo controller for /progress/:progress_id sub get { my $self = shift; my $progress_id = $self->param("progress_id"); my $storage = Time::Progress::Stored::Storage::Redis->new({ redis => $self->redis, }); my $response_json = $storage->retrieve($progress_id) or return $self->render(status => 404, json => {}); return $self->render(json => { progress => $response_json }); }
As the long running piece of work comes in, start the progress reporting.
Create the $progress object with a storage to store the progress reports. Initialize it with the number of actions you want to perform.
For each time something is done, call $progress->advance(). This might update the actual report.
By default only about 100 steps will be reported (each percent progressed). You can increase this by specifying the attribute "report_every", e.g. to report every single time something happens, set it to 1 (this might slow things down though).
"advance" can optionally be called with an activity description string, indicating what's happening right now. This can be large scale steps, or include details about this specific action being performed.
At the end, call $progress->done, just to make sure the reporting knows it has reached the end. This will set is_done: 1 in the report.
The client should send requests to the endpoint periodically to get the current progress report. The request contains the progress id, and it's simply fetched and returned as a JSON payload.
See above for the contents of the payload.
This is for when the long running process is running as one web request.
A web page would typically provide a progress id while making the initial Ajax request to kick off the long running process.
The client then sends Ajax requests to the HTTP endpoint (using the id) every second to get updates on the progress. It does this until the long running progress has responded.
If you look at the report payload you'll see there is a variety of information to provide user feedback, e.g. percent done, time left, what's going on etc.
This is for when the long running process is running as a background process, e.g. in a job queue.
A web page would typically kick off the long running process. Instead of the client passing in a progress id, the $progress object defaults to a unique id (or you can just make one up yourself), and the HTTP endpoint can report $progress->progress_id back to the client in the response.
The client then sends Ajax requests to the HTTP endpoint (using the id) every second to get updates on the progress. It does this until the progress report "is_done" is set to 1.
Time::Progress::Stored can use various backends to store the status.
Current backends are Redis and Memory. Memory is mainly for testing and not very useful for real life scenarios.
The backends are tiny and it would be simple to write another backend for e.g. a file, or using one of the Cache or CHI modules. Patches more than welcome.
If you're testing with a single threaded test server like Morbo, the web browser's Ajax requests won't be processed until the long running process has finished, so you'll never see the progress until it's all done.
The number of times you're planning to call ->advance().
A Storage object to store reports in.
Default: Time::Progress::Stored::Storage::Memory, but that's probably only useful for testing.
The id to identify this progress report with. Either set by the client, or defaulted to a unique URL-param safe value you can return to the client.
Number of calls to ->advance() for every time a new progress report is generated and stored.
Default: a value that results in a report for every percent of progress.
The current iteration. You could in theory set this yourself, but it's recommended to call ->advance().
Same as ->current, but for the current activity.
Advance the progress one iteration. Optionally set the current $activity.
Mark the progress as being completed, and set the is_done key in the report.
Johan Lindstrom, <johanl [AT] cpan.org>
<johanl [AT] cpan.org>
https://github.com/jplindstrom/p5-Time-Progress-Stored
Please report any bugs or feature requests on GitHub:
https://github.com/jplindstrom/p5-Time-Progress-Stored/issues.
Copyright 2016- Broadbean Technologies, All Rights Reserved.
This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
Thanks to Broadbean for providing time to open source this during one of the regular Hack-days.
This module uses the excellent Time::Progress module, which you should be using if you just need a progress bar in a command line script.
To install Time::Progress::Stored, copy and paste the appropriate command in to your terminal.
cpanm
cpanm Time::Progress::Stored
CPAN shell
perl -MCPAN -e shell install Time::Progress::Stored
For more information on module installation, please visit the detailed CPAN module installation guide.