Provided by: libminion-perl_10.25+dfsg-4_all bug

NAME

       Minion::Guide - An introduction to Minion

OVERVIEW

       This document contains an introduction to Minion and explains the most important features
       it has to offer.

INTRODUCTION

       Essentials every Minion developer should know.

   Job queue
       Job queues allow you to process time and/or computationally intensive tasks in background
       processes, outside of the request/response lifecycle of web applications. Among those
       tasks you'll commonly find image resizing, spam filtering, HTTP downloads, building
       tarballs, warming caches and basically everything else you can imagine that's not super
       fast.

         Mojo::Server::Prefork                              +--------------+                     Minion::Worker
         |- Mojo::Server::Daemon [1]       enqueue job ->   |              |   -> dequeue job    |- Minion::Job [1]
         |- Mojo::Server::Daemon [2]                        |  PostgreSQL  |                     |- Minion::Job [2]
         |- Mojo::Server::Daemon [3]   retrieve result <-   |              |   <- store result   |- Minion::Job [3]
         +- Mojo::Server::Daemon [4]                        +--------------+                     |- Minion::Job [4]
                                                                                                 +- Minion::Job [5]

       They are not to be confused with time based job schedulers, such as cron or systemd
       timers. Both serve very different purposes, and cron jobs are in fact commonly used to
       enqueue Minion jobs that need to follow a schedule. For example to perform regular
       maintenance tasks.

   Mojolicious
       You can use Minion as a standalone job queue or integrate it into Mojolicious applications
       with the plugin Mojolicious::Plugin::Minion.

         use Mojolicious::Lite -signatures;

         plugin Minion => {Pg => 'postgresql://sri:s3cret@localhost/test'};

         # Slow task
         app->minion->add_task(poke_mojo => sub ($job, @args) {
           $job->app->ua->get('mojolicious.org');
           $job->app->log->debug('We have poked mojolicious.org for a visitor');
         });

         # Perform job in a background worker process
         get '/' => sub ($c) {
           $c->minion->enqueue('poke_mojo');
           $c->render(text => 'We will poke mojolicious.org for you soon.');
         };

         app->start;

       Background worker processes are usually started with the command
       Minion::Command::minion::worker, which becomes automatically available when an application
       loads Mojolicious::Plugin::Minion.

         $ ./myapp.pl minion worker

       The worker process will fork a new process for every job that is being processed. This
       allows for resources such as memory to be returned to the operating system once a job is
       finished. Perl fork is very fast, so don't worry about the overhead.

         Minion::Worker
         |- Minion::Job [1]
         |- Minion::Job [2]
         +- ...

       By default up to four jobs will be processed in parallel, but that can be changed with
       configuration options or on demand with signals.

         $ ./myapp.pl minion worker -j 12

       Jobs can be managed right from the command line with Minion::Command::minion::job.

         $ ./myapp.pl minion job

       You can also add an admin ui to your application by loading the plugin
       Mojolicious::Plugin::Minion::Admin. Just make sure to secure access before making your
       application publicly accessible.

         # Make admin ui available under "/minion"
         plugin 'Minion::Admin';

   Deployment
       To manage background worker processes with systemd, you can use a unit configuration file
       like this.

         [Unit]
         Description=My Mojolicious application workers
         After=postgresql.service

         [Service]
         Type=simple
         ExecStart=/home/sri/myapp/myapp.pl minion worker -m production
         KillMode=process

         [Install]
         WantedBy=multi-user.target

   Consistency
       Every new job starts out as "inactive", then progresses to "active" when it is dequeued by
       a worker, and finally ends up as "finished" or "failed", depending on its result. Every
       "failed" job can then be retried to progress back to the "inactive" state and start all
       over again.

                                                             +----------+
                                                             |          |
                                                    +----->  | finished |
          +----------+            +--------+        |        |          |
          |          |            |        |        |        +----------+
          | inactive |  ------->  | active |  ------+
          |          |            |        |        |        +----------+
          +----------+            +--------+        |        |          |
                                                    +----->  |  failed  |  -----+
               ^                                             |          |       |
               |                                             +----------+       |
               |                                                                |
               +----------------------------------------------------------------+

       The system is eventually consistent and will preserve job results for as long as you like,
       depending on "remove_after" in Minion. But be aware that "failed" results are preserved
       indefinitely, and need to be manually removed by an administrator if they are out of
       automatic retries.

       While individual workers can fail in the middle of processing a job, the system will
       detect this and ensure that no job is left in an uncertain state, depending on
       "missing_after" in Minion. Jobs that do not get processed after a certain amount of time,
       depending on "stuck_after" in Minion, will be considered stuck and fail automatically. So
       an admin can take a look and resolve the issue.

FEATURES

       Minion has many great features. This section is still very incomplete, but will be
       expanded over time.

   Priorities
       Every job enqueued with "enqueue" in Minion has a priority. Jobs with a higher priority
       get performed first, the default priority is 0. Priorities can be positive or negative,
       but should be in the range between 100 and "-100".

         # Default priority
         $minion->enqueue('check_links', ['https://mojolicious.org']);

         # High priority
         $minion->enqueue('check_links', ['https://mojolicious.org'], {priority => 30});

         # Low priority
         $minion->enqueue('check_links', ['https://mojolicious.org'], {priority => -30});

       You can use "retry" in Minion::Job to raise or lower the priority of a job.

         $job->retry({priority => 50});

   Job results
       The result of a job has two parts. First there is its state, which can be "finished" for a
       successfully processed job, and "failed" for the opposite. And second there's a "result"
       data structure, that may be "undef", a scalar, a hash reference, or an array reference.
       You can check both at any time in the life cycle of a job with "job" in Minion, all you
       need is the job id.

         # Check job state
         my $state = $minion->job($job_id)->info->{state};

         # Get job result
         my $result = $minion->job($job_id)->info->{result};

       While the "state" will be assigned automatically by Minion, the "result" for "finished"
       jobs is usually assigned manually with "finish" in Minion::Job.

         $minion->add_task(job_with_result => sub ($job) {
           sleep 5;
           $job->finish({message => 'This job should have taken about 5 seconds'});
         });

       For jobs that "failed" due to an exception, that exception will be assigned as "result".

         $minion->add_task(job_that_fails => sub ($job) {
           sleep 5;
           die 'This job should always fail after 5 seconds';
         });

       But jobs can also fail manually with "fail" in Minion::Job.

         $minion->add_task(job_that_fails_with_result => sub ($job) {
           sleep 5;
           $job->fail({errors => ['This job should fail after 5 seconds']});
         });

       Retrieving job results is of course completely optional, and it is very common to have
       jobs where the result is unimportant.

   Named queues
       Each job can be enqueued with "enqueue" in Minion into arbitrarily named queues,
       independent of all their other properties. This is commonly used to have separate classes
       of workers, for example to ensure that free customers of your web service do not
       negatively affect your service level agreements with paying customers. The default named
       queue is "default", but aside from that it has no special properties.

         # Use "default" queue
         $minion->enqueue('check_links', ['https://mojolicious.org']);

         # Use custom "important" queue
         $minion->enqueue('check_links', ['https://mojolicious.org'], {queue => 'important'});

       For every named queue you can start as many workers as you like with the command
       Minion::Command::minion::worker. And each worker can process jobs from multiple named
       queues. So your workers can have overlapping responsibilities.

         $ ./myapp.pl minion worker -q default -q important

       There is one special named queue called "minion_foreground" that you should avoid using
       directly. It is reserved for debugging jobs with "foreground" in Minion.

   Job progress
       Progress information and other job metadata can be stored in notes at any time during the
       life cycle of a job with "note" in Minion::Job. The metadata can be arbitrary data
       structures constructed with scalars, hash references and array references.

         $minion->add_task(job_with_progress => sub ($job) {
           sleep 1;
           $job->note(progress => '25%');
           sleep 1;
           $job->note(progress => '50%');
           sleep 1;
           $job->note(progress => '75%');
           sleep 1;
           $job->note(progress => '100%');
         });

       Notes, similar to job results, can be retrieved with "job" in Minion, all you need is the
       job id.

         # Get job metadata
         my $progress = $minion->job($job_id)->info->{notes}{progress};

       You can also use notes to store arbitrary metadata with new jobs when you create them with
       "enqueue" in Minion.

         # Create job with metadata
         $minion->enqueue('job_with_progress', [], {notes => {progress => 0, something_else => [1, 2, 3]}});

       The admin ui provided by Mojolicious::Plugin::Minion::Admin allows searching for jobs
       containing a certain note, so you can also use them to tag jobs.

   Delayed jobs
       The "delay" option of "enqueue" in Minion can be used to delay the processing of a job by
       a certain amount of seconds (from now).

         # Job will not be processed for 60 seconds
         $minion->enqueue('check_links', ['https://mojolicious.org'], {delay => 20});

       You can use "retry" in Minion::Job to change the delay.

         $job->retry({delay => 10});

   Expiring jobs
       The "expire" option of "enqueue" in Minion can be used to limit for how many seconds (from
       now) a job should be valid before it expires and gets deleted from the queue.

         # Job will vanish if it is not dequeued within 60 seconds
         $minion->enqueue('check_links', ['https://mojolicious.org'], {expire => 60});

       You can use "retry" in Minion::Job to reset the expiration time.

         $job->retry({expire => 30});

   Custom workers
       In cases where you don't want to use Minion together with Mojolicious, you can just skip
       the plugins and write your own worker scripts.

         #!/usr/bin/perl
         use strict;
         use warnings;

         use Minion;

         # Connect to backend
         my $minion = Minion->new(Pg => 'postgresql://postgres@/test');

         # Add tasks
         $minion->add_task(something_slow => sub ($job, @args) {
           sleep 5;
           say 'This is a background worker process.';
         });

         # Start a worker to perform up to 12 jobs concurrently
         my $worker = $minion->worker;
         $worker->status->{jobs} = 12;
         $worker->run;

       The method "run" in Minion::Worker contains all features you would expect from a Minion
       worker and can be easily configured with "status" in Minion::Worker. For even more
       customization options Minion::Worker also has a very rich low level API you could for
       example use to build workers that do not fork at all.

   Task plugins
       As your Mojolicious application grows, you can move tasks into application specific
       plugins.

         package MyApp::Task::PokeMojo;
         use Mojo::Base 'Mojolicious::Plugin', -signatures;

         sub register ($self, $app, $config) {
           $app->minion->add_task(poke_mojo => sub ($job, @args) {
             $job->app->ua->get('mojolicious.org');
             $job->app->log->debug('We have poked mojolicious.org for a visitor');
           });
         }

         1;

       Which are loaded like any other plugin from your application.

         # Mojolicious
         $app->plugin('MyApp::Task::PokeMojo');

         # Mojolicious::Lite
         plugin 'MyApp::Task::PokeMojo';

   Task classes
       For more flexibility, or if you are using Minion as a standalone job queue, you can also
       move tasks into dedicated classes. Allowing the use of Perl features such as inheritance
       and roles. But be aware that support for task classes is still EXPERIMENTAL and might
       change without warning!

         package MyApp::Task::PokeMojo;
         use Mojo::Base 'Minion::Job', -signatures;

         sub run ($self, @args) {
           $self->app->ua->get('mojolicious.org');
           $self->app->log->debug('We have poked mojolicious.org for a visitor');
         }

         1;

       Task classes are registered just like any other task with "add_task" in Minion and you can
       even register the same class with multiple names.

         $minion->add_task(poke_mojo => 'MyApp::Task::PokeMojo');

MORE

       You can continue with Mojolicious::Guides now or take a look at the Mojolicious wiki
       <https://github.com/mojolicious/mojo/wiki>, which contains a lot more documentation and
       examples by many different authors.

SUPPORT

       If you have any questions the documentation might not yet answer, don't hesitate to ask in
       the Forum <https://forum.mojolicious.org> or the official IRC channel "#mojo" on
       "irc.libera.chat" (chat now! <https://web.libera.chat/#mojo>).