Provided by: libpar-perl_1.019-1_all bug

NAME

       PAR::Tutorial - Cross-Platform Packaging and Deployment with PAR

SYNOPSIS

       This is a tutorial on PAR, first appeared at the 7th Perl Conference.  The HTML version of
       this tutorial is available online as <http://search.cpan.org/perldoc?PAR::Tutorial>

DESCRIPTION

   On Deploying Perl Applications
        % sshnuke.pl 10.2.2.2 -rootpw="Z1ON0101"
        Perl v5.6.1 required--this is only v5.6.0, stopped at sshnuke.pl line 1.
        BEGIN failed--compilation aborted at sshnuke.pl line 1.

       •   Q: "Help! I can't run your program!"

       •   A1: Install Perl & "perl -MCPAN -e'install(...)'"

           •   How do we know which modules are needed?

           •   New versions of CPAN modules may break "sshnuke.pl"

       •   A2: Install Perl & "tar zxf my_perllib.tgz"

           •   Possibly overwriting existing modules; not cross-platform at all

       •   A3: Use the executable generated by "perlcc sshnuke.pl"

           •   Impossible to debug; "perlcc" usually does not work anyway

   PAR, the Perl Archive Toolkit
       •   Do what JAR (Java Archive) does for Perl

           •   Aggregates modules, scripts and other files into a Zip file

           •   Easy to generate, update and extract

           •   Version consistency: solves forward-compatibility problems

           •   Developed by community: "par@perl.org"

       •   PAR files can be packed into self-contained scripts

           •   Automatically scans perl script for dependencies

           •   Bundles all necessary 3rd-party modules with it

           •   Requires only core Perl to run on the target machine

           •   PAR also comes with "pp", the Perl Packager:

                % pp -o sshnuke.exe sshnuke.pl # stand-alone executable!

   Simple Packaging
       •   PAR files are just Zip files with modules in it

       •   Any Zip tools can generate them:

            % zip foo.par Hello.pm World.pm        # pack two modules
            % zip -r bar.par lib/          # grab all modules in lib/

       •   To load modules from PAR files:

            use PAR;
            use lib "foo.par";             # the .par part is optional
            use Hello;

       •   This also works:

            use PAR "/home/mylibs/*.par";  # put all of them into @INC
            use Hello;

   PAR Loaders
       •   Use "par.pl" to run files inside a PAR archive:

            % par.pl foo.par               # looks for 'main.pl' by default
            % par.pl foo.par test.pl       # runs script/test.pl in foo.par

       •   Same thing, with the stand-alone "parl" or "parl.exe":

            % parl foo.par                 # no perl or PAR.pm needed!
            % parl foo.par test.pl         # ditto

       •   The PAR loader can prepend itself to a PAR file:

           •   "-b" bundles non-core modules needed by "PAR.pm":

                % par.pl -b -O./foo.pl foo.par # self-contained script

           •   "-B" bundles core modules in addition to "-b":

                % parl -B -O./foo.exe foo.par  # self-contained binary

   Dependency Scanning
       •   Recursively scan dependencies with "scandeps.pl":

            % scandeps.pl sshnuke.pl
            # Legend: [C]ore [X]ternal [S]ubmodule [?]NotOnCPAN
            'Crypt::SSLeay'       => '0', #  X   #
            'Net::HTTP'           => '0', #      #
            'Crypt::SSLeay::X509' => '0', # S    # Crypt::SSLeay
            'Net::HTTP::Methods'  => '0', # S    # Net::HTTP
            'Compress::Zlib'      => '0', #  X   # Net::HTTP::Methods

       •   Scan an one-liner, list all involved files:

            % scandeps.pl -V -e "use Dynaloader;"
            ...
            # auto/DynaLoader/dl_findfile.al [autoload]
            # auto/DynaLoader/extralibs.ld [autoload]
            # auto/File/Glob/Glob.bs [data]
            # auto/File/Glob/Glob.so [shared]
            ...

   Perl Packager: "pp"
       •   Combines scanning, zipping and loader-embedding:

            % pp -o out.exe src.pl         # self-contained .exe
            % out.exe                      # runs anywhere on the same OS

       •   Bundle additional modules:

            % pp -o out.exe -M CGI src.pl  # pack CGI + its dependencies, too

       •   Pack one-liners:

            % pp -o out.exe -e 'print "Hi!"'   # turns one-liner into executable

       •   Generate PAR files instead of executables:

            % pp -p src.pl                 # makes 'source.par'
            % pp -B -p src.pl              # include core modules

   How it works
       •   Command-line options are almost identical to "perlcc"'s

           •   Also supports "gcc"-style long options:

                % pp --gui --verbose --output=out.exe src.pl

       •   Small initial overhead; no runtime overhead

       •   Dependencies are POD-stripped before packing

       •   Loads modules directly into memory on demand

       •   Shared libraries (DLLs) are extracted with File::Temp

       •   Works on Perl 5.6.0 or above

       •   Tested on Win32 (VC++ and MinGW), FreeBSD, NetBSD, Linux, MacOSX, Cygwin, AIX,
           Solaris, HP-UX, Tru64...

   Aggregating multiple programs
       •   A common question:

            > I have used pp to make several standalone applications which work
            > great, the only problem is that for each executable that I make, I am
            > assuming the parl.exe is somehow bundled into the resulting exe.

       •   The obvious workaround:

            You can ship parl.exe by itself, along with .par files built
            by "pp -p", and run those PAR files by associating them to parl.exe.

       •   On platforms that have "ln", there is a better solution:

            % pp --output=a.out a.pl b.pl  # two scripts in one!
            % ln a.out b.out               # symlink also works
            % ./a.out                      # runs a.pl
            % ./b.out                      # runs b.pl

   Cross-platform Packages
       •   Of course, there is no cross-platform binary format

       •   Pure-perl PAR packages are cross-platform by default

           •   However, XS modules are specific to Perl version and platform

           •   Multiple versions of a XS module can co-exist in a PAR file

       •   Suppose we need "out.par" on both Win32 and Finix:

            C:\> pp --multiarch --output=out.par src.pl
            ...copy src.pl and out.par to a Finix machine...
            % pp --multiarch --output=out.par src.pl

       •   Now it works on both platforms:

            % parl out.par                 # runs src.pl
            % perl -MPAR=out.par -e '...'  # uses modules inside out.par

   The Anatomy of a PAR file
       •   Modules can reside in several directories:

            /                      # casual packaging only
            /lib/                  # standard location
            /arch/                 # for creating from blib/
            /i386-freebsd/         # i.e. $Config{archname}
            /5.8.0/                # i.e. Perl version number
            /5.8.0/i386-freebsd/   # combination of the two above

       •   Scripts are stored in one of the two locations:

            /                      # casual packaging only
            /script/               # standard location

       •   Shared libraries may be architecture- or perl-version-specific:

            /shlib/(5.8.0/)?(i386-freebsd/)?

       •   PAR files may recursively contain other PAR files:

            /par/(5.8.0/)?(i386-freebsd/)?

   Special files
       •   MANIFEST

           •   Index of all files inside PAR

           •   Can be parsed with "ExtUtils::Manifest"

       •   META.yml

           •   Dependency, license, runtime options

           •   Can be parsed with "YAML"

       •   SIGNATURE

           •   OpenPGP-signed digital signature

           •   Can be parsed and verified with "Module::Signature"

   Advantages over perlcc, PerlApp and Perl2exe
       •   This is not meant to be a flame

           •   All three maintainers have contributed to PAR directly; I'm grateful

       •   perlcc

           •   "The code generated in this way is not guaranteed to work... Use for production
               purposes is strongly discouraged." (from perldoc perlcc)

           •   Guaranteed to not work is more like it

       •   PerlApp / Perl2exe

           •   Expensive: Need to pay for each upgrade

           •   Non-portable: Only available for limited platforms

           •   Proprietary: Cannot extend its features or fix bugs

           •   Obfuscated: Vendor and black-hats can see your code, but you can't

           •   Inflexible: Does not work with existing Perl installations

   MANIFEST: Best viewed with Mozilla
       •   The URL of "MANIFEST" inside "/home/autrijus/foo.par":

            jar:file:///home/autrijus/foo.par!/MANIFEST

       •   Open it in a Gecko browser (e.g. Netscape 6+) with Javascript enabled:

       •   No needed to unzip anything; just click on files to view them

   META.yml: Metadata galore
       •   Static, machine-readable distribution metadata

           •   Supported by "Module::Build", "ExtUtils::MakeMaker", "Module::Install"

       •   A typical "pp"-generated "META.yml" looks like this:

            build_requires: {}
            conflicts: {}
            dist_name: out.par
            distribution_type: par
            dynamic_config: 0
            generated_by: 'Perl Packager version 0.03'
            license: unknown
            par:
              clean: 0
              signature: ''
              verbatim: 0
              version: 0.68

       •   The "par:" settings controls its runtime behavior

   SIGNATURE: Signing and verifying packages
       •   OpenPGP clear-signed manifest with SHA1 digests

           •   Supported by "Module::Signature", "CPANPLUS" and "Module::Build"

       •   A typical "SIGNATURE" looks like this:

            -----BEGIN PGP SIGNED MESSAGE-----
            Hash: SHA1

            SHA1 8a014cd6d0f6775552a01d1e6354a69eb6826046 AUTHORS
            ...
            -----BEGIN PGP SIGNATURE-----
            ...

            -----END PGP SIGNATURE-----
       •   Use "pp" and "cpansign" to work with signatures:

            % pp -s -o foo.par bar.pl      # make and sign foo.par from bar.pl
            % cpansign -s foo.par  # sign this PAR file
            % cpansign -v foo.par  # verify this PAR file

   Perl Servlets with Apache::PAR
       •   Framework for self-contained Web applications

           •   Similar to Java's "Web Application Archive" (WAR) files

           •   Works with mod_perl 1.x or 2.x

       •   A complete web application inside a ".par" file

           •   Apache configuration, static files, Perl modules...

           •   Supports Static, Registry and PerlRun handlers

           •   Can also load all PARs under a directory

       •   One additional special file: "web.conf"

            Alias /myapp/cgi-perl/ ##PARFILE##/
            <Location /myapp/cgi-perl>
                Options +ExecCGI
                SetHandler perl-script
                PerlHandler Apache::PAR::Registry
            </Location>

   Hon Dah, A-par-che!
       •   First, make a "hondah.par" from an one-liner:

            # use the "web.conf" from the previous slide
            % pp -p -o hondah.par -e 'print "Hon Dah!\n"' \
                 --add web.conf
            % chmod a+x hondah.par

       •   Add this to "httpd.conf", then restart apache:

            <IfDefine MODPERL2>
            PerlModule Apache2
            </IfDefine>
            PerlAddVar PARInclude /home/autrijus/hondah.par
            PerlModule Apache::PAR

       •   Test it out:

            % GET http://localhost/myapp/cgi-perl/main.pl
            Hon Dah!

       •   Instant one-liner web application that works!

   On-demand library fetching
       •   With LWP installed, your can use remote PAR files:

            use PAR;
            use lib 'http://aut.dyndns.org/par/DBI-latest.par';
            use DBI;    # always up to date!

       •   Modules are now cached under $ENV{PAR_GLOBAL_TEMP}

       •   Auto-updates with "LWP::Simple::mirror"

           •   Download only if modified

           •   Safe for offline use after the first time

           •   May use "SIGNATURE" to prevent DNS-spoofing

       •   Makes large-scale deployment a breeze

           •   Upgrades from a central location

           •   No installers needed

   Code Obfuscation
       •   Also known as source-hiding techniques

           •   It is not encryption

           •   Offered by PerlApp, Perl2Exe, Stunnix...

       •   Usually easy to defeat

           •   Take optree dump from memory, feed to "B::Deparse"

           •   If you just want to stop a casual "grep", "deflate" already works

       •   PAR now supports pluggable input filters with "pp -f"

           •   Bundled examples: Bleach, PodStrip and PatchContent

           •   True encryption using "Crypt::*"

           •   Or even _product activation_ over the internet

       •   Alternatively, just keep core logic in your server and use RPC

   Accessing packed files
       •   To get the host archive from a packed program:

            my $zip = PAR::par_handle($0); # an Archive::Zip object
            my $content = $zip->contents('MANIFEST');

       •   Same thing, but with "read_file()":

            my $content = PAR::read_file('MANIFEST');

       •   Loaded PAR files are stored in %PAR::LibCache:

            use PAR '/home/mylibs/*.par';
            while (my ($filename, $zip) = each %PAR::LibCache) {
                print "[$filename - MANIFEST]\n";
                print $zip->contents('MANIFEST');
            }

   Packing GUI applications
       •   GUI toolkits often need to link with shared libraries:

            # search for libncurses under library paths and pack it
            % pp -l ncurses curses_app.pl  # same for Tk, Wx, Gtk, Qt...

       •   Use "pp --gui" on Win32 to eliminate the console window:

            # pack 'src.pl' into a console-less 'out.exe' (Win32 only)
            % pp --gui -o out.exe src.pl

       •   "Can't locate Foo/Widget/Bar.pm in @INC"?

           •   Some toolkits (notably Tk) autoloads modules without "use" or "require"

           •   Hence "pp" and "Module::ScanDeps" may fail to detect them

           •   Tk problems mostly fixed by now, but other toolkits may still break

           •   You can work around it with "pp -M" or an explicit "require"

           •   Or better, send a short test-case to "par@perl.org" so we can fix it

   Precompiled CPAN distributions
       •   Installing XS extensions from CPAN was difficult

           •   Some platforms do not come with a compiler (Win32, MacOSX...)

           •   Some headers or libraries may be missing

           •   PAR.pm itself used to suffer from both problems

       •   ...but not anymore -- "Module::Install" to the rescue!

            # same old Makefile.PL, with a few changes
            use inc::Module::Install;      # was "use ExtUtils::MakeMaker;"
            WriteMakefile( ... );          # same as the original
            check_nmake();                 # make sure the user have nmake
            par_base('AUTRIJUS');          # your CPAN ID or a URL
            fetch_par() unless can_cc();   # use precompiled PAR only if necessary

       •   Users will not notice anything, except now it works

           •   Of course, you still need to type "make par" and upload the precompiled package

           •   PAR users can also install it directly with "parl -i"

   Thank you!
       •   Additional resources

           •   Mailing list: "par@perl.org"

           •   Subscribe: Send a blank email to "par-subscribe@perl.org"

           •   List archive: <http://nntp.x.perl.org/group/perl.par>

           •   PAR::Intro: <http://search.cpan.org/dist/PAR/lib/PAR/Intro.pod>

           •   Apache::PAR: <http://search.cpan.org/dist/Apache-PAR/>

           •   Module::Install: <http://search.cpan.org/dist/Module-Install/>

       •   Any questions?

   Overview of PAR.pm's Implementation
       •   Here begins the scary part

           •   Grues, Dragons and Jabberwocks abound...

           •   You are going to learn weird things about Perl internals

       •   PAR invokes four areas of Perl arcana:

           •   @INC code references

           •   On-the-fly source filtering

           •   Overriding "DynaLoader::bootstrap()" to handle XS modules

           •   Making self-bootstrapping binary executables

       •   The first two only works on 5.6 or later

           •   DynaLoader and %INC are there since Perl 5 was born

           •   PAR currently needs 5.6, but a 5.005 port is possible

   Code References in @INC
       •   On 1999-07-19, Ken Fox submitted a patch to P5P

           •   To _enable using remote modules_ by putting hooks in @INC

           •   It's accepted to come in Perl 5.6, but undocumented until 5.8

           •   Type "perldoc -f require" to read the nitty-gritty details

       •   Coderefs in @INC may return a fh, or undef to 'pass':

            push @INC, sub {
                my ($coderef, $filename) = @_;  # $coderef is \&my_sub
                open my $fh, "wget ftp://example.com/$filename |";
                return $fh;        # using remote modules, indeed!
            };

       •   Perl 5.8 let you open a file handle to a string, so we just use that:

                   open my $fh, '<', \($zip->memberNamed($filename)->contents);
                   return $fh;

       •   But Perl 5.6 does not have that, and I don't want to use temp files...

   Source Filtering without Filter::* Modules
       •   ... Undocumented features to the rescue!

           •   It turns out that @INC hooks can return two values

           •   The first is still the file handle

           •   The second is a code reference for line-by-line source filtering!

       •   This is how "Acme::use::strict::with::pride" works:

            # Force all modules used to use strict and warnings
            open my $fh, "<", $filename or return;
            my @lines = ("use strict; use warnings;\n", "#line 1 \"$full\"\n");
            return ($fh, sub {
                return 0 unless @lines;
                push @lines, $_; $_ = shift @lines; return length $_;
            });

   Source Filtering without Filter::* Modules (cont.)
       •   But we don't really have a filehandle for anything

       •   Another undocumented feature saves the day!

       •   We can actually omit the first return value altogether:

            # Return all contents line-by-line from the file inside PAR
            my @lines = split(
                /(?<=\n)/,
                $zip->memberNamed($filename)->contents
            );
            return (sub {
                $_ = shift(@lines);
                return length $_;
            });

   Overriding DynaLoader::bootstrap
       •   XS modules have dynamically loaded libraries

           •   They cannot be loaded as part of a zip file, so we extract them out

           •   Must intercept DynaLoader's library-finding process

       •   Module names are passed to "bootstrap" for XS loading

           •   During the process, it calls "dl_findfile" to locate the file

           •   So we install pre-hooks around both functions

       •   Our "_bootstrap" just checks if the library is in PARs

           •   If yes, extract it to a "File::Temp" temp file

               •   The file will be automatically cleaned up when the program ends

           •   It then pass the arguments to the original "bootstrap"

           •   Finally, our "dl_findfile" intercepts known filenames and return it

   Anatomy of a Self-Contained PAR executable
       •   The par script ($0) itself

           •   May be in plain-text or native executable format

       •   Any number of embedded files

           •   Typically used to bootstrap PAR's various dependencies

           •   Each section begins with the magic string "FILE"

           •   Length of filename in pack('N') format and the filename (auto/.../)

           •   File length in pack('N') and the file's content (not compressed)

       •   One PAR file

           •   Just a regular zip file with the magic string "PK\003\004"

       •   Ending section

           •   A pack('N') number of the total length of FILE and PAR sections

           •   Finally, there must be a 8-bytes magic string: "\012PAR.pm\012"

   Self-Bootstrapping Tricks
       •   All we can expect is a working perl interpreter

           •   The self-contained script *must not* use any modules at all

           •   But to process PAR files, we need XS modules like Compress::Zlib

       •   Answer: bundle all modules + libraries used by PAR.pm

           •   That's what the "FILE" section in the previous slide is for

           •   Load modules to memory, and write object files to disk

           •   Then use a local @INC hook to load them on demand

       •   Minimizing the amount of temporary files

           •   First, try to load PerlIO::scalar and File::Temp

           •   Set up an END hook to unlink all temp files up to this point

           •   Load other bundled files, and look in the compressed PAR section

           •   This can be much easier with a pure-perl "inflate()"; patches welcome!

   Thank you (again)!
       •   Any questions, please?

SEE ALSO

       PAR, pp, par.pl, parl

       ex::lib::zip, Acme::use::strict::with::pride

       App::Packer, Apache::PAR, CPANPLUS, Module::Install

AUTHORS

       Audrey Tang <cpan@audreyt.org>

       You can write to the mailing list at <par@perl.org>, or send an empty mail to
       <par-subscribe@perl.org> to participate in the discussion.

       Please submit bug reports to <bug-par@rt.cpan.org>.

COPYRIGHT

       Copyright 2003, 2004, 2005, 2006 by Audrey Tang <cpan@audreyt.org>.

       This document is free documentation; you can redistribute it and/or modify it under the
       same terms as Perl itself.

       See LICENSE.