Provided by: libxml-rss-simplegen-perl_11.11-2_all bug


       XML::RSS::SimpleGen - for writing RSS files


         # A complete screen-scraper and RSS generator here:

         use strict;
         use XML::RSS::SimpleGen;
         my $url = q<>;

         rss_new( $url, "eXile", "Moscow-based Alternative Newspaper" );
         rss_language( 'en' );
         rss_webmaster( '' );

         get_url( $url );

          m{<h4>\s*<a href='/(.*?)'.*?>(.*?)</a>\s*</h4>\s*<p.*?>(.*?)<a href='/}sg
         ) {
           rss_item("$url$1", $2, $3);

         die "No items in this content?! {{\n$_\n}}\nAborting"
          unless rss_item_count();

         rss_save( 'exile.rss', 45 );


       This module is for writing RSS files, simply. It transparently handles all the unpleasant
       details of RSS, like proper XML escaping, and also has a good number of Do-What-I-Mean
       features, like not changing the modtime on a written-out RSS file if the file content
       hasn't changed, and like automatically removing any HTML tags from content you might pass

       This module isn't meant to have the full expressive power of RSS; instead, it provides
       functions that are most commonly needed by RSS-writing programs.


       This module provides a bunch of functions for starting an RSS feed in memory, putting
       items into it, and saving it to disk (or printing it as a string, as in a CGI). If you
       prefer an object-oriented interface (obviously more useful if you're composing several
       feeds at once), then you can use this module as a class whose methods are the same as the
       function names minus "rss_". Except for this detail of the naming, the functions and
       methods are the same, behave the same, and take the same arguments.

       That is, this functional code:

         use XML::RSS::SimpleGen;
         my $url = q<>;

         rss_new( $url, "eXile" );
         rss_language( 'en' );
         get_url( $url );

       does the same work as this OO code:

         use XML::RSS::SimpleGen ();
         my $url = q<>;
         my $rss = XML::RSS::SimpleGen->new( $url, "eXile");
         $rss->language( 'en' );
         $rss->get_url( $url );

       (Note that the function "get_url" doesn't have a leading "rss_", so its method name is the
       same as its function name.  It's the one exception.)

       If this talk of objects puzzles you, see HTML::Tree::AboutObjects in the "HTML-Tree" dist,
       and/or see the chapter "User's View of Object-Oriented Modules" in my book Perl & LWP
       (<>).  (The book is also useful as an
       extended discussion of screen-scraping.)

       Note: in the code below, I use the word "accessor" a lot, to refer to a function or method
       that you can call two possible ways: 1) like "foo(val)" to set the "foo" attribute to the
       value val, or 2) like "foo()" to return the value of the "foo" attribute.


       "rss_new( url );"
       "rss_new( url, title );"
       "rss_new( url, title, description );"
       or: "$rss = XML::RSS::SimpleGen->new(...);"
           This function creates a new RSS feed in memory.  This should be the first
           "rss_whatever" function you call in your program.  If you call it again, it erases the
           current object (if any) and sets up a new one according to whatever parameters you

           The parameters are the full URL, the title, and the description of the site (or page)
           that you're providing an RSS feed of. The description is optional, but you should
           provide at least a URL and title.


             rss_new( $url, "eXile", "Moscow-based Alternative Newspaper" );

             rss_new( '', "Bazouki News!" );

           (As a method, XML::RSS::SimpleGen->new simply returns a new RSS object.)

       the accessor "rss_language(language_tag)"
           This declares what language this RSS feed is in.  It must be an RFC3066-style language
           tags like "en", or "en-US", or "zh-TW".  (See I18N::LangTags::List for a list.)  If
           you don't set the feed's language, it defaults to "en", for generic English.

           If you call this function without a parameter, it returns the current value of the RSS
           feed's language.  For example:

             print "I'm making an RSS feed for ", rss_language(), "!\n";

           The same is true for all the functions that I label as "accessors".

       the accessor "rss_item_limit(number)"
           This sets the maximum number of items that this feed will show.

           The default value is 0, meaning that there is no maximum.

           If you set it to a positive number N, then the feed will show only the first N items
           that you declare with "rss_item". (Or, if you set "rss_history_file", then the newest
           N items that you declare with "rss_item".)

           If you set it to a negative number -N, then the feed will show only the last N items
           that you declare with "rss_item". (Or, if you set "rss_history_file", then the oldest
           N items you declare with "rss_item", which is unlikely to be useful!)

       the accessor "rss_webMaster(email-address)"
           This declares what email address you, the RSS generator manager, can be reached at.

             rss_webMaster( '' );

       "rss_history_file( filename )"
           This declares that you want this RSS feed to keep track of what items are new, and to
           list them first when the RSS is emitted.  To do this, the RSS generator has to store
           information in a file, where it tracks its "history", i.e., when was the first time it
           saw given URLs, and the most recent time it saw given URLs.

           Typical usage is:

             rss_history_file( 'thisrssfeed.dat' );

           You should call "rss_history_file" before you make any calls to "rss_item".

           The history-file feature is meant for cases where your RSS-generator program calls
           "rss_item" on every link it sees, but only wants the new links to appear in the RSS
           output. (This can be a good approach if you're making an RSS feed of a page like
           "" where there's some new links (to the recently added
           stories), but also links to some days-old stories, and also links to some always-there
           things like "Archive Search" and "Contact Us" pages.

           Once you call rss_history_file, the specified file is read in.  The in-memory history
           (stored in the RSS object) is updated as you call "rss_item".  But the file isn't
           updated until you call rss_save.

           (A do-what-I-mean side effect of calling "rss_history_file" is that it sets
           rss_item_limit to 25 if it is currently 0.)

           (Incidentally, if you're using rss_history_file as part of a CGI that emits RSS data,
           instead of a program that just saves to an RSS file, then things will get complicated.
           You'll need to call an internal method to explicitly commit the history file to disk,
           and you'll need a semaphore file to avoid race conditions. Email me for full info.)

       "rss_item( url );"
       "rss_item( url, title );"
       "rss_item( url, title, description );"
           This adds a new item to the current feed. You will need to specify the URL to add (and
           it should be a valid-looking URL, starting with "something:", and not containing any
           spaces). You may also specify the title, but it's optional. And finally, you can
           optionally specify a description. (You can remember this because it starts with the
           essential item first, and progresses toward the most optional.)

           Leading and tailing whitespace is removed from whichever of url, title, and
           description are defined values, and HTML is parsed out.

           A simple usage:

               "Harper's Magazine's Weekly Review"

           Although in practice, a typical call won't have string constants, but will instead be
           like the example in the Synopsis sectios, namely:

             rss_item("$url$1", $2, $3);

           Incidentally, as a do-what-I-mean feature, if the first parameter doesn't look like a
           URL but one of the others does, then this error is silently forgiven.  This is so you
           can occasionally slip up and forget the order of the parameters.

           (In the unlikely event where you need to avoid the HTML-removal features, you can do
           this by passing scalar-references instead of normal strings, like so: "rss_item($url,
           $title, \$not_to_be_escaped)".)

           This returns the number of items you've declared.  I anticipate that its main usage
           will be something like:

             die "What, no objects found at $url ?!"
              unless rss_item_count();

           or, maybe...

             exit unless rss_item_count();

           ...depending on how/whether you'd want to react to cases where you don't see anything
           to put into an RSS feed.

           Note that the parens are optional, since this command takes no options (just like
           Perl's "time()" function).

       "rss_image( url, h, w );"
           This declares that you want to declare a particular image as the logo for this feed.
           Most feeds don't have such a thing, and most readers just ignore it anyway, but if you
           want to declare it, this function is how.  The three parameters, which are all
           required, are: the image's URL, its height in pixels, and its width in pixels.
           According to various specs, the width should/must be between 1 and 144, an the height
           should/must be between 1 and 400.

           A typical usage:

             rss_image("", 106, 140);

           Be careful not to mix up the height and width.

       "rss_save( filename );"
       "rss_save( filename, max_age_days );"
           This saves the RSS date to the file you specify.  If the RSS data hasn't changed, the
           file (and its modtime) aren't altered.  The optional max_age_days parameter means that
           if ever the file exists, and its content hasn't changed for that many days or longer,
           then the program should die with a warning message.  For example, in the case of a
           screen-scraper for a site that we know should (in theory) change its content at least
           weekly, we might save the RSS file with:

             rss_save("whatever.rss", 17);
              # Scream if the feed is unchanged for 17 days.

           The seventeen there is gotten by assuming that just maybe the site might skip two
           weeks for a vacation now and then, and might even put out the pre-vacation issue a few
           days early -- but that if ever the program notices that the data hasn't changed for 17
           days, then it should emit error messages.  If you want to disable this feature on a
           one-time basis, just change the modtime (like via "touch") on the whatever.rss file.

           If you don't specify a "max_age_days" value, then this whole complain-if-it's-old
           feature is disabled.

           This returns the RSS-XML data as a string.  This function is called internally by the
           rss_save function; but you might want to call it explicitly, as in a CGI, where your
           CGI would probably end like this:

             print "Content-type: application/xml\n\n", rss_as_string();

       "get_url( url );"
       "$content = get_url( url );"
       or: "$content = $rss->get_url(...);"
       or: "$content->get_url(...);"
           This tries to get the content of the given url, and returns it.

           This is quite like LWP::Simple's "get" function, but with some additional features:

           ·   If it can't get the URL's content at first, it will sleep for a few seconds and
               try again, up to about five times. (This is to avoid the case of the URL being
               temporarily inaccessible simply because the DNS is a bit slow, or because the
               server is too busy.)

           ·   If it can't get the content, even after several retries, it will abort the program
               (like a "die").  If you want to override this behavior, then call it as "eval {
               get_url($url) };"

           ·   If you call the function in void context (i.e., not using its return value), then
               the function assigns the URL's content to $_.  That's so you can write nice
               concise code like this:

                          get_url $thatsite;
                          m/Top Stories Tonight/ or die "What, no top stories?";
                          while( m{<a class="top" href="(.*?)">(.*?)</a>}g ) {
                            rss_item("$thatsite/$1", $2);

           ·   This returns the content of the URL not exactly as-is, but after changing its
               newlines to native format.  That is, if the contents of the URL use CR-LF pairs to
               express newlines, then "get_url" changes these to "\n"'s before returning the
               content.  (Similarly for old MacOS newline format.)  Clearly this is wrong in
               you're dealing with binary data; in that case, use LWP::Simple's "get" directly.

           ·   Finally, as a resource-conversation measure, this function will also try to call
               "sleep" a few times if it sees several quick calls to itself coming from a program
               that seems to be running under crontab.  As most of my RSS-generators are
               crontabbed, I find it very useful that I can have however many "get_url"'s in my
               crontabbed programs without worrying that they'll take even a noticeable part of
               the server's bandwidth.

       "rss_hourly" or "rss_daily" or "rss_twice_daily" or "rss_thrice_daily" or "rss_weekly" or
           Calling one of these functions declares that this feed is usually generated at the
           same time(s) every day (or every week, in the case of "rss_weekly"). And, where it's
           not just once a day/week, these multiple times a day are evenly spaced.  These
           functions then set the feed's "updatePeriod", "updateBase", "updateFrequency",
           "skipHours", "skipDays", and "ttl" elements appropriately, so that RSS readers can
           know at at what times there could (or couldn't) be new content in this feed.

           In other words: use "rss_twice_daily" if this feed is updated at about the same time
           every day and then again 12 hours later.  Use "rss_thrice_daily" if this feed is
           updated at the same time daily, and then 8 hours later, and then 8 hours later.  And
           use "rss_every_other_hour" if the feed updates at about n minutes past every even
           numbered hour, or every odd-numbered hour.

           Clearly I mean these functions to be used in programs that are crontabbed to run at
           particular intervals, as with a crontab line like one of these:

                  52 * * * *         ~/thingy   # => rss_hourly
                  52 23 * * *        ~/thingy   # => rss_daily
                  52 4,16 * * *      ~/thingy   # => rss_twice_daily
                  52 5,13,21 * * *   ~/thingy   # => rss_thrice_daily
                  52 23 * * 3        ~/thingy   # => rss_weekly
                  52 */2 * * *       ~/thingy   # => rss_every_other_hour

           Clearly there aren't "rss_interval" functions for all the scheduling possibilities
           programs -- if you have a program that has to run at 6am, 8am, 1pm, and 4pm, there's
           no function for that.  However, the above crontab lines (or with minor changes, like
           "1,9,17" instead of "5,13,21") are just fine for almost every RSS feed I've run.

           An aside: I recommend running the programs at about 52 minutes past the hour,
           generally in series, like so:

                  52 5,13,21 * * *   ~/thingy ; ~/dodad ; ~/makething ; ~/gizmo

           However, your mileage may vary.

           Incidentally, these functions take no arguments, so the parentheses are optional.
           That is, these two lines do the same thing:


       These are functions that you probably won't need often, or at all.  I include these for
       the sake of completeness, and so that advanced users might find them useful in some cases.

       "rss_skipHours( gmt_hour_num, gmt_hour_num, ... );"
           This function directly sets the "skipHours" element's values to the specified GMT hour

       "rss_updateHours( gmt_hour_num, gmt_hour_num, ... );"
           This function is a wrapper around "rss_skipHours" -- you call "rss_updateHours" with a
           list of GMT hour numbers, and "rss_updateHours" will call "rss_skipHours(0 .. 23)"
           except without whatever hour numbers you specified.

           If you call with an empty list (i.e., "rss_updateHours();"), then we uses "gmtime" to
           find out the current hour (and rounds it up if it's after 50 minutes past), basically
           just as if you'd called:

                 rss_updateHours( (gmtime(600+time()))[2] );

       "rss_skipDays( gmt_day_num, gmt_day_num, ... );"
       "rss_skipDays( gmt_day_name, gmt_day_name, ... );"
           This function directly sets the "skipDays" element's values to the specified weekdays.
           Note that this accepts either integers (like 6 for Saturday, Sunday being either 0 or
           7), or their exact English names.

           If you use the "skipDays" field, consider that it refers to days figured by GMT, not
           local time.  For example, if I say to skip Saturdays, that means Saturdays GMT, which
           in my timezone (Alaska) starts in the middle of Friday afternoon.

       "rss_updateDays( gmt_day_num, gmt_day_num, ... );"
       "rss_updateDays( gmt_day_name, gmt_day_name, ... );"
           This function is a wrapper around "rss_skipDays" -- you call "rss_updateDays" with a
           list of GMT day names/numbers, and "rss_updateDays" will call "rss_skipDays(0 .. 6)"
           except without whatever days you specified.

           If you call with an empty list (i.e., "rss_updateDays();"), then we uses "gmtime" to
           find out the current day (GMT!), basically just as if you'd called:

                 rss_updateDays( (gmtime(600+time()))[6] );

       "rss_updatePeriod( periodstring );"
           This function directly sets the "sy:updatePeriod" element's value to the period
           specified.  You must specify one of the strings: "yearly", "monthly", "weekly",
           "daily", "hourly".  I advise using "weekly" only if you know what you're doing, and
           "yearly", "monthly" only if you really know what you're doing.

       "rss_updatePeriod( periodstring, int, base );"
           This is a shortcut for "rss_updatePeriod(periodstring); rss_updateFrequency(int)"

       "rss_updatePeriod( periodstring, int, base );"
           This is a shortcut for "rss_updatePeriod(periodstring); rss_updateFrequency(int);

       "rss_updateBase( iso_date_string );"
       "rss_updateBase( epoch_time );"
           This function directly sets the "sy:updateBase" element's value to the moment
           specified.  If you pass in an epoch time, it is converted to an ISO date string.

       the accessor "rss_updateFrequency( integer );"
           This function directly sets the "sy:updateFrequency" element's value to the value
           specified.  The value has to be a nonzero positive integer.

           For example, this means that this feed updates at/by the start of every hour and 30
           minutes past:

             rss_updatePeriod('hourly');  # 2*hourly means "twice an hour"

           Recall that this can also be done with the the "rss_updatePeriod( per, freq, base )"
           shortcut, like so:

             rss_updateBase('hourly', 2, '2000-01-01T00:00-00:00');

       the accessor "rss_retention(number)"
           If you are using an "rss_history_file(file)", the history file will accrete a list of
           all URLs it has seen.  But to keep this file from potentially getting immense, items
           that haven't been seen for a while are thrown out.  The period of time a feed's items
           go unseen before each is forgotten is called that feed's retention, and is expressed
           in seconds.

           The default retention value is 32 days (i.e., 32*24*60*60, the number of seconds in 32
           days).  If you wanted to change it to just a week, you would do this with

           As a special case, a zero or negative value for the retention means to never clear
           anything from the history file, no matter how long it has gone unseen.

       "rss_add_comment( strings );"
           Call this function if you want to add extra XML comments to this RSS file.  For
           example, if you call this:

                     "Our terms of use:",
                     "Any questions? Ask",

           ...then this RSS feed will contain this XML fairly early on in the file:

                   <!-- Our terms of use: -->
                   <!-- Any questions? Ask -->

       the accessor "rss_css( url )"
           This defines the given URL as being the XML-CSS stylesheet for this RSS feed.  The
           default value is "./rss.css" if "-e "rss.css"" is true, otherwise is the value

       the accessor "rss_xsl( url )"
           This defines the given URL as being the XML-XSL stylesheet for this RSS feed.  The
           default value is none.

       The accessors "rss_url( string ), rss_title( string ), rss_description( string )"
           These define this feed's URL, title, and description.  These functions are just for
           completeness, since it's simpler to just specify any/all of these parameters in the
           call to "rss_new".

       the accessor "rss_ttl( number )"
           This sets the parameter of this RSS feed's "ttl" element, which suggests how long (in
           minutes, not seconds!) an RSS reader should wait after it polls a feed until it polls
           it again.  For example, "rss_ttl(90)" would suggest that a reader should not poll this
           feed more often than every 90 minutes.

           (This element is somewhat obsolescent next to the newer and more informative
           "sy:update*" elements, but is included for backward compatibility.)

       the accessor "rss_allow_duplicates( boolean )"
           This controls whether or not duplicate items are filtered out out the feed.  By
           default this is on.  Note that duplicates are detected only by their URL, so if you
           call this:

                   rss_item('', "Give!");
                   rss_item('', "We need money!");

           ...then only the first will appear in the feed, since the second item has a URL that
           is already being saved in this feed.  (However, "rss_item_count" is still 2, because
           filtering out duplicates is something that only happens as the feed is saved.)

       the accessor "rss_docs( url )"
           This sets the value of the not-generally-useful "doc" RSS element.  The default value
           is "./about_rss.html" if "-e "about_rss.html"" is true, otherwise

       the accessors "rss_image_url(url), rss_image_width(number), rss_image_height(number),
       rss_image_title(text), rss_image_link(url), rss_image_description(text)"
           These are for manually setting the values of this feed's image element's subelements:

                         <url> (rss_image_url)         </url>
                       <width> (rss_image_width)       </width>
                      <height> (rss_image_height)      </height>
                       <title> (rss_image_title)       </title>
                        <link> (rss_image_link)        </link>
                 <description> (rss_image_description) </description>

           You rarely need to call any of these "rss_image_whatever" functions -- usually just
           calling "rss_image( url, h, w );" is enough.


       RSS feeds emitted by this module are basically according to v0.92 RSS, with a very few
       extensions from v2.0 RSS.  They are not RDF files.







       You might also like my book Perl and LWP, which discusses the many screen-scraping
       techniques that you would use for extracting data from HTML to make into RSS feeds:

       <> -- examples of Perl programs that produce RSS's
       (which are visible at <> )


       Copyright (c) 2003,4 Sean M. Burke.  All rights reserved.

       This library is free software; you can redistribute it and/or modify it under the same
       terms as Perl itself.

       This program is distributed in the hope that it will be useful, but without any warranty;
       without even the implied warranty of merchantability or fitness for a particular purpose.

       Portions of the data tables in this module are derived from the entity declarations in the
       W3C XHTML specification.

       Currently (January 2004), that's these three:


       Portions of the code in this module were adapted from parts of Gisle Aas's LWP::Simple and
       the old (v2.x) version of his HTML::Parser.


       Sean M. Burke ""