NAME

       Marpa::R2::Changes - Differences between Marpa::R2 and Marpa::XS

About this document

       This document describes the incompatible differences between Marpa::XS and Marpa::R2.
       (Differences that do not give rise to incompatibility are outside of its scope.)  It is
       intended for readers already familiar with Marpa::XS, who are writing new applications for
       Marpa::R2, and for readers migrating Marpa::XS applications and tools to Marpa::R2.

Changes

   Additional reserved symbol names
       Marpa::XS reserved, for its internal use, all symbol names ending with the right square
       bracket (""]"").  In addition, Marpa::RS reserved symbols ending with the right
       parenthesis ("")""), the right angle bracket ("">""), and the right curly bracket (""}"").
       Any other valid Perl string remains an acceptable symbol name.

   The return value of the read() method has changed
       The return value of the Marpa::R2 recognizer's "read()" method differs from its Marpa::XS
       equivalent.  In Marpa::XS it returned the number of distinct terminals (by symbol ID)
       allowed in the next "read()".  In Marpa::R2 it returns the number of recognizer events
       that occurred during the read.  Examples of recognizer events are exhaustion, the Earley
       sets exceeding a designated "warning" level, and other circumstances settable by the user.
       For more detail, see the documentation of recognizer's "read" method.

   Rule LHS's are no longer a source of action names
       In Marpa::XS, if there was no explicit action name for a rule, Marpa would try to find a
       closure that had the same name as the rule's LHS.  The use of rule LHS's as action names
       had a potential for unpleasant surprises.  A surprise could occur if the rule's LHS
       coincided with a function name without the prorgrammer realizing or intending it.  This
       kind of 'action at a distance' bug can be very hard to detect and trace.

       It was originally thought that implicitly using the LHS as the name of an action would be
       convenient enough to outweigh the dangers.  But in fact, this feature wound up being
       little used.  And accidental resolution via a rule LHS was a danger for all users, whether
       they used the feature or not.  For these reasons, as well as potential optimization and
       efficiency considerations, Marpa::R2 no longer does implicit action resolution using a
       rule LHS.

   Different rules with the same rank now appear in arbitrary order
       In ranking parse trees, if two rule instances are for different rules but have the same
       rule rank, they will now appear in arbitrary order.  This is probably the behavior that
       programmers have always expected.

       In Marpa::XS, when the "null_ranking" named argument of rules was in use for one of the
       rules, specific guarantees were made for the order in some of the cases.  The intent was
       to be orthogonal with the guarantees made for the ranking of null variants within the same
       rule.  These additional guarantees proved useless in practice, cumbersome to implement,
       and, when documented, opaque and unintuitive.  In Marpa::R2 they have been dropped.

   Null actions now come from the rules
       In Marpa::XS null actions were specified by symbol.  This created a dual semantics -- one
       for non-nulled rules, and another for nulled rules.   The conventions and behaviors of the
       two semantics were quite dissimilar.  The rules for their coordination were complicated,
       and it was possible for a programmer expecting one semantics, to be surprised by a result
       from the other.

       In Marpa::R2 the semantics of nulled rules is the same as that of non-nulled rules, and
       the semantics of nulled symbols comes from the semantics of the nulled rules.  This
       requires rule evaluation closures to be aware they might be called for nulled rules.  But
       it greatly simplifies the semantics conceptually.  For more detail, see
       Marpa::R2::NAIF::Semantics::Null.

   Actions can now be constants
       If an action name resolves to a constant, that constant is the action.  The effect is the
       same as if the action name resolved to a function that returned that constant, except that
       it is more efficient.

       Perl cannot reliably distinguish between non-existent symbols and symbols whose value is
       "undef", so constants whose value is "undef" are not allowed.  The "::undef" reserved
       action name can be used instead.

   Actions names beginning with ""::"" are reserved
       Action names which start with ""::"" are reserved.  ""::undef"" is a safe way of specify a
       constant whose value is "undef".  Use of a reserved name which has not yet been defined
       causes an exception to be thrown.

   The "default_null_value" named argument for grammars has been removed
       Symbols no longer have null values, so the "default_null_value" named argument of grammars
       has been removed.

   The "null_value" symbol property has been removed
       Symbols no longer have null values.  Use of the "null value" symbol property now causes an
       exception.

   The token value argument of read() has changed
       The Marpa::R2 recognizer's "read()" method differs from its Marpa::XS equivalent.  In
       Marpa::R2, If "read()"'s token value argument is omitted, then the value of the token will
       be a Perl "undef".  If "read()"'s token value is given explicitly, then that explicit
       value will be the value of the token.  In particular, an explicit "undef" token value
       argument will behave differently from an omitted token value argument.  For details, see
       the documentation of recognizer's "read" method.

   The token value argument of "alternative()" has changed
       The Marpa::R2 recognizer's "alternative()" method differs from its Marpa::XS equivalent.
       Its token value argument must now be a reference to the token value, not the token value
       itself, as in Marpa::XS.  If alternative's token value argument is omitted or a Perl
       "undef", then the value of the token will be a Perl "undef".  If alternative's token value
       argument is reference to "undef", then the value of the token is a Perl "undef".  For
       details, see the documentation of the "alternative" method.

   Marpa::R2::Recognizer::value() does not accept named arguments
       In the Marpa::XS recognizer, the "new()", "set()" and value() methods all accepted named
       arguments.  As of Marpa::R2, the "value()" method will no longer do so.

       Allowing named arguments for the "value()" was a holdover from a previous interface, which
       also seemed like it might be a convenience.  But, since it was even more important that
       the "value()" method be convenient as the termination test controlling a loop over the
       parse results, a lot of special logic was added to deal with arguments which only made
       sense before the first pass of the loop, etc., etc.

       Eliminating named arguments from the "value()" method eliminates a variety of special
       cases and, as a result, the documentation of the "value()" method is now simpler, shorter
       and clearer.  Anything that could be done by providing named arguments to the "value()"
       method can be done more using the recognizer's "set()" method, and the code will be
       clearer for it.

   Marpa's grammar rewriting is now invisible
       Internally, Marpa rewrites its grammars.  In Marpa::XS, most details of these rewrites
       were invisible, but not all.  In Marpa::R2, all internal rules and symbols are now
       completely invisible to the user, even in the tools for debugging grammars.

   By default, the non-LHS symbols are the terminals
       Traditionally, a symbol has been a terminal if it is not on the LHS of any rule, and vice
       versa.  This is now the default in Marpa::R2, replacing the more complicated, and less
       intuitive, scheme that was in Marpa::XS.  Marpa::R2 still allows the user to use any non-
       nulling symbol as a terminal, including those symbols that appear on the LHS of a rule,
       but this is now an option, and never the default.  For more, see "Terminal symbols" in
       Marpa::R2::NAIF::Grammar.

   The lhs_terminals grammar named argument has been eliminated
       The lhs_terminals named argument of grammar objects implemented what is now the default
       behavior.  Since it no longer performs a function, its use is now a fatal error.

   Nulling symbols cannot be terminals
       In Marpa::XS, it was possible for a symbol to be both nulling and a terminal.  In practice
       that meant that the symbol was nulling, but that, on input, that property could be
       overriden, and a specific instance of the nulling symbol could be made non-nulling.  This
       behavior was worse than useless and non-intuitive -- it was dangerous and logically
       inconsistent.

       Marpa::R2 will not allow a nulling symbol to be used as a terminal.  To the extent that
       the Marpa::XS behavior made sense, it can be duplicated by creating a symbol which is the
       LHS of two rules, one empty, and the other rule with a RHS consisting of exactly one
       terminal symbol.

   A sequence must have a unique LHS
       The LHS of a sequence rule may not be on the LHS of any other rule, whether another
       sequence rule, or a BNF rule.  This is not as severe a restriction as it might sound --
       while sequences cannot share the same LHS with other rules directly, they can do so
       indirectly.  For details, see "Duplicate rules" in Marpa::R2::NAIF::Grammar.

       In Marpa::XS, the definition of when a sequence was a duplicate was more liberal, but it
       was also complicated and non-intuitive.  The new definition is simpler and more intuitive,
       and its greater restrictiveness is easy to work around.

   The terminal status of a symbol is locked once set
       Once a symbol is marked as a terminal or a non-terminal, its terminal status cannot be
       changed.  We doubt this will affect any actual applications.  It would only affect an
       application that changes symbols from their default status to non-terminal, and then only
       if they attempted to mark the same symbol as a terminal at another point.  Few Marpa::R2
       applications change symbols from their default terminal status, and none to my knowledge
       mark symbols as non-terminals.

   Evaluation of infinite loops has been changed
       Infinite loops (cycles) are still, by default, fatal errors.  For those considering
       programming with them, and evaluating parses from grammars with cycles, the semantics of
       cycles is now more closely specified.  For details of the new semantics, see
       Marpa::R2::NAIF::Semantics::Infinite.

   The range of values allowed for ranks has been clarified
       Symbols and rules have numeric ranks.  Previously, no mention was made of range of values
       allowed.  This is implemented-defined, except that the magnitudes of the ends of the range
       will always be at least the 28th power of 2, less 1.  That is, numbers in the range
       between -134,217,727 and 134,217,727 will always be allowed as ranks.

Copyright and License

         Copyright 2014 Jeffrey Kegler
         This file is part of Marpa::R2.  Marpa::R2 is free software: you can
         redistribute it and/or modify it under the terms of the GNU Lesser
         General Public License as published by the Free Software Foundation,
         either version 3 of the License, or (at your option) any later version.

         Marpa::R2 is distributed in the hope that it will be useful,
         but WITHOUT ANY WARRANTY; without even the implied warranty of
         MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
         Lesser General Public License for more details.

         You should have received a copy of the GNU Lesser
         General Public License along with Marpa::R2.  If not, see
         http://www.gnu.org/licenses/.