Provided by: critcl_3.1.17+dfsg-1_all bug


       critcl_use - Using Critcl


       Welcome  to the C Runtime In Tcl, CriTcl for short, a system to build C extension packages
       for Tcl on the fly, from C code embedded within Tcl scripts, for  all  who  wish  to  make
       their code go faster.

       This  document  is a (hopefully) gentle introduction to Critcl by way of a series of small

       Readers which came directly to this document through a search or similar, and are thus  in
       need  of  an  overview of the whole system, are advised to read the Introduction To CriTcl

       The examples here cover both how to embed C into  Tcl  with  it,  and  how  to  build  the
       distributable  packages.   As such the intended audience are both writers of packages with
       embedded C, and people building such packages.  To make things easier the two themes  each
       have  their  own  section  in this document, enabling all readers to quickly skip the part
       they are not interested in.

       The sources of Critcl, should you have gotten them, contain several larger examples  show-
       casing various aspects of the system. These demonstration packages can all be found in the
       sub-directory "examples/" of the sources.


       This is the section for developers writing, or wishing to write,  a  package  embedding  C
       into Tcl via critcl.

       I  guess that we are allowed to asssume that you, gentle reader, are here because you have
       written some Tcl code which is not fast enough (any more) and you  wish  to  make  it  "go
       faster" by replacing parts (or all) of it with speedy C.

       Another,  and  I  believe  reasonable  assumption  to  make would be that you have already
       investigated and ruled out or done things like changes to data structures  and  algorithms
       which  reduce O(n*n) work to O (n*log n), O(n), or even O(1).  Of course, nothing prevents
       you from forging ahead here even if you have not done such. Still, even  in  that  case  I
       would recommend that you consider investigating this line of making your code go faster as

       Now, with these introductory words out of the way, lets jump into the meat of things.

       Starting simple, let us assume that the Tcl code in question is something like

                  proc math {x y z} {
                      return [expr {(sin($x)*rand())/$y**log($z)}]

       with the expression pretending to be something very complex and slow. Converting this to C
       we get:

                  critcl::cproc math {double x double y double z} double {
                      double up   = rand () * sin (x);
                      double down = pow(y, log (z));
                      return up/down;

       Notable about this translation:

       [1]    All the arguments got type information added to them, here "double".  Like in C the
              type precedes the  argument  name.  Other  than  that  it  is  pretty  much  a  Tcl
              dictionary, with keys and values swapped.

       [2]    We now also have to declare the type of the result, here "double", again.

       [3]    The  reference  manpage  lists  all  the  legal  C types supported as arguments and

       When writing bindings to external libraries critcl::cproc is usually the  most  convenient
       way  of  writing the lower layers. This is however hampered by the fact that critcl on its
       own only supports a few standard (arguably the most import) standard  types,  whereas  the
       functions  we  wish  to  bind most certainly will use much more, specific to the library's

       The critcl commands argtype, resulttype and their adjuncts are provided to help  here,  by
       allowing a developer to extend critcl's type system with custom conversions.

       This and the three following sections will demonstrate this, from trivial to complex.

       The  most  trivial use is to create types which are aliases of existing types, standard or
       other. As an alias it simply copies and uses  the  conversion  code  from  the  referenced

       Our  example  is  pulled from an incomplete project of mine, a binding to Jeffrey Kegler's
       libmarpa library  managing  Earley  parsers.  Several  custom  types  simply  reflect  the
       typedef's  done  by  the  library,  to  make the critcl::cprocs as self-documenting as the
       underlying library functions themselves.

                  critcl::argtype Marpa_Symbol_ID     = int
                  critcl::argtype Marpa_Rule_ID       = int
                  critcl::argtype Marpa_Rule_Int      = int
                  critcl::argtype Marpa_Rank          = int
                  critcl::argtype Marpa_Earleme       = int
                  critcl::argtype Marpa_Earley_Set_ID = int


                  method sym-rank: proc {
                      Marpa_Symbol_ID sym
                      Marpa_Rank      rank
                  } Marpa_Rank {
                      return marpa_g_symbol_rank_set (instance->grammar, sym, rank);


       A more involved custom argument type would be to map from Tcl  strings  to  some  internal
       representation, like an integer code.

       The  first example is taken from the tclyaml package, a binding to the libyaml library. In
       a few places we have to map readable names for block styles, scalar styles,  etc.  to  the
       internal enumeration.

                  critcl::argtype yaml_sequence_style_t {
                      if (!encode_sequence_style (interp, @@, &@A)) return TCL_ERROR;


                  critcl::ccode {
                      static const char* ty_block_style_names [] = {
                          "any", "block", "flow", NULL

                      static int
                      encode_sequence_style (Tcl_Interp* interp, Tcl_Obj* style,
                                             yaml_sequence_style_t* estyle)
                          int value;
                          if (Tcl_GetIndexFromObj (interp, style, ty_block_style_names,
                                                   "sequence style", 0, &value) != TCL_OK) {
                              return 0;
                          *estyle = value;
                          return 1;


                  method sequence_start proc {
                      pstring anchor
                      pstring tag
                      int implicit
                      yaml_sequence_style_t style
                  } ok {
                      /* Syntax: <instance> seq_start <anchor> <tag> <implicit> <style> */


       It  should be noted that this code precedes the advent of the supporting generator package
       critcl::emap. using the generator the definition of the mapping becomes much simpler:

                  critcl::emap::def yaml_sequence_style_t {
                      any   0
                      block 1
                      flow  2

       Note that the generator will not  only  provide  the  conversions,  but  also  define  the
       argument  and result types needed for their use by critcl::cproc.  Another example of such
       a semi-trivial argument  type  can  be  found  in  the  CRIMP  package,  which  defines  a
       Tcl_ObjType  for image values. This not only provides a basic argument type for any image,
       but also derived types which check that the image has a specific format. Here we  see  for
       the  first  time  non-integer  arguments,  and  the  need  to  define the C types used for
       variables holding the C level value, and  the  type  of  function  parameters  (Due  to  C
       promotion rules we may need different types).

                  critcl::argtype image {
                      if (crimp_get_image_from_obj (interp, @@, &@A) != TCL_OK) {
                          return TCL_ERROR;
                  } crimp_image* crimp_image*


                      set map [list <<type>> $type]
                      critcl::argtype image_$type [string map $map {
                          if (crimp_get_image_from_obj (interp, @@, &@A) != TCL_OK) {
                              return TCL_ERROR;
                          if (@A->itype != crimp_imagetype_find ("crimp::image::<<type>>")) {
                              Tcl_SetObjResult (interp,
                                                Tcl_NewStringObj ("expected image type <<type>>",
                              return TCL_ERROR;
                      }] crimp_image* crimp_image*


       The  adjunct  command  critcl::argtypesupport  is for when the conversion needs additional
       definitions, for example a helper structure.

       An example of this can be found among the standard types of  critcl  itself,  the  pstring
       type.  This  type  provides  the C function with not only the string pointer, but also the
       string  length,  and  the  Tcl_Obj*  this  data  came  from.  As  critcl::cproc's  calling
       conventions allow us only one argument for the data of the parameter a structure is needed
       to convey these three pieces of information.

       Thus the argument type is defined as

                  critcl::argtype pstring {
                      @A.s = Tcl_GetStringFromObj(@@, &(@A.len));
                      @A.o = @@;
                  } critcl_pstring critcl_pstring

                  critcl::argtypesupport pstring {
                      typedef struct critcl_pstring {
                          Tcl_Obj* o;
                          char*    s;
                          int      len;
                      } critcl_pstring;

       In the case of such a structure being large we may wish to allocate it on the heap instead
       of  having  it  taking  space on the stack. If we do that we need another adjunct command,
       critcl::argtyperelease. This command specifies the code required  to  release  dynamically
       allocated  resources  when  the  worker  function  returns, before the shim returns to the
       caller in Tcl.  To keep things simple our example is synthetic, a modification of  pstring
       above,  to  demonstrate  the technique. An actual, but more complex example is the code to
       support the variadic args argument of critcl::cproc.

                  critcl::argtype pstring {
                      @A = (critcl_pstring*) ckalloc(sizeof(critcl_pstring));
                      @A->s = Tcl_GetStringFromObj(@@, &(@A->len));
                      @A->o = @@;
                  } critcl_pstring* critcl_pstring*

                  critcl::argtypesupport pstring {
                      typedef struct critcl_pstring {
                          Tcl_Obj* o;
                          char*    s;
                          int      len;
                      } critcl_pstring;

                  critcl::argtyperelease pstring {
                      ckfree ((char*)) @A);

       Note, the above example shows only the most simple case of an allocated argument,  with  a
       conversion  that  cannot  fail (namely, string retrieval). If the conversion can fail then
       either allocation has to be defered to  happen  only  on  successful  conversion,  or  the
       conversion code has to release the allocated memory itself in the failure path, because it
       will never reach the code defined via critcl::argtyperelease in that case.

       All of the previous sections dealt with argument conversions, i.e. going from Tcl into  C.
       Custom result types are for the reverse direction, from C to Tcl.  This is usually easier,
       as most of the time errors should not be possible. Supporting  structures,  or  allocating
       them on the heap are not really required and therefore not supported.

       The  example  of a result type shown below was pulled from KineTcl. It is a variant of the
       builtin result type Tcl_Obj*, aka object. The builtin conversion assumes that  the  object
       returned  by  the  function has a refcount of 1 (or higher), with the function having held
       the reference, and releases that reference after placing the value into the interp result.
       The conversion below on the other hand assumes that the value has a refcount of 0 and thus
       that decrementing it is forbidden, lest it be released much to  early,  and  crashing  the

                  critcl::resulttype KTcl_Obj* {
                      if (rv == NULL) { return TCL_ERROR; }
                      Tcl_SetObjResult(interp, rv);
                      /* No refcount adjustment */
                      return TCL_OK;
                  } Tcl_Obj*

       This  type  of  definition  is  also  found  in  Marpa and recent hacking hacking on CRIMP
       introduced it there as well. Which is why this definition became a builtin  type  starting
       with version 3.1.16, under the names Tcl_Obj*0 and object0.

       Going  back to errors and their handling, of course, if a function we are wrapping signals
       them in-band, then the conversion of such results has to deal with that. This happens  for
       example in KineTcl, where we find

                  critcl::resulttype XnStatus {
                      if (rv != XN_STATUS_OK) {
                          Tcl_AppendResult (interp, xnGetStatusString (rv), NULL);
                          return TCL_ERROR;
                      return TCL_OK;

                  critcl::resulttype XnDepthPixel {
                      if (rv == ((XnDepthPixel) -1)) {
                          Tcl_AppendResult (interp,
                                            "Inheritance error: Not a depth generator",
                          return TCL_ERROR;
                      Tcl_SetObjResult (interp, Tcl_NewIntObj (rv));
                      return TCL_OK;

       In  A  Simple Procedure we demonstrated how easy a translation to C can be. Is it still as
       easy when we introduce something moderately complex like handling  a  variable  number  of
       arguments  ?  A  feature  which  is  needed  to  handle commands with options and optional
       arguments ?

       Well, starting with version 3.1.16 critcl::cproc  does  have  full  support  for  optional
       arguments,  args-style  variadics,  and  default values, extending its range to everything
       covered by the builtin proc. The only thing not truly supported  are  options  (i.e.  flag
       arguments) of any kind.

       For  the moment, and the example, let us pretend that we can use critcl::cproc only if the
       number of arguments is fully known beforehand, i.e. at the time of declaration.   Then  we
       have to use critcl::ccommand to handle the translation of the procedure shown below:

                  proc math {args} {
                      set sum 0
                      foreach y $args { set sum [expr {$sum + $y}] }
                      return $sum

       Its  advantage:  Access to the low-level C arguments representing the Tcl arguments of the
       command. Full control over argument conversion, argument validation, etc.

       Its disadvantage: Access to the low-level C arguments representing the  Tcl  arguments  of
       the  command.  Assuming  the  burden  of  having  to  write  argument conversion, argument
       validation, etc. Where critcl::cproc handles the task of converting from Tcl to  C  values
       (for  arguments)  and  back (for the result), with critcl::command it is the developer who
       has to write all this code.

       Under our restriction the translation of the example is:

                  critcl::ccommand math {cd ip oc ov} {
                      double sum = 0;
                      double y;

                      oc --;
                      while (oc) {
                          if (Tcl_GetDoubleFromObj (ip, ov[oc], &y) != TCL_OK) {
                              return TCL_ERROR;
                          sum += y;
                          oc --;

                      Tcl_SetObjResult (ip, Tcl_NewDoubleObj (sum));
                      return TCL_OK:

       Notable about this translation:

       [1]    As promised/threatened, all the conversions between  the  Tcl  and  C  domains  are
              exposed, and the developer should know her way around Tcl's C API.

       [2]    The four arguments "cd ip oc ov" are our names for the low-level arguments holding

              [1]    ClientData (reference)

              [2]    Tcl_Interp (reference)

              [3]    Number of arguments, and

              [4]    Array of argument values, each a Tcl_Obj*.

              This  list  of  arguments,  while  not  optional in itself, is allowed to be empty,
              and/or to contain empty strings as argument names. If we do that critcl will supply
              standard names for the missing pieces, namely:

              [1]    clientdata

              [2]    interp

              [3]    objc

              [4]    objv

       Now,  letting  go  of our pretenses regarding the limitations of critcl::cproc, due to the
       support it does have for args-style variadics (since version 3.1.16) we can write  a  much
       simpler translation:

                  critcl::cproc math {double args} double {
                      double sum = 0;

                      args.c --;
                      while (args.c) {
                          sum += args.v[args.c];
                          args.c --;
                      return sum;

       Here  we  assume  that  we have a Tcl procedure which returns a fixed string. In the final
       product we are going to C to hide this string from the casual user.

                  proc somedata {} {
                return {... A large blob of characters ...}

       The translation of this is simple and easy:

                  package require critcl

                  critcl::cdata somedata {... A large blob of characters ...}

       There is nothing really notable here.

       Often just defining Tcl commands in C, as demonstrated in the sections A Simple Procedure,
       Handling  A  Variable Number Of Arguments, and Data As A Tcl Command is not really enough.
       For example we may have several of our new C commands using the same code over  and  over,
       and  we  wish  avoid  this duplication. Or we wish to pull in declarations and definitions
       from some external library.

       In both cases we require the ability to embed an unstructured block of C  code  which  can
       contain  whatever  we want, defines, functions, includes, etc. without being directly tied
       to Tcl commands.  The command critcl::code provides us with exactly that.  As our  example
       now  an  excerpt  taken from real code, the top of the "sha1c.tcl" critcl file in the sha1
       module of Tcllib:

                  critcl::ccode {
                      #include "sha1.h"
                      #include <stdlib.h>
                      #include <assert.h>

                      Tcl_ObjType sha1_type; /* fast internal access representation */

                      static void
                      sha1_free_rep(Tcl_Obj* obj)
                          SHA1_CTX* mp = (SHA1_CTX*) obj->internalRep.otherValuePtr;


       We see here the beginning of the C code defining a custom Tcl_ObjType holding the data  of
       a SHA1 context used during the incremental calculation of sha1 hashes.

       While  one  might believe that there is no need for commands which returns constant values
       that is not true. Commands reporting on compile-time configuration, like version  numbers,
       available features, etc. are at least one use case for such commands.

       The  reason  for creating critcl commands to support them ?  Convenience to the user, yes,
       but further than that, the ability to optimize the internals, i.e. the generated code.

       A cproc would be easy to write, but incurs overhead due to a superfluous work function.  A
       ccommand  has  no  overhead, except that of the user having to write the argument checking
       and result conversion.

       Using critcl::cconst is both convenient and without  code  overhead.   Our  example  is  a
       function  found  in  package  tcl-linenoise, that is, if cconst had existed at the time of
       writing. It returns a configuration value reporting to the policy  layer  if  an  extended
       mode for hidden input is available from the bound linenoise, or not.

                  critcl::cconst linenoise::hidden_extended boolean 1

       When  writing a critcl-based package to make a third-party library available to scripts we
       do not only have to make the relevant functions available as commands, often we also  have
       to know all the various constants, flags, etc. these functions take.

       Rather  than  writing  such magic numbers directly we would greatly prefer to use symbolic
       names instead.  Instead of providing one or more  commands  to  list  and  map  the  magic
       numbers  to  strings  critcl  only  provides a single command which allows the export of C
       defines and enumeration values, mapping them to Tcl variables of the  given  names,  whose
       values are the associated magic numbers.

       This  is  good  enough  because the developers of the third-party library were very likely
       like us and wanted to use symbolic names instead of magic numbers. Which in C are declared
       as via defines and enumeration types. We just have to lift them up.

       Our  example  comes from cryptkit, a Tcl binding to cryptlib, a cryptography library.  The

                  critcl::cdefines CRYPT_* ::crypt

       maps all Cryptlib specific #defines and enums into the namespace ::crypt,  telling  critcl
       to create aliases to the symbols.


                  critcl::cdefines {
                  } ::crypt

       maps the listed defines into the namespace ::crypt.

       An  important thing to note: These commands do not create the defines in the C level. They
       only lift pre-existing material.  Which can come  from  the  headers  of  the  third-party
       library, the usual case, but also from Blocks of arbitrary C.

       A  corrollary  to  the  above:  What  is not where, cannot be lifted. All listed names and
       patterns which have no actual C code declaring them are ignored, i.e. not mapped.

       A notable thing in the example shown in the section about Blocks of arbitrary C is the

                  #include "sha1.h"

       statement. Where does this header come from ?  Looking at the Tcllib module we  will  find
       that  the  header  is actually a sibling to the "sha1c.tcl" file containing the embedded C
       code.  However, critcl does not know  that.  It  has  to  be  told.   While  without  that
       knowledge  it  will  invoke  the compiler just fine, the compilation will fail because the
       header is not on the include paths used by the compiler, and therefore will not be found.

       For this we have the critcl::cheaders command. It enables us to either tell  the  compiler
       the path(s) where the required headers can be found, using

                  critcl::cheaders -I/path/to/headers/

       or to tell it directly which headers we are using and where they live:

                  critcl::cheaders sha1.h

       And  now  critcl knows that "sha1.h" is important, and that it lives besides the ".critcl"
       file which referenced it (because of the relative path  used).   Note  that  this  doesn't
       absolve  us  of the need to "#include" the header through a critcl::ccode block. This only
       tells critcl where it lives so that it can configure the compiler with the proper  include
       paths to actually find it on use.

       Further note that a C development environment is usually configured to find all the system
       headers, obviating the need for a critcl::cheaders declaration when  such  are  used.  For
       these  a  plain  "#include"  in a critcl::ccode block is good enough.  In other words, the
       second form of invoking critcl::cheaders is pretty much only for headers  which  accompany
       the ".critcl" file.

       In  all  of  the  examples shown so far the C code was fully embedded in a ".critcl" file.
       However, if the C part is large it can make sense to break it out of  the  ".critcl"  file
       into one or more separate proper ".c" file(s).

       The  critcl::csources  command  can  then  be used to make this code known to the original
       ".critcl" file again.  This command accepts the paths to the ".c" files as arguments,  and
       glob  patterns  as well.  Our example comes from the struct::graph package in Tcllib.  Its
       core C functions are in separate files, and the ".critcl" code then makes them known via:

              namespace eval ::struct {
                  # Supporting code for the main command.
                  critcl::cheaders graph/*.h
                  critcl::csources graph/*.c


       which tells critcl that these files are  in  the  subdirectory  "graph"  relative  to  the
       location of "graph_c.tcl", which is the relevant ".critcl" file.

       This  example  also  demonstrates  again the use of critcl::cheaders, which we also saw in
       section Finding header files.

       When creating a package exposing some third-party library to Tcl Finding header  files  is
       only  the  first  part,  to  enable  failure-free  compilation.  We  also have to find the
       library/ies themselves so that they can be linked to our package. This is described  here.
       The  last  issue,  Lifting  constants from C to Tcl for the use by scripts is handled in a
       separate section and example.

       The relevant  command  is  critcl::clibraries.  Its  basic  semantics  are  like  that  of
       critcl::cheaders,  i.e.  It  enables  us to tell the linker the path(s) where the required
       libraries can be found, using

                  critcl::clibraries -L/path/to/libraries/

       name them

                  critcl::clibraries -lfoo

       or tell it directly which libraries we are using and where they live:

                  critcl::clibraries /path/to/library/

       This last way of using should be avoided however, as it intermingles searching and naming,
       plus the name is platform dependent.

       For  OS  X we additionally have the critcl::framework command which enables us to name the
       frameworks used by our package.  Note that this command can be  used  unconditionally.  If
       the build target is not OS X it is ignored.

       The  commands  critcl::cflags  and critcl::ldflags enable you to provide custom options to
       the compile and link phases for a ".critcl" file.

       This usually becomes necessary if the C code in question comes from an external library we
       are writing a Tcl binding for, with multiple configurations to select, non-standard header
       locations, and other things.  Among the latter, especially platform-specific settings, for
       example byteorder.

       This makes critcl::check an important adjunct command, as this is the API for Checking The
       Environment, and then selecting the compile & link flags to use.

       I currently have no specific example to demonstrate these commands.

       Often enough only pieces of a package require recoding in C to boost the whole system. Or,
       alternatively,  the  package  in question consists of a low-level layer C with a Tcl layer
       above encoding policies and routing to the proper low-level calls, creating a nicer (high-
       level) API to the low-level functionality, etc.

       For  all  of  this  we  have  to be able to write a package which contains both C and Tcl,
       nevermind the fact the C parts are embedded in Tcl.

       The easiest way to structure such a  package  is  to  have  several  files,  each  with  a
       different  duty. First, a ".critcl" file containing the embedded C, and second one or more
       ".tcl" files providing the Tcl parts.   Then  use  the  critcl::tsources  command  in  the
       ".critcl"  file  to  link  the two parts together, declaring the ".tcl" files as necessary
       companions of the C part.

                  package require critcl

                  critcl::tsources your-companion.tcl ; # Companion file to use

                  ... embedded C via critcl commands ...

       With a declaration as shown above the companion file will be  automatically  sourced  when
       the C parts are made available, thus making the Tcl parts available as well.

       There  is  one  special case of Having both C and Tcl functionality which deserves its own

       The possibility of not having the fast C code on some platform, and  using  a  slower  Tcl
       implementation  of  the  functionality. In other words, a fallback which keeps the package
       working in the face of failure to build the C parts. A more concrete example of this would
       be a module implementing the SHA hash, in both C and Tcl, and using the latter if and only
       if the C implementation is not available.

       There two major possibilities in handling such a situation.

       [1]    Keep all the pieces separated. In that  scenario  our  concrete  example  would  be
              spread  over  three packages. Two low-level packages sha::c and sha::tcl containing
              the two implementations of the algorithm, and, thirdly, a coordinator  package  sha
              which loads either of them, based on availability.

              The  Tcllib  bundle  of  packages  contains a number of packages structured in this
              manner, mostly in the struct module.

              Writing the C and Tcl parts should be simple by now, with all the examples  we  had
              so  far. The only non-trivial part is the coordinator, and even that if and only if
              we wish to make it easy to write a testsuite which can check both branches, C,  and
              Tcl without gymnastics. So, the most basic coordinator would be

                  set sha::version 1
                  if {[catch {
                      package require sha::c $sha::version
                  }]} {
                      package require sha::tcl $sha::version
                  package provide sha $sha::version

              It  tries  to  load  the  C  implementation  first,  and  falls  back  to  the  Tcl
              implementation if that fails. The code as  is  assumes  that  both  implementations
              create  exactly the same command names, leaving the caller unaware of the choice of

              A concrete example of this scheme can be found in Tcllib's md5  package.  While  it
              actually  uses  ythe  Trf  as  its  accelerator, and not a critcl-based package the
              principle is the same. It also demonstrates the need for additional glue code  when
              the  C  implementation doesn't exactly match the signature and semantics of the Tcl

              This basic coordinator can be easily extended to try more than two packages to  get
              the  needed implementation. for example, the C implementation may not just exist in
              a sha::c package, but also bundled somewhere  else.  Tcllib,  for  example,  has  a
              tcllibc  package which bundles all the C parts of its packages which have them in a
              single binary.

              Another direction to take it in is to  write  code  which  allows  the  loading  of
              multiple  implementations  at  the  same  time,  and then switching between them at
              runtime. Doing this requires effort to keep the implementations out of each  others
              way,  i.e.  they  cannot provide the same command names anymore, and a more complex
              coordinator as well, which is able to map from the public command names to whatever
              is provided by the implementation.

              The  main  benefit  of  this  extension  is that it makes testing the two different
              implementations easier, simply run through the same set of  tests  multiple  times,
              each  time with different implementation active. The disadvantage is the additional
              complexity of the coordinator's internals. As a larger example  of  this  technique
              here   is  the  coordinator  "modules/struct/queue.tcl"  handling  the  C  and  Tcl
              implementations of Tcllib's struct::queue package:

                  # queue.tcl --
                  #       Implementation of a queue data structure for Tcl.

                  package require Tcl 8.4
                  namespace eval ::struct::queue {}

                  ## Management of queue implementations.

                  # ::struct::queue::LoadAccelerator --
                  #       Loads a named implementation, if possible.

                  proc ::struct::queue::LoadAccelerator {key} {
                      variable accel
                      set r 0
                      switch -exact -- $key {
                          critcl {
                              # Critcl implementation of queue requires Tcl 8.4.
                              if {![package vsatisfies [package provide Tcl] 8.4]} {return 0}
                              if {[catch {package require tcllibc}]} {return 0}
                              set r [llength [info commands ::struct::queue_critcl]]
                          tcl {
                              variable selfdir
                              if {
                                  [package vsatisfies [package provide Tcl] 8.5] &&
                                  ![catch {package require TclOO}]
                              } {
                                  source [file join $selfdir queue_oo.tcl]
                              } else {
                                  source [file join $selfdir queue_tcl.tcl]
                              set r 1
                          default {
                              return -code error "invalid accelerator/impl. package $key: must be one of [join [KnownImplementations] {, }]"
                      set accel($key) $r
                      return $r

                  # ::struct::queue::SwitchTo --
                  #       Activates a loaded named implementation.

                  proc ::struct::queue::SwitchTo {key} {
                      variable accel
                      variable loaded

                      if {[string equal $key $loaded]} {
                          # No change, nothing to do.
                      } elseif {![string equal $key ""]} {
                          # Validate the target implementation of the switch.

                          if {![info exists accel($key)]} {
                              return -code error "Unable to activate unknown implementation \"$key\""
                          } elseif {![info exists accel($key)] || !$accel($key)} {
                              return -code error "Unable to activate missing implementation \"$key\""

                      # Deactivate the previous implementation, if there was any.

                      if {![string equal $loaded ""]} {
                          rename ::struct::queue ::struct::queue_$loaded

                      # Activate the new implementation, if there is any.

                      if {![string equal $key ""]} {
                          rename ::struct::queue_$key ::struct::queue

                      # Remember the active implementation, for deactivation by future
                      # switches.

                      set loaded $key

                  # ::struct::queue::Implementations --
                  #       Determines which implementations are
                  #       present, i.e. loaded.

                  proc ::struct::queue::Implementations {} {
                      variable accel
                      set res {}
                      foreach n [array names accel] {
                          if {!$accel($n)} continue
                          lappend res $n
                      return $res

                  # ::struct::queue::KnownImplementations --
                  #       Determines which implementations are known
                  #       as possible implementations.

                  proc ::struct::queue::KnownImplementations {} {
                      return {critcl tcl}

                  proc ::struct::queue::Names {} {
                      return {
                          critcl {tcllibc based}
                          tcl    {pure Tcl}

                  ## Initialization: Data structures.

                  namespace eval ::struct::queue {
                      variable  selfdir [file dirname [info script]]
                      variable  accel
                      array set accel   {tcl 0 critcl 0}
                      variable  loaded  {}

                  ## Initialization: Choose an implementation,
                  ## most preferred first. Loads only one of the
                  ## possible implementations. And activates it.

                  namespace eval ::struct::queue {
                      variable e
                      foreach e [KnownImplementations] {
                          if {[LoadAccelerator $e]} {
                              SwitchTo $e
                      unset e

                  ## Ready

                  namespace eval ::struct {
                      # Export the constructor command.
                      namespace export queue

                  package provide struct::queue 1.4.2

              In this implementation the  coordinator  renames  the  commands  of  the  low-level
              packages  to  the  public  commands,  making  the future dispatch as fast as if the
              commands had these names anyway, but also forcing a spike of bytecode recompilation
              if  switching  is ever done at the runtime of an application, and not just used for
              testing, and possibly disrupting introspection by the commands, especially if  they
              move between different namespaces.

              A  different  implementation  would be to provide the public commands as procedures
              which consult a variable to  determine  which  of  the  loaded  implementations  is
              active, and then call on its commands. This doesn't disrupt introspection, nor does
              it trigger bytecode recompilation on switching. But it takes more time to  dispatch
              to  the  actual  implementation, in every call of the public API for the package in

              A concrete example of this scheme can be found in Tcllib's crc32 package.

       [2]    Mix the pieces together. Please note that while I am describing how  to  make  this
              work  I  strongly  prefer  and recommend to use the previously shown approach using
              separate files/packages. It is much easier to understand and  maintain.  With  this
              warning done, lets go into the nuts and bolts.

              If we care only about mode "compile & run" things are easy:

                  package require critcl

                  if {![critcl::compiling]} {
                      proc mycommand {...} {

                  } else {
                      critcl::cproc mycommand {...} {

              The command critcl::compiling tells us whether we have a compiler available or not,
              and in the latter case we implement our command in Tcl.

              Now what happens when we invoke mode "generate package" ?  ... compiler failure ...
              ...  ok    -  C  code - everything fine ... fail - no package ? or just no C code ?
              declare self as tsource, to be used ?  ... platform-specific C/Tcl -- uuid.

       By default critcl is a bit inconsistent  between  modes  "compile  &  run"  and  "generate
       package".  The  result of the latter is a standard Tcl package which loads and sources all
       of its files immediately when it is  required.  Whereas  "compile  &  run"  defers  actual
       compilation,  linking,  and  loading  until the first time one of the declared commands is
       actually used, making this very lazy.

       This behaviour can be quite unwanted if Tcl companion files,  or  other  users  of  the  C
       commands use introspection to determine the features they have available. Just using [info
       commands] doesn't cut it, the auto_index array has to be checked as  well,  making  things
       quite inconvenient for the users.

       To  fix  this  issue  at  the source, instead of in each user, be it inside of the package
       itself, or other packages, we have the command critcl::load.  Used as the last command  in
       a  ".critcl"  file  it  forces  the  compile,  link, and load trinity, ensuring that all C
       commands are available immediately.

                  package require critcl

                  ... Declare C procedures, commands, etc.

                  critcl::load ; # Force build and loading.

       Note that is not allowed, nor possible to use critcl  commands  declaring  anything  after
       critcl::load has been called. I.e., code like

                  package require critcl

                  ... Declare C procedures, commands, etc.

                  critcl::load ; # Force build and loading.

                  ... More declarations of C code, ...
                  critcl::code { ... }

       will result in an error. The only package-related commands still allowed are

       [1]    critcl::done

       [2]    critcl::failed

       [3]    critcl::load

       as  these  only  query  information,  namely  the  build status, and are protected against
       multiple calls.


       As said several times, by default critcl defers the compile and  link  steps  for  a  file
       until  it  is needed, i.e. the first command of the ".critcl" file in question is actually

       This not only has the effect of lazily loading the package's functionality, but also, when
       developing  using  mode  "compile & run", of us not seeing any errors in our code until we
       are actually trying to run some demonstration.

       If we do not wish to have such a delay we have to be able to force at least the  execution
       of the compile step.

       The  command critcl::failed is exactly that. When called it forcibly builds the C code for
       the ".critcl" file it is part of, and returns a boolean vlaue signaling failure (true), or
       success (false).

                  package require critcl

                  ... Declare C procedures, commands, etc.

                  if {[critcl::failed]} {
                      ... signal error

       It  is  related  and similar to critcl::load, the command to overcome the lazy loading, as
       shown in section Unlazy Packages.

       Like it is not allowed, nor possible to  use  critcl  commands  declaring  anything  after
       critcl::failed  has  been  called,  making  it  pretty  much  the last critcl command in a
       ".critcl" file.  Code like

                  package require critcl

                  ... Declare C procedures, commands, etc.

                  if {[critcl::failed]} { ... }

                  ... More declarations of C code, ...
                  critcl::code { ... }

       will result in an error. The only package-related commands still allowed are

       [1]    critcl::done

       [2]    critcl::failed

       [3]    critcl::load

       as these only query information, namely  the  build  status,  and  are  protected  against
       multiple calls.

       When building the shared library from the embedded C sources one of the things critcl does
       for us is to provide the Tcl headers, especially the stubs declarations.

       By default these are the Tcl 8.4 headers and stubs, which covers 90% of  the  cases.  What
       when  the  package  in  question  is  meant  for use with Tcl 8.5 or higher, using C-level
       features of this version of Tcl.

       Use the critcl::tcl command to declare to critcl the minimum version of  Tcl  required  to
       operate  the  package.  This  can be either 8.4, 8.5, or 8.6, and critcl then supplies the
       proper headers and stubs.

                  package require critcl
                  critcl::tcl 8.5

                  ... Declare your code ...

       ... requires compiling against the Tk headers,  and  linking  with  Tk's  stubs.  For  our
       convenience we have a simple, single command to activate all the necessary machinery, with
       critcl supplying the header files and stubs C code, instead of having to make it  work  on
       our own via critcl::cflags, critcl::ldflags, critcl::cheaders, critcl::csources.

       This command is critcl::tk.

                  package require critcl
                  critcl::tk ; # And now critcl knows to put in the Tk headers and other support.

                  ... Declare your code ...

       Please  note  that  this doesn't release you from the necessity of learning Tk's C API and
       how to use it to make a widget work. Sorry.

       ... may be necessary when creating a binding to some third-party library. The headers  for
       this  library may be found in non-standard locations, ditto for the library/ies itself. We
       may not have the headers and/or library on the build host. Types  with  platform-dependent
       sizes and definitions. Endianness issues. Any number of things.

       TEA-based  packages  can use autoconf and various predefined macros to deal with all this.
       We have the Power Of Tcl (tm) and critcl::check.

       This command takes a piece of C code as argument, like critcl::ccode. Instead of saving it
       for later it however tries to compile it immediately, using the current settings, and then
       returns a boolean value reporting on the success (true) or failure (false). From there  we
       can then branch to different declarations.

       As example let us check for the existence of some header "FOO.h":

                  package require critcl

                  if {[critcl::check {
                      #include <FOO.h>
                  }]} {
                      ... Code for when FOO.h is present.
                  } else {
                      ... Code for when FOO.h is not present.

       Should  we,  on  the  other hand, wish to search for the header ourselves, in non-standard
       locations we have the full power of Tcl available, i.e. loops, the file and glob commands,
       etc.,  which  can then be followed by a critcl::cheader command to declare the location we
       found (See also Finding header files).

       A nice extension to critcl would be a package collecting pocedures for common  tasks  like
       that,  sort  of  like an autoconf for Tcl. critcl::config seems to be nice name for such a

       Obvious adjunct commands which can be driven by results from critcl::check are






       Less obvious, yet still valid are also







       and pretty much everything else you can imagine.

       When writing packages it is always good manners to  provide  prospective  users  with  the
       license  the  package is under, so that they can decide whether they truly want to use the
       package, or not.

       As critcl-based packages often consist of only a single file a nice way of doing  that  is
       to  embed the license in that file. By using a critcl command, namely critcl::license this
       information is then also available to the critcl application, which  can  put  it  into  a
       standard location, i.e. "license.terms", of the generated packages.

       I currently have no specific example to demonstrate the command.


       This  is  the section for developers having to generate packages from ".critcl" files, i.e
       binaries for deployment,

       ... Is easy. Running

                  critcl -help

       prints the basics of using the application to stdout.

       The default mode of the critcl application is to take a series of ".critcl"  files,  build
       their  binaries,  and  leave  them  behind  in  the result cache. When the files are later
       actually used the compile and link steps can be skipped, leading to shorter load times.

       The command line for this is

                  critcl foo.tcl

       or, to process multiple files

                  critcl foo.tcl bar.tcl ...

       One thing to be aware of, should critcl find that the cache already contains  the  results
       for  the  input  files,  no  building will be done. If you are sure that these results are
       outdated use the option -force to force(sic!) critcl to rebuild the binaries.

                  critcl -force foo.tcl

       For debugging purposes it may be handy to see the generated  intermediate  ".c"  files  as
       well. Their removal from the cache can be prevented by specifying the option -keep.

                  critcl -keep foo.tcl

       These can be combined, of course.

       To build the binary package for a ".critcl" file, instead of Pre-Filling The Result Cache,
       simply specify the option -pkg.

                  critcl -pkg foo.tcl

       This will geneate a package named foo.  A simpler alternative to the above is

                  critcl -pkg foo

       The application will automatically assume that the input file to look for is "foo.tcl".

       But what when the name of the input file is not the name of the package to build ? This we
       can handle as well:

                  critcl -pkg foo bar.tcl

       The argument foo specifies the name, and "bar.tcl" is the file to process.

       Going  back to the very first example, it is of course possible to use an absolute path to
       specify the file to process:

                  critcl -pkg /path/to/foo.tcl

       The package name derived from that is still foo.

       Here we assume that you know the basics of how to build a package.  If  not,  please  read
       section Building A Package first.

       By  default  critcl  will  place  all newly-made packages in the subdirectory "lib" of the
       current working directory.  I.e. running

                  critcl -pkg foo

       will create the directory "lib/foo" which contains all the files of the package.

       When this behaviour is unwanted the option -libdir is  available,  allowing  the  explicit
       specification of the destination location to use.

                  critcl -pkg -libdir /path/to/packages foo

       A  common  use might be to not only build the package in question, but to also immediately
       install it directly in the path where the user's tclsh will be able to find it.  Assuming,
       for  example,  that  the  tclsh in question is installed at "/path/to/bin/tclsh", with the
       packages searched for under "/path/to/lib" ([info library]), the command

                  critcl -pkg -libdir /path/to/lib foo

       will build the package and place it in the directory "/path/to/lib/foo".

       Here we assume that you know the basics of how to build a package.  If  not,  please  read
       section Building A Package first.

       An  important issue, when there is trouble with the package, debugging becomes necessary a
       evil.  Critcl supports this through the -debug option.  Using  it  enables  various  build
       modes which help with that.

       For  example,  to  activate  the Tcl core's built-in memory debugging subsystem build your
       package with

                  critcl -pkg -debug memory foo

       The  resulting  binary  for  package  foo  will  use  Tcl's  debug-enabled  (de)allocation
       functions,  making  them  visible to Tcl's memory command. This of course assumes that the
       Tcl core used was also built for memory debugging.

       Further, built your package with

                  critcl -pkg -debug symbols foo

       to see the foo's symbols (types, functions, variables, etc.)   when  inspecting  a  "core"
       file it is involved in with a symbolic debugger,

       To activate both memory debugging and symbols use either

                  critcl -pkg -debug all foo


                  critcl -pkg -debug symbols -debug memory foo

       The  configuration settings critcl uses to drive the compiler, linker, etc. are by default
       selected based on the platform it is run on, to generate binaries which properly  work  on
       this platform.

       There  is  one  main use-case for overriding this selection, which is done with the option

       [1]    Cross-compilation. The building of binaries for a platform T while critcl  actually
              runs  on  platform  B.  The standard configuration of critcl currently has settings
              for two cross-compilation targets. So, to build 32bit Windows binaries on  a  Linux
              host which has the Xmingw cross-compilation development environment installed use

                  critcl -pkg -target mingw32 foo

              Similarly,  building a package for use on ARM processors while critcl is running in
              an Intel environment use

                  critcl -pkg -target linux-arm foo

              Note that both configurations assume that the cross-compiling compiler, linke, etc.
              are found first in the PATH.

       The  compiler  configurations  coming  with  critcl  currently  cover all hosts having gcc
       installed (the foremost among these are Linux and OS X), plus the native compilers of  the
       more  common  unix-like  operating systems, i.e. Solaris, HP-UX, and AIX, and, on the non-
       unix side, Windows.

       Developers using operating systems and compilers outside of this range will either have to
       install  a  gcc-based  development  environment, i.e. get into the covered range, or write
       their own custom configuration and then tell critcl about it.

       The latter is the easier part,  given  that  critcl  supports  the  option  -config  whose
       argument is the path to the file containing the custom configuration(s). I.e.

                  critcl -config /path/to/config ...

       will run critcl with the custom configuration in "/path/to/config", with the other options
       and arguments as explained in previous sections. Depending on the choice of name  for  the
       new  configuration(s)  this  may  or  may  not  require  a  -target  option  to select the
       configuration needed.

       For the former, the writing of the custom configuration, the  reader  is  refered  to  the
       section  "Configuration  Internals"  of  the  CriTcl  Package  Reference for the necessary
       details.  This is an advanced topic pretty much out of scope for this tutorial beyond what
       was already said.

       Sometimes  the  use  of  critcl::headers  might  not  be  enough for a package to find its
       headers. Maybe they are outside of the paths checked by  the  setup  code.   To  help  the
       application  recognizes  the option -I which allows the user to supply a single additional
       include path to use during the build phase of the package.

       Simply use

                  critcl -I /path/to/header ...

       and the specified header will be handed to the package to be built.

       To see a list containing the names of all the available configurations, run

                  critcl -targets

       The configuration settings for either the default or user-chosen target can  be  inspected
       on stdout with

                  critcl -show


                  critcl -show -target TARGET

       The raw contents of the configuration file used by critcl are dumped to stdout with

                  critcl -showall

       All of the above can of course be combined with custom configuration files.


       Jean Claude Wippler, Steve Landers, Andreas Kupries


       This  document,  and  the  package  it  describes, will undoubtedly contain bugs and other
       problems.  Please report them at   Ideas
       for  enhancements  you  may have for either package, application, and/or the documentation
       are  also  very  welcome   and   should   be   reported   at
       kupries/critcl/issues as well.


       C code, Embedded C Code, code generator, compile & run, compiler, dynamic code generation,
       dynamic  compilation,  generate  package,  linker,  on  demand   compilation,   on-the-fly


       Glueing/Embedded C code


       Copyright (c) Jean-Claude Wippler
       Copyright (c) Steve Landers
       Copyright (c) 2011-2015 Andreas Kupries