The Misc-Extensions system provides several macros that I like to use.

See src/defs.lisp for the package declarations.

For writing code in a mostly-functional style, Common Lisp's multiple-value feature is extremely useful (I make heavy use of it in FSet, for example, both internally and in the API). But the most important primitive the language provides for receiving multiple values from a call, multiple-value-bind, is a bit clunky to use. First, its long name would be appropriate for a rarely used operation, but I, at least, want to do this a lot. Second, combining it with other bindings can be done only by nesting; where cl:let and cl:let* let you bind multiple variables to the values of their respective init-forms, multiple-value-bind receives values from only a single form.

Misc-Extensions provides three related macros to elegantly and succinctly bind lists of one or more variables to the one or more values of their respective init-forms. All of these allow more than one variable in a binding clause; the last element of the clause is the init-form, and the preceding elements are the variables to be bound to the values of the init-form.

The two that I expect most people will want to use are mvlet and mvlet*. mvlet, like cl:let, makes all bindings in parallel, so the init-forms can refer to variables of the same names in the containing scope. mvlet*, like c:let*, makes them sequentially, so each init-form can refer to any of the variables bound in the preceding clauses. Except for being able to bind multiple values, they are almost perfectly upward-compatible with the CL operators, with a minor caveat I will return to below.

nlet generalizes the other two by making it possible to do any combination of parallel and sequential binding. It allows the binding clauses to be nested to indicate sequential binding: more deeply nested clauses are within the scopes of bindings made by less deeply nested clauses. Within a level, bindings are parallel. I'm personally fond of nlet, but my guess is that most people will find the syntax a little off-putting.

Here are examples of all three.

  (let ((a (foo)))
    ...
    (mvlet ((a b (bar))
            (c (baz a))  ; outer 'a'
            ...)
      ...)

    (mvlet* ((a b (bar))
             (c (baz a))  ; inner 'a'
             ...)
      ...)

    (nlet ((a b (bar))
           (c (baz a))  ; outer 'a'
           ...)
      ...)

    (nlet ((a b (bar))
           ((c (baz a)))  ; inner 'a'
           ...)
      ...))

In all four examples we see a and b being bound to the first two values of (bar) in the first clause, but whether the reference to a in the second clause refers to the a in the outer scope or the one that was just bound depends on whether mvlet or mvlet* was used, or in the nlet case, the nesting level of the second clause.

A more complex nlet example:

  (nlet ((a b c (zot))
         ((d (quux a c))
          ((e f (mumble b d))
           (g (mung a))))
         ((h (frobozz c))
          ((i (xyzzy h))))
         (*print-level* 3))
    ...)

First a, b, and c are bound to the first three values of (zot), and in parallel, *print-level* is bound to 3; then d and h are bound; then e, f, g, and i are bound.

As this example illustrates, all bindings at a given nesting level are done in parallel, with all bindings at a deeper level following. Stylistically, it is expected that init-forms in nested clauses will refer only to variables bound in containing clauses. However, this is not enforced; for instance, the init-form for g could refer to h, since the latter is bound one level out.

The macros correctly handle declare forms at the beginning of the body, emitting the declarations at the correct level within the expansion so that they apply to the binding named. — Well, almost. In order to do that, they have to be able to detect type declarations, which can start with a type name rather than the symbol type; and as it turns, out, CL has no portable interface for determining whether a symbol is a type name. (You can find out whether it's a class using find-class, but there's no portable way to tell whether a symbol has been deftyped.) There is a hackish way to do it that works on SBCL, Clozure (CCL), ECL, CLASP, Franz Allegro, and LispWorks, but not, as of this writing, on ABCL. On ABCL, the macros use a heuristic that could fail, but which I doubt will ever fail in practice. But, if you want to avoid any risk of error, start your type declarations with the symbol type if the type is user-defined.

The symbol nlet is exported from package new-let. It also exports the same macro under the name let, so that it can shadow cl:let; this is how I use it, though again, I suspect most people will not want to do that. Anyway, because new-let:let is exported, if you (:use new-let) in your package declaration, you will also need to include a :shadowing-import-from clause to say which version of let you want to use.

mvlet and mvlet* are initially exported from new-let as well, but to make them a little more convenient to use (especially for those not yet expert in dealing with CL packages), I created another package mvlet that re-exports only these two symbols. So you can say (:use mvlet) to get them, without having to shadowing-import anything.

Historical note: I first wrote nlet in 1980, on the MIT Lisp Machine, and have used it ever since in my own projects.

GMap is an iteration macro for Common Lisp. It was conceived as a generalization of mapcar (hence the name). It is intended for cases when mapcar doesn't suffice because the things you want to map over aren't in lists, or you need to collect the results of the mapping into something other than a list.

That is, gmap is probably the right thing to use when you are using iteration to perform the same computation on each element of some collection, as opposed to changing your state in some complicated way on every iteration of a loop. It's conceptually reasonable to imagine all the iterations of a gmap as happening in parallel, just as you might with mapcar. It also supports arbitrary reductions of the results of the mapped function; more on this below.

GMap is explicitly intended only for iterations that fit this "map-reduce" model. It is not trying to be a "universal" iteration construct. People have asked me what guarantees it offers concerning the ordering of the various operations it performs, and the answer is none, other than those obviously imposed by the data flow (a result can't be used until it is computed). I think that the question was rooted in experience of people using loop or other iteration constructs and supplying variable update expressions with side effects, so that there was "crosswise" data flow between the iterations. I strongly advise that such side effects be avoided in gmap calls. If you find yourself wanting to use them, either there is a better way (more on this below), or else gmap simply isn't the right tool for the job. In short, you should think of gmap very much as a functional iteration construct.

In general, my philosophy about iteration in Lisp is that there are many ways to do it, for the very good reason that there are many kinds of iterations, and one should use the tool appropriate for the particular case one is confronted with. So, for example, I almost never write a gmap form with side effects. If I'm just iterating over a list for effect, I'll use dolist. For iterations with cross-iteration data flow ("loop-carried dependences" is the compiler developer's term) or where the iteration space is not well defined in advance (e.g., iterating over lines being read from a file) I might use good old do, or I might even write the code tail-recursively, counting on the fact that most CL implementations these days do tail-call optimization at least between functions defined in a single labels form.

So when I do use gmap, it's specifically intended to convey to someone reading the code that the function being mapped is side-effect-free, so that the calls to it are independent of one another. I strongly urge adherence to this rule.

Even with that constraint, I find that occasions to use gmap are not at all uncommon. It has proven handy over the years.

The difference between mapcar and gmap is that with gmap, you explicitly indicate what kinds of collections the elements come from and how to combine the successive values of the mapped function into a result. For example, the following two expressions are equivalent:

(mapcar #'foo this-list that-list)

and

(gmap (:result list) #'foo (:arg list this-list) (:arg list that-list))

[Side note to existing GMap users: this is the new, GMap 4.0 syntax. The older syntax is considered mildly deprecated, but will continue to be supported indefinitely. More on this below.]

The (:result list) subform indicates that gmap is to build a list; the :arg subforms tell it that this-list and that-list are in fact lists of elements over which foo is to be mapped. Other types are known besides list; for example, string, if used as an argument type, causes its argument to be viewed as a string; the values it supplies to the function being mapped are the successive characters of the string.

Like mapcar, gmap accepts any number of argument specs; each one supplies one argument (or more, in some cases) to each call to the function being mapped. Also like mapcar, gmap terminates its iteration when any of the arguments runs out of elements.

For a small collection of examples, look at test-new-syntax in tests.lisp.

Historical note: GMap is now one of several extensible iteration macros, but it's actually among the oldest; along with nlet, I first wrote it in 1980, in Lisp Machine Lisp.

The mapped function is called with one or two arguments from each argument generator (the ones that generate two arguments are noted below). It may return multiple values; some result collectors make use of the additional values (again, see below).

A literal :id may be supplied as an abbreviation for the identity function; it is equivalent to #'values. (nil may be written instead of :id, but this usage is a bit unclear and is deprecated.)

The set of argument types is extensible. Thus you can adapt gmap to other kinds of data structures over which you would like to iterate. For details of how to do this, see def-arg-type in the source file, and study the existing definitions.

An argument type can explicitly indicate that it supplies more than one argument to the function being mapped. That function must have one or more additional parameters at the corresponding point in its parameter list. For example:

(gmap (:result list) #'(lambda (x y z) (cons x (+ y z)))
      (:arg alist '((a . 47) (b . 72)))
      (:arg list '(2 6)))
==>
((A . 49) (B . 78))

Here x and y receive the pairs of the alist, and z receives the elements of the list.

• constant value: Yields value on every iteration.

• list list: Yields successive elements of list.

• improper-list list: Yields the successive elements of list, which may be improper; any non-cons tail terminates the iteration.

• alist alist: Yields, as two values, the successive pairs of alist.

• plist plist: Yields, as two values, the successive pairs of elements of `plist'; that is, there is one iteration for each two elements.

• hash-table table: Yields, as two values, the successive pairs of table. (Warning: the ordering of pairs is Lisp-implementation-dependent and should not be relied on.)

• tails list: Yields the successive tails (cdrs) of list, starting with list itself, which may be improper.

• index start &optional stop &key incr fixnums?: Yields integers in the interval [start, stop) if incr (which defaults to 1) is positive; or in the interval [stop, start) if incr is negative. Specifically, in the upward case, the values begin with start and increase by incr until >= stop; in the downward case, the values begin with start - incr and decrease by incr until < stop. All values are assumed to be fixnums unless fixnums? is a literal nil. stop can be omitted or a literal nil to indicate an unbounded sequence. start can be omitted to start at 0.

• index-inc start stop &key incr fixnums?: ("Index, INClusive") Yields integers in the interval [start, stop]. Specifically, in the upward case (incr > 0), the values begin with start and increase by incr until > stop; in the downward case, the values begin with start and decrease by incr until < stop. All values are assumed to be fixnums unless fixnums? is a literal nil. stop can be a literal nil to indicate an unbounded sequence.

• exp initial-value base: Yields an exponential sequence whose first value is initial-value, and whose value is multiplied by base on each iteration.

• vector vec &key start stop incr: Yields elements of vector vec. start and stop may be supplied to select a subsequence of vec; incr may be supplied (it must be positive) to select every second element etc. For performance, you may prefer simple-vector.

• simple-vector vec &key start stop incr: Yields elements of vector vec, which is assumed to be simple, and whose size is assumed to be a fixnum. start and stop may be supplied to select a subsequence of vec; incr may be supplied (it must be positive) to select every second element etc.

• string str &key start stop incr: Yields elements of string str. start and stop may be supplied to select a subsequence of vec; incr may be supplied (it must be positive) to select every second element etc. For performance, you may prefer simple-string.

• simple-string str &key start stop incr: Yields elements of string str, which is assumed to be simple, and whose size is assumed to be a fixnum. start and stop may be supplied to select a subsequence of str; incr may be supplied (it must be positive) to select every second element etc.

• array ary &key start stop incr: Yields elements of array ary, which may be multidimensional and/or specialized. Access is via row-major-aref, so the elements are yielded in row-major order. start and stop, which are row-major indices, may be supplied to select a subsequence, and incr to skip one or more elements at each step.

GMap, unlike mapcar, has the ability to perform arbitrary reductions on the results returned by the function being mapped. So, cases where you might have written

(reduce #'+ (mapcar ...))

can be replaced with a single gmap call, which is also more efficient in that it doesn't materialize the intermediate result list:

(gmap (:result sum) ...)

GMap takes the view that consing up a collection of function results is a kind of reduction — a slightly unusual view, perhaps, but not unreasonable. So it treats collecting the results and summing them, for example, as instances of the same pattern.

As with the argument types, the set of result types is extensible. For details of how to do this, see def-result-type in the source file, and study the existing definitions.

A result type can explicitly indicate that it expects the function being mapped to return multiple values, which it can turn into multiple arguments to a reduction function. Also, there is the special result type values, which takes two or more result specs, and expects the function being mapped to return the same number of values; it reduces each value according to the corresponding result spec, then finally returns all the reduction results as multiple values. For example:

(gmap (:result values list sum) #'(lambda (x y) (values x y))
      (:arg alist '((a . 7) (b . 19))))
==>
(A B)   ; first value
26      ; second value

Additionally, there is a :consume feature that allows a single reduction to consume multiple values from the function being mapped; see the source for details. — Earlier, I promised a "better way" (search for that phrase) to handle cases where you need cross-iteration data flow. The use of multiple values and :consume can solve many of these problems without dirtying your code with side effects.

• list &key filterp: Returns a list of the values, optionally filtered by filterp (which can be :id to filter out nil).

• alist &key filterp: Consumes two values from the mapped function; returns an alist of the pairs. Note that filterp, if supplied, must take two arguments.

• plist &key filterp: Consumes two values from the mapped function; returns a plist of the pairs. Note that filterp, if supplied, must take two arguments.

• hash-table &key test size rehash-size rehash-threshold filterp: Consumes two values from the mapped function; returns a hash-table of the pairs. If any of test, size, rehash-size, or rehash-threshold are supplied, they are passed to make-hash-table. Note that filterp, if supplied, must take two arguments.

• append &key filterp: Returns the result of appending the values, optionally filtered by filterp (which can be :id to filter out nil).

• nconc &key filterp: Returns the result of nconcing the values, optionally filtered by filterp (which can be :id to filter out nil).

• and: If one of the values is false, terminates the iteration and returns false; otherwise returns the last value. Does not work as an operand of values. (Generalizes cl:every.)

• or: If one of the values is true, terminates the iteration and returns it; otherwise, returns false. Does not work as an operand of :values. (Generalizes cl:some.)

• sum &key filterp: Returns the sum of the values, optionally filtered by filterp (which can be :id to filter out nil).

• product &key filterp: Returns the product of the values, optionally filtered by filterp (which can be :id to filter out nil).

• count: Returns the number of true values.

• max &key filterp key: Optionally filters the values by filterp, then returns the maximum, or if key is supplied, the value with the maximum key (if that's not unique, returns the first one); or nil if no values were supplied (or survived filtering). Example:

  (gmap (:result max :key #'cdr) nil (:arg list alist))
  

  returns the (first) pair of alist with the maximum cdr.

  If key is :second-value, the second value of the mapped function is used; for example,

  (gmap (:result max :key :second-value) nil (:arg alist an-alist))
  

  returns the (first) car of an-alist with the maximum corresponding cdr.

• min &key filterp key: Optionally filters the values by filterp, then returns the minimum, or if key is supplied, the value with the minimum key (if that's not unique, returns the first one); or nil if no values were supplied (or survived filtering). Example:

  (gmap (:result min :key #'cdr) nil (:arg list alist))
  

  returns the (first) pair of alist with the minimum cdr.

  If key is :second-value, the second value of the mapped function is used; for example,

  (gmap (:result min :key :second-value) nil (:arg alist an-alist))
  

  returns the (first) car of an-alist with the minimum corresponding cdr.

• vector &key use-vector length fill-pointer adjustable filterp: Constructs a vector containing the results. If use-vector is supplied, the argument will be filled with the results and returned; if fill-pointer is true and adjustable is true, it must have a fill pointer and be adjustable, and values will be appended to it with vector-push-extend; if fill-pointer is true and adjustable is false, it must have a fill pointer, and values will be appended to it with vector-push; otherwise, the vector is assumed to be simple and must be large enough to hold the results. (Recall that vector-push has no effect if the vector is full.)

  If use-vector is not supplied, a vector will be constructed and returned; if length is supplied, returns a simple vector of the specified length (which must be sufficient to hold the results); otherwise, returns a simple vector of the correct length (but to do this, it must cons a temporary list).

  In any case, if filterp is supplied, it is a predicate of one argument, the value of the function being mapped, that says whether to include it in the result (filterp can be :id to filter out nil).

• string &key use-string length fill-pointer adjustable filterp: Constructs a string containing the results. If use-string is supplied, the argument will be filled with the results and returned; if fill-pointer is true and adjustable is true, it must have a fill pointer and be adjustable, and values will be appended to it with vector-push-extend; if fill-pointer is true and adjustable is false, it must have a fill pointer, and values will be appended to it with vector-push; otherwise, the vector is assumed to be simple and must be large enough to hold the results. (Recall that vector-push has no effect if the vector is full.)

  If use-string is not supplied, a string will be constructed and returned; if length is supplied, returns a simple string of the specified length (which must be sufficient to hold the results); otherwise, returns a simple string of the correct length (but to do this, it must cons a temporary list).

  In any case, if filterp is supplied, it is a predicate of one argument, the value of the function being mapped, that says whether to include it in the result (filterp can be :id to filter out nil).

• array dims &key element-type initial-element filterp: Constructs an array containing the results. Passes dims, and element-type and initial-element if supplied, to make-array. If the array is multidimensional, fills it in row-major order.

For most of GMap's existence, it has had a slightly different syntax from that shown above. The :arg and :result keywords were not used; instead, the argument and result types were defined as keywords themselves. For instance, the first example above would look like this:

(gmap (:list) #'foo (:list this-list) (:list that-list))
;; The parens around the result are optional; you can also write:
(gmap :list #'foo (:list this-list) (:list that-list))

The reason I made them keywords was so that uses of them, as in this example, wouldn't look like function calls; at least, the presence of the colons would presumably give the reader of the code, who might not be familiar with GMap, a clue that something out of the ordinary was going on. The problem with this, of course, is that it abandoned the modularity which is the entire point of the package system: there can be only one definition of a given keyword as an argument type or as a result type.

The 4.0 release fixes this by introducing :arg and :result to visually mark these syntactic elements, and by changing all the predefined types to use non-keyword symbols exported either from cl: or from gmap:. However, it's important not to break existing code; so here's what I have done. def-gmap-arg-type and def-gmap-res-type, if given a name that is not a keyword symbol, now also define the keyword symbol with the same name; but before they do that, they check that it is not already defined by a previous call supplying a different non-keyword name; if it is, they signal an error.

With this change, the old syntax will continue to work, but collisions, where two systems try to define argument or result types with the same symbol-name, will be detected; previously, the one loaded last would "win".

(Personally, I've settled on a compromise, wherein I write result types and and or in the old syntax, as :and and :or; it's not plausible that these could be defined as types. For other result types, and all argument types, I use the new syntax.)

To return the position of the largest value in a list numbers:

(gmap (:result max :key :second-value) nil (:arg index 0) (:arg list numbers))

For very small lambda expressions, the six characters taken by the word lambda can be a significant fraction of the total. If the body doesn't reference all the parameters, moreover, you'll want to add a (declare (ignore ...)) for the unused ones, which adds to the verbosity. The fn macro helps with both annoyances. Obviously, its name is quite short. (Those willing to set foot on the slippery slope of Unicode could use λ, of course, but I've been afraid to go there yet, lest my code wind up as an unintelligible mass of obscure mathematical symbols and cat emojis.) And, you can use either a bare _ or a name beginning with _ as a parameter name, to indicate an ignored parameter; fn automatically inserts the required ignore declaration.

One catch, though, is that if you inadvertently write #'(fn ...), you will get a probably-unintelligible compiler error. Just delete the #'. (Lisp Machine Lisp had something called "lambda macros" that solved this problem, but the feature didn't make it into Common Lisp.)

I still consider these experimental and probably not terribly useful, though I do use one occasionally. If curiosity gets the better of you, have a look at contexts.text. Briefly, it's an alternate syntax for combinations of let and labels.

Macro deflex var &optional val doc:

Declares var as a global lexical variable, and if val is supplied and var is not already bound, initializes it to val. doc, if supplied, is taken as a documentation string. In some implementations (e.g. Scieneer), locally rebinding the same name is not permitted; in most, it is permitted but creates a new lexical variable, with no effect on the global one.

Macro deflex-reinit is the same except that it reinitializes the variable if it's already bound, like cl:defparameter.

Macro isetq &rest var-val-pairs:

Some implementations, notably SBCL, issue a warning if you use setq to set a new global variable in the REPL. (I guess they want you to use defvar first.) isetq prevents this warning, using the same trick deflex uses to declare a global lexical. isetq is not recommended for use in programs.

It's not uncommon to use labels to define a set of several mutually-recursive functions, whose code can fill a screen or two, then have the body of the labels kick off the recursion with a one-liner that just calls one of the functions. This strikes me as a little hard to read — you have to scroll all the way to the bottom to find the initial values of the recursion variables — and also a little wasteful of indentation space. So I introduced a macro rlabels, which puts the initial call first as a single subform, then takes the function-binding clauses as its &body parameter, saving 7 columns of indentation.

There are also rflet and rmacrolet, but I don't think I've ever used them.

As everyone knows, defclass forms tend to be rather verbose and repetitive:

(defclass frob (widget)
    ((color :initarg :color :reader frob-color
       :initform (error "A color must be specified.")
       :documentation "The color of the frob.")
     (height :initarg :height :accessor frob-height
       :initform 3.0
       :documentation "The height of the frob in cm.")
     ...)
  (:documentation "The class of all frobs.")

I've written a define-class macro to shorten them. It is completely upward compatible with defclass; you could just replace defclass with define-class and have a valid invocation that would produce the same result. But define-class provides several features to make the definition more succinct. Use only the ones you like! The big change is that where defclass slot descriptions have a strict alternating keyword-value syntax, define-class is more flexible.

• A doc string for the class can be placed just before the slot specs (visually similar to where doc strings for functions go).
• The first element of a slot specifier, which is normally a slot name, can instead be a list (slot-name initform); an :initform option will be generated.
• :may-init generates :initarg :slot-name.
• :must-init generates :initarg :slot-name, and also an :initform that signals an error if the argument is not provided to make-instance.
• :readable generates :reader gf-name, and :accessible generates :accessor gf-name, where gf-name is either the slot name or, if a :conc-name class option is supplied, that string prepended to the slot name.
• Additionally, :constant is an abbreviation for :must-init :readable if the spec contains no initform, or :may-init :readable if there is an initform (in either syntax)
• And :variable is an abbreviation for :may-init :accessible
• A doc string can appear anywhere in the slot options; :documentation will be inserted.
• Or, you can use :doc as an abbreviation for :documentation.

So, here's the above example using define-class:

(define-class frob (widget)
  "The class of all frobs."
  ((color :constant "The color of the frob.")
   ((height 3.0) :variable "The height of the frob in cm."))
  (:conc-name #:frob-))

Let me emphasize, you can still use the defclass slot options in any case where the above features do not do what you want; for instance, if you want a different reader/accessor name for a particular slot than what define-class would have used.

BTW, for GNU Emacs users, if you look at the bottom of src/define-class.lisp, you will see an Emacs patch that will improve the fontification of define-class slot doc strings, in the case where they are not preceded by :documentation / :doc.