Macros in Lisp are much more capable than macros in other programming languages. Rather than just providing a simple shorthand notation, Lisp macros give you the capability to truly extend the language. In this chapter we'll learn about the program transforming capabilities of macros as we see how to properly construct macros to solve a wide variety of problems.
Mention macros to most programmers, perhaps even you, and the first image that comes to mind is string substitution -- you use a macro to glue together a few parameters in a new way, maybe with a bit of compile-time decision making thrown in. And because of the typical (in languages other than Lisp) disparity between the macro language and the programming language, the difficulty of writing a macro increases much faster than its complexity.
Lisp macros are Lisp programs that generate other Lisp programs. The generated Lisp code has a fully-parenthesized notation, as does the macro that generates the code. In the simplest case, a macro substitutes forms within a template, clearly establishing a visual correspondence between the generating code and the generated code. Complex macros can use the full power of the Lisp language to generate code according to the macro parameters; often a template form is wrapped in code that constructs appropriate subforms, but even this approach is just a typical use pattern and not a requirement (or restriction) of the Lisp macro facility.
In the following sections, we'll examine the mechanism by which the Lisp system translates code generated by a macro, then we'll see several increasingly sophisticated examples of macros. We'll finish this chapter with a comparison of macros versus the use of inline function declarations.
You define a macro with a DEFMACRO form, like this:
(defmacro name (arguments ...) body)
DEFMACRO is like DEFUN, but instead of
returning values, the body of the DEFMACRO returns a
Lisp form. (As we'll see shortly, there's a very simple way to generate
this form with selected subforms replaced by parameters from the macro
call or computed by the macro's program.)
Your program "calls" a macro the same way it calls a function,
but the behavior is quite different. First, none of the macro's
parameters are evaluated. Ever. Macro parameters are bound literally
to the corresponding arguments in the macro definition. If you pass
(* 7 (+ 3 2)) to a macro, the argument in the body of
the macro definition is bound to the literal list (* 7 (+ 3
2)), and not the value 35.
Next, the macro expander is invoked, receiving all of
the actual parameters bound to their corresponding arguments as
named by the DEFMACRO form. The macro expander is just
the body of the DEFMACRO form, which is just Lisp code;
the only catch is that the Lisp system expects the macro expander to
return a Lisp form.
The Lisp system then evaluates whatever form the macro expander returns. If the returned form is a macro, it gets expanded. Otherwise, the form is evaluated by the rules we first learned in Chapter 3, Lesson 2.
The preceding paragraph is conceptually correct. However, a Lisp implementation may expand macros at different times. A macro could be expanded just once, when your program is compiled. Or it could be expanded on first use as your program runs, and the expansion could be cached for subsequent reuse. Or the macro could be expanded every time it's used. A properly written macro will behave the same under all of these implementations.
In Chapter 21 we'll expand upon some of the things you can express with argument lists.
The simplest way to generate a form in the body of your macro
expander is to use the backquote (`) reader
macro. This behaves like the quote (') reader
macro, except for when a comma (,) appears in the
backquoted form.
A comma is only permitted in a backquoted form. If you use a comma in a quoted form, Lisp will signal an error when it reads the form.
Like quote, backquote suppresses evaluation. But a comma within a backquoted form "unsuppresses" evaluation for just the following subform.
? `(The sum of 17 and 83 is ,(+ 17 83)) (THE SUM OF 17 AND 83 IS 100)
Compare the preceding example, which used backquote, with the similar form using quote (and omitting the comma).
? '(The sum of 17 and 83 is (+ 17 83)) (THE SUM OF 17 AND 83 IS (+ 17 83))
You can probably imagine how backquote and comma provide a template with substitution capabilities. This is just what we need for our macro expander. Here are a couple of simple examples.
; Define the macro
? (defmacro swap (a b) ; NOTE: This is a restricted version of ROTATEF
`(let ((temp ,a))
(setf ,a ,b)
(setf ,b temp)))
SWAP
; First invocation
? (let ((x 3)
(y 7))
(swap x y) ; macro call
(list x y))
(7 3)
; Let's see the form generated by SWAP:
? (pprint (macroexpand-1 '(swap x y)))
(LET ((TEMP X))
(SETF X Y)
(SETF Y TEMP))
; Second invocation
? (let ((c (cons 2 9))) ; (2 . 9)
(swap (car c) (cdr c))
c)
(9 . 2)
; And the expansion of its macro call
? (pprint (macroexpand-1 '(swap (car c) (cdr c))))
(LET ((TEMP (CAR C)))
(SETF (CAR C) (CDR C))
(SETF (CDR C) TEMP))
; Here's the second invocation again, "macroexpanded" by hand.
? (let ((c (cons 2 9)))
(LET ((TEMP (CAR C)))
(SETF (CAR C) (CDR C))
(SETF (CDR C) TEMP))
c)
(9 . 2)
(PPRINT (MACROEXPAND-1 'macro-call)) is a very
handy tool to see what form your macro expander generates. (Don't worry
if the output from your Lisp system looks exactly as shown here; there
may be some differences in layout.)
As you look at these examples, the important things to note are that:
Here are some more macro definitions. Experiment with these in your Lisp system to see what they do.
(defmacro sortf (place) `(setf ,place (sort ,place))) (defmacro togglef (place) `(setf ,place (not ,place))) (defmacro either (form1 form2) ; (random 2) returns 0 or 1 `(if (zerop (random 2)) ,form1 ,form2))
Macros start to get interesting when they do more than a simple textual substitution. In this section, we'll explore a real-world example of using a macro to extend Lisp into the problem domain. In addition to providing a macro expander, our new macro will automatically generate an environment that will be referenced by the expander. Our example will show how to move computations from run-time to compile-time, and how to share information computed at compile-time.
Let's say you're working on an interactive game that makes heavy
use of the trigonometric function sine r in computing player
motion and interaction. You've already determined that calling the
Lisp function SIN is too time-consuming; you also know
that your program will work just fine with approximate results for
the computation of sine r. You'd like to define a
LOOKUP-SIN macro to do the table lookup at runtime;
you'd also like to hide the details of table generation, an
implementation detail with which you'd rather not clutter your
program's source code.
Your macro will be invoked as (LOOKUP-SIN
radians divisions), where
radians is always in the range of zero to one-quarter pi,
and divisions is the number of discrete values available
as the result of LOOKUP-SIN. At runtime, the macro
expander will just compute the index into a lookup table, and return
the value from the table. The table will be generated at
compile-time (on most Lisp systems). Furthermore, only one table
will ever be generated for a given value of divisions in
the macro call.
Here's the code. The comments and documentation strings should help you to understand the code as you read it. I'll provide further explanation below.
;; This is where we cache all of the sine tables generated
;; during compilation. The tables stay around at runtime
;; so they can be used for lookups.
(defvar *sin-tables* (make-hash-table)
"A hash table of tables of sine values. The hash is keyed
by the number of entries in each sine table.")
;; This is a helper function for the LOOKUP-SIN macro.
;; It is used only at compile time.
(defun get-sin-table-and-increment (divisions)
"Returns a sine lookup table and the number of radians quantized
by each entry in the table. Tables of a given size are reused.
A table covers angles from zero to pi/4 radians."
(let ((table (gethash divisions *sin-tables* :none))
(increment (/ pi 2 divisions)))
(when (eq table :none)
;; Uncomment the next line to see when a table gets created.
;;(print '|Making new table|)
(setq table
(setf (gethash divisions *sin-tables*)
(make-array (1+ divisions) :initial-element 1.0)))
(dotimes (i divisions)
(setf (aref table i)
(sin (* increment i)))))
(values table increment)))
;; Macro calls the helper at compile time, and returns an
;; AREF form to do the lookup at runtime.
(defmacro lookup-sin (radians divisions)
"Return a sine value via table lookup."
(multiple-value-bind (table increment)
(get-sin-table-and-increment divisions)
`(aref ,table (round ,radians ,increment))))
If you still don't see the point of all this code after having
read the introduction to this section and the comments in the code,
here it is: when your program runs, it executes just
AREF (and associated ROUND) to look up the
sin r value.
? (pprint (macroexpand-1 '(lookup-sin (/ pi 4) 50)))
(AREF #(0.0 0.0314107590781283 0.06279051952931337
[additional entries not shown]
0.9980267284282716 0.9995065603657316 1.0)
(ROUND (/ PI 4) 0.031415926535897934))
;; Note that the macro call makes no mention of a lookup table.
;; Tables are generated as-needed by (and for) the compiler.
? (lookup-sin (/ pi 4) 50)
0.7071067811865476
In the macroexpansion, the #(...) is the printed
representation of the lookup table for 50 divisions of the quarter
circle. This table is stored in the *SIN-TABLES* hash
table, where it is shared by every macro call to
(LOOKUP-SIN angle 50). We don't
even have to do a hash lookup at runtime, because the macro expander
has captured the free variable TABLE from the
MULTIPLE-VALUE-BIND form in LOOKUP-SIN.
Macros that define macros are used infrequently, partly because it's hard to think of a good use for this technique and partly because it's difficult to get right. The following macro, based upon an example in Paul Graham's "On Lisp" book, can be used to define synonyms for the names of Lisp functions, macros, and special forms.
? (defmacro defsynonym (old-name new-name)
"Define OLD-NAME to be equivalent to NEW-NAME when used in
the first position of a Lisp form."
`(defmacro ,new-name (&rest args)
`(,',old-name ,@args)))
DEFSYNONYM
? (defsynonym make-pair cons)
MAKE-PAIR
? (make-pair 'a 'b)
(A . B)
Macros are always a little bit dangerous because code containing a macro call does not automatically get updated if you change the definition of the macro. You can always establish your own convention to help you remember that you need to recompile certain code after you change a macro definition. But there's always the possibility that you'll forget, or make a mistake.
Ultimately, the likelihood that you'll inadvertently end up with
code that was compiled with an old version of a macro is directly
proportional to how often you're likely to change the macro. I'll
probably never need to change the LOOKUP-SIN macro from
the previous section once it's defined and working. On the other
hand, a macro like DEFSYNONYM practically begs to be
used again and again as you generate new code. If you change your
mind about the old name to associate with a given new
name, all of your previously compiled code will still refer to
the old name that you had decided upon previously.
;; WARNING: This example illustrates a practice to avoid!
;; Here's some core algorithm
? (defun process-blah-using-algorithm-zark (...) ...)
PROCESS-BLAH-USING-ALGORITHM-ZARK
;; And here's where I use the algorithm, perhaps calling it
;; from many other places in DO-STUFF besides the one I've shown.
? (defun do-stuff (...)
...
(process-blah-using-algorithm-zark ...)
...)
DO-STUFF
;; Try it out...
? (do-stuff ...)
[results based upon process-blah-using-algorithm-zark]
;; OK, this looks good. But I think I'll clean up the
;; appearance of DO-STUFF by defining an abbreviation
;; for that really long core algorithm name.
? (defsynonym process-blah-using-algorithm-zark proc)
PROC
;; Now I'll rewrite DO-STUFF to use the abbreviation.
? (defun do-stuff (...)
...
(proc ...)
...)
DO-STUFF
;; And make sure it still works.
? (do-stuff ...)
[results based upon process-blah-using-algorithm-zark]
... Some time later ...
;; Oh, here's a better core algorithm.
? (defun process-blah-using-algorithm-glonkfarkle (...) ...)
PROCESS-BLAH-USING-ALGORITHM-GLONKFARKLE
;; I'll change the synonym for PROC to 'be' the new algorithm.
? (defsynonym process-blah-using-algorithm-glonkfarkle proc)
PROC
... Some time later ...
;; Time to use DO-STUFF again...
? (do-stuff ...)
[results based upon process-blah-using-algorithm-zark]
;; Hey!! These results don't seem to use the new algorithm.
;; What could be wrong? The code LOOKS right...
The problem, of course, is that the second use of
DEFSYNONYM redefined the PROC macro, and I
didn't notice that DO-STUFF needed to be recompiled to
pick up the changed definition.
My advice: Don't try to be clever by using macros like
DEFSYNONYM. Stick with descriptive names that are as
long as necessary, and use an editor that supports symbol completion
(see Chapter 27). Remember, there's
only one way to not abbreviate a name; using abbreviations
increases the chance that you'll use the wrong one.
You have to be be careful when you define a macro that introduces
new variables in its expansion. The REPEAT macro, below,
offers us a shorthand way of repeating a body of code a certain number
of times.
? (defmacro repeat (times &body body)
`(dotimes (x ,times)
,@body))
REPEAT
? (repeat 3 (print 'hi))
HI
HI
HI
NIL
This seems to do the right thing, but the variable X is
going to cause problems. The following example should give us
the same results as the last example.
? (setq x 'hi) HI ? x HI ? (repeat 3 (print x)) 0 1 2 NIL
The variable X in the macro expander
shadowed the global X that we tried to
reference in the body. Another way to say this is that
X is free in the body of the
REPEAT form, but it was captured by the
definition of X in the macro expander; this prevents
the body form from reaching the intended variable X.
The obvious solution is to use a different variable name in the
macro expander -- one that won't conflict with any name we'll ever
use in our code that calls the REPEAT macro. You might
think that some kind of naming convention would work, but there's
always the chance that some programmer will come along later and
violate the convention. We need a foolproof approach.
Lisp provides a GENSYM function to generate symbols
that are guaranteed to be unique. No programmer can ever
write a symbol name that conflicts with a symbol created by
GENSYM. Here is how we use GENSYM to
create a name for the variable needed in the macro expander for the
REPEAT macro.
? (defmacro repeat (times &body body)
(let ((x (gensym)))
`(dotimes (,x ,times)
,@body)))
REPEAT
? x
HI
? (repeat 3 (print x))
HI
HI
HI
NIL
With this new REPEAT macro, we compute a new symbol
in the LET form, and substitute this symbol in the
macro expander form. To see why this works, let's look at an expansion:
? (macroexpand-1 '(repeat 5 (print x)) (DOTIMES (#:G8524 5) (PRINT X))
#:G8524 is a unique uninterned symbol. You can
see that it's uninterned by the #: prefix. But how does Lisp
guarantee the uniqueness of this symbol? The Lisp reader guarantees that
any symbol it reads with the #: prefix is unique. Compare
the following:
? (eq 'a 'a) T ? (eq '#:a '#:a) NIL
Even though the #:A symbols print the same, they are
different.
Generating variable names to be used in macro expanders has another application. This next macro definition has a subtle problem:
? (defmacro cube (n)
`(* ,n ,n ,n))
CUBE
? (cube 3)
27
? (let ((n 2))
(cube (incf n)))
60
In the second case, (INCF N) should have provided the
value 3 to CUBE and the result should have been identical
to the first test. Let's take a look at the expansion again, to see
what happened.
? (macroexpand-1 '(cube (incf n))) (* (INCF N) (INCF N) (INCF N))
The problem is obvious: CUBE's argument, (INCF
N) is being evaluated multiple times. As a rule, this is a bad
thing to do, because it violates our assumptions about the way Lisp
evaluates forms. We fix this problem by arranging for the macro expander
to evaluate CUBE's argument just once.
? (defmacro cube (n)
(let ((x (gensym)))
`(let ((,x ,n))
(* ,x ,x ,x))))
CUBE
? (let ((n 2))
(cube (incf n)))
27
We created a unique symbol outside of the macro expander, then
used this symbol in the expander as the name of a variable to hold
the result of evaluating CUBE's argument. The
LET form in the macro expander is the only
place where CUBE's argument is referenced, so it gets
evaluated exactly once for each call to CUBE.
Lisp allows functions to be inlined by the compiler. In other words, rather than compiling a call to the function, the compiler may substitute the function's body, thus saving the overhead of a function call. Substituting the function's body is generally expensive in terms of space, since a function body's code is usually longer than the code of its calling sequence.
It's important to understand that Lisp allows functions to
be inlined. Like all other declarations -- save the SPECIAL
declaration -- an INLINE declaration may be treated as
advisory or ignored entirely by the compiler.
Here are some examples of how to inline a function. In the first case,
function F is inlined everywhere it is used (assuming that
the compiler supports inlining). In the second case, function P
is compiled with information to support inlining, but is only inlined
in the presence of a declaration, as in function Q.
; Case 1 -- F may always be inlined (declaim (inline f)) (defun f (...) ...) (defun g (...) (f ...) ...) (defun h (...) (f ...) ...) ; Case 2 - P may be inlined only following a declaration (declaim (inline p)) (defun p (...) ...) (declaim (notinline p)) (defun q (...) (declare (inline p)) (p ...) ; inlined ...) (defun r (...) (p ...) ; not inlined ...)
Macros may be used in place of INLINE declarations for
cases where code absolutely must be inlined despite the presence
(or absence) of compiler support for inlining.
In general, though, you should use macros for language extension, and not for efficiency hacks. The risk of forgetting to recompile after having changed a macro definition can cause hard-to-find bugs that will waste a lot of your development effort.
My advice: Don't use macros as a substitute for inlining unless you can find no other way to achieve desired performance; of course, such efforts should be guided by the results of profiling your code (see Chapter 28) and preferably only when your code is already stable and debugged. You should also reexamine your decision with each new release of your Lisp compiler, and whenever you port your program to another Lisp platform.