| After weeks of on-again, off-again development, I've finally finished a set of macro debugging tools. If you're just learning about macros, you can see how they work in real time. If you're a Lisp guru, you can track down hard-to-find expansion bugs. For a primer on this library, I'll motivate each tool by example. Take evanrmurphy's macro issues at http://arclanguage.org/item?id=11416: (mac rdefop (r)
`(defop ,r req
(pr "Schedule for " ,r " goes here")))
(each r routes*
(rdefop r))
We want to know why the macro in the loop isn't defining a new URL handler for each route. Arc has macex and macex1, but they're rudimentary. arc> (macex1 '(each r routes* (rdefop r)))
(let gs1722 routes* (if (alist gs1722) ((rfn gs1723 (gs1724) (when (acons
gs1724) (let r (car gs1724) (rdefop r)) (gs1723 (cdr gs1724)))) gs1722) (isa
gs1722 (quote table)) (maptable (fn r (rdefop r)) gs1722) (for gs1724 0 (-
(len gs1722) 1) (let r (gs1722 gs1724) (rdefop r)))))
Yikes. Turns out each is a macro. We just want to look inside at the rdefop expansion. You might (macex1 '(rdefop r)), which would be okay for this example, but (a) this process is far too manual and (b) in more complex code, you want to see how all the macros affect each other -- weird things can happen with expansions.So, first we have macex-all, which combs through an expression and expands every macro entirely until no more macros are left. As you might guess, it gets unwieldy, though is sometimes useful. arc> (macex-all '(each r routes* (rdefop r)))
((fn (gs2090) (if (alist gs2090) (((fn (gs2091) (assign gs2091 (fn (gs2092)
(if (acons gs2092) ((fn () ((fn (r) ((fn () ((fn () ((fn () (atomic-invoke
(fn nil ((fn (gs2094 gs2096 gs2097) ((fn (gs2095) (sref gs2094 gs2095
gs2096)) gs2097)) redirector* (quote r) nil))))))) ((fn () (atomic-invoke (fn
nil ((fn (gs2099 gs2101 gs2102) ((fn (gs2100) (sref gs2099 gs2100 gs2101))
gs2102)) srvops* (quote r) (fn (gs2093 req) ((fn (gs2098) ((fn (gs2103)
(save-optime (quote r) (- (msec) gs2098)) gs2103) ((fn () (call-w/stdout
gs2093 (fn nil (prn) (pr "Schedule for " r " goes here")))))))
(msec))))))))))) (car gs2092)) (gs2091 (cdr gs2092)))))))) nil) gs2090) (isa
gs2090 (quote table)) (maptable (fn r ((fn () ((fn () ((fn () (atomic-invoke
(fn nil ((fn (gs2105 gs2107 gs2108) ((fn (gs2106) (sref gs2105 gs2106
gs2107)) gs2108)) redirector* (quote r) nil))))))) ((fn () (atomic-invoke (fn
nil ((fn (gs2110 gs2112 gs2113) ((fn (gs2111) (sref gs2110 gs2111 gs2112))
gs2113)) srvops* (quote r) (fn (gs2104 req) ((fn (gs2109) ((fn (gs2114)
(save-optime (quote r) (- (msec) gs2109)) gs2114) ((fn () (call-w/stdout
gs2104 (fn nil (prn) (pr "Schedule for " r " goes here")))))))
(msec))))))))))) gs2090) ((fn (gs2092 gs2115 gs2116) ((fn () (assign gs2092
gs2115) (((fn (gs2117) (assign gs2117 (fn (gs2118) (if gs2118 ((fn () ((fn
(r) ((fn () ((fn () ((fn () (atomic-invoke (fn nil ((fn (gs2120 gs2122
gs2123) ((fn (gs2121) (sref gs2120 gs2121 gs2122)) gs2123)) redirector*
(quote r) nil))))))) ((fn () (atomic-invoke (fn nil ((fn (gs2125 gs2127
gs2128) ((fn (gs2126) (sref gs2125 gs2126 gs2127)) gs2128)) srvops* (quote r)
(fn (gs2119 req) ((fn (gs2124) ((fn (gs2129) (save-optime (quote r) (- (msec)
gs2124)) gs2129) ((fn () (call-w/stdout gs2119 (fn nil (prn) (pr "Schedule
for " r " goes here"))))))) (msec))))))))))) (gs2090 gs2092)) (assign gs2092
(+ gs2092 1)) (gs2117 (< gs2092 gs2116)))))))) nil) (< gs2092 gs2116))))) nil
0 (+ (- (len gs2090) 1) 1)))) routes*)
That hardly helps. What we really want is to cherry-pick which macros to expand and actually see them expand step-by-step so it's understandable. For that, we have macsteps.While we could do (macsteps '(each r routes* (rdefop r)))
it still expands too many things. Luckily, macsteps takes in some optional flags: 'show or 'hide, which work as you probably expect. arc> (macsteps '(each r routes* (rdefop r)) 'show 'rdefop)
Showing only: rdefop
Expression:
(each r routes* (rdefop r))
Macro Expansion:
(rdefop r)
==> (defop r req
(pr "Schedule for " r " goes here"))
Expression:
(each r routes*
(defop r req
(pr "Schedule for " r " goes here")))
nil
But this cherry-picking has to happen as you call macsteps. What if we wanted to expand defop now that we've seen rdefop expands into it? We could do (macsteps '(each r routes* (rdefop r)) 'show 'rdefop 'defop)
but we just wind up with the same problem again. We might also do (macsteps '(each r routes* (rdefop r)) 'hide 'each)
to only hide each and show everything else, but rdefop could keep expanding and expanding past the point we actually care about.Enter macstep, which is like macsteps but interactive. arc> (macstep '(each r routes* (rdefop r)))
Expression:
(each r routes* (rdefop r))
macstep> help
COMMAND ABBREV DESCRIPTION
step :s Show the next macro expansion, if any
skip :n Expand the next macro silently
back :b Go to the previous step
stop :a Abort this session
help :h Display this message
(hide ...) n/a Hide the expansions of specified macros
(show ...) n/a Stop hiding specified macros
(help x) n/a More help on command x
Expression:
(each r routes* (rdefop r))
macstep> (hide each)
Hidden: each
Expression:
(each r routes* (rdefop r))
macstep> step
*** Skipping hidden macro each
Macro Expansion:
(rdefop r)
==> (defop r req
(pr "Schedule for " r " goes here"))
Expression:
(each r routes*
(defop r req
(pr "Schedule for " r " goes here")))
macstep> step
*** Skipping hidden macro each
Macro Expansion:
(defop r req
(pr "Schedule for " r " goes here"))
==> (do (wipe (redirector* (quote r)))
(defop-raw r (gs2174 req)
(w/stdout gs2174
(prn)
(pr "Schedule for " r " goes here"))))
Expression:
(each r routes*
(do (wipe (redirector* (quote r)))
(defop-raw r (gs2174 req)
(w/stdout gs2174
(prn)
(pr "Schedule for " r " goes here")))))
At each step, macstep searches for a macro to expand, then expands it.This is pretty useful. In the last expression above we can see r getting quoted, which starts tipping us off. So maybe we want to expand defop-raw now. macstep> :s
*** Skipping hidden macro each
Macro Expansion:
(do (wipe (redirector* (quote r)))
(defop-raw r (gs2179 req)
(w/stdout gs2179
(prn)
(pr "Schedule for " r " goes here"))))
==> ((fn () (wipe (redirector* (quote r))) (defop-raw r (gs2179 req) (w/stdout gs2179 (prn) (pr "Schedule for " r " goes here")))))
Expression:
(each r routes*
((fn () (wipe (redirector* (quote r))) (defop-raw r (gs2179 req) (w/stdout gs2179 (prn) (pr "Schedule for " r " goes here"))))))
Oops! Turns out do is a macro, and not a very interesting one. We can back out of this and hide the macros we don't want to expand. macstep> back
Expression:
(each r routes*
(do (wipe (redirector* (quote r)))
(defop-raw r (gs2179 req)
(w/stdout gs2179
(prn)
(pr "Schedule for " r " goes here")))))
macstep> (hide do wipe)
Hidden: do, each, wipe
Expression:
(each r routes*
(do (wipe (redirector* (quote r)))
(defop-raw r (gs2179 req)
(w/stdout gs2179
(prn)
(pr "Schedule for " r " goes here")))))
macstep> :s
*** Skipping hidden macro each
*** Skipping hidden macro do
*** Skipping hidden macro wipe
Macro Expansion:
(defop-raw r (gs2179 req)
(w/stdout gs2179
(prn)
(pr "Schedule for " r " goes here")))
==> (= (srvops* (quote r))
(fn (gs2179 req)
(let gs2184 (msec)
(do1 (do (w/stdout gs2179
(prn)
(pr "Schedule for " r " goes here")))
(save-optime 'r
(- (msec) gs2184))))))
Expression:
(each r routes*
(do (wipe (redirector* (quote r)))
(= (srvops* (quote r))
(fn (gs2179 req)
(let gs2184 (msec)
(do1 (do (w/stdout gs2179
(prn)
(pr "Schedule for " r " goes here")))
(save-optime 'r
(- (msec) gs2184))))))))
But this is starting to feel too manual again: we have to do tree traversals in our heads to predict which macro is going to be next.So finally, macwalk is another interactive macro expander that takes us through the actual traversal of the code, expanding macros as it finds them. This is closest to how Arc sees code compiling, focusing on one subexpression at a time. arc> (macwalk '(each r routes* (rdefop r)))
Expression --> (each r routes* (rdefop r))
macwalk> help
COMMAND ABBREV DESCRIPTION
step :s Step into the next subexpression, expanding macros
skip :i Step into the next subexpression, ignoring macros
over :o Step over this level, ignoring subexpressions
stop :a Abort this session
help :h Display this message
(hide ...) n/a Hide the expansions of specified macros
(show ...) n/a Stop hiding specified macros
(help x) n/a More help on command x
Expression --> (each r routes* (rdefop r))
macwalk> skip
*** Skipping macro each
Value ==> each
Value ==> r
Value ==> routes*
Subexpression --> (rdefop r)
macwalk> step
Macro Expansion ==>
(defop r req
(pr "Schedule for " r " goes here"))
macwalk> :s
Macro Expansion ==>
(do (wipe (redirector* (quote r)))
(defop-raw r (gs2190 req)
(w/stdout gs2190
(prn)
(pr "Schedule for " r " goes here"))))
macwalk> :i
*** Skipping macro do
Value ==> do
Subexpression --> (wipe (redirector* (quote r)))
macwalk> over
Value ==> (wipe (redirector* (quote r)))
Subexpression -->
(defop-raw r (gs2190 req)
(w/stdout gs2190
(prn)
(pr "Schedule for " r " goes here")))
macwalk> :s
Macro Expansion ==>
(= (srvops* (quote r))
(fn (gs2190 req)
(let gs2194 (msec)
(do1 (do (w/stdout gs2190
(prn)
(pr "Schedule for " r " goes here")))
(save-optime 'r
(- (msec) gs2194))))))
macwalk> stop
Value ==>
(= (srvops* (quote r))
(fn (gs2190 req)
(let gs2194 (msec)
(do1 (do (w/stdout gs2190
(prn)
(pr "Schedule for " r " goes here")))
(save-optime 'r
(- (msec) gs2194))))))
Value ==>
(do (wipe (redirector* (quote r)))
(= (srvops* (quote r))
(fn (gs2190 req)
(let gs2194 (msec)
(do1 (do (w/stdout gs2190
(prn)
(pr "Schedule for " r " goes here")))
(save-optime 'r
(- (msec) gs2194)))))))
Value ==>
(each r routes*
(do (wipe (redirector* (quote r)))
(= (srvops* (quote r))
(fn (gs2190 req)
(let gs2194 (msec)
(do1 (do (w/stdout gs2190
(prn)
(pr "Schedule for " r " goes here")))
(save-optime 'r
(- (msec) gs2194))))))))
Now we can see why the original loop wasn't working: the r in (rdefop r) winds up getting quoted in select positions.Using these tools we can see, on multiple levels, how macros work. Hopefully this will make macros clearer for people, be it for the learner taking their first steps or the expert slamming their head against a tough bug. Speaking of bugs, let me know if there are any! URL: http://bitbucket.org/fallintothis/macdebug | 
