Mimeographs from the Future

As I get back into writ­ing Lisp code — via Clo­jure — I am find­ing a few things that I’m trip­ping up over on a reg­u­lar basis. Some of this is pre-conceived notions of behav­ior, and some is just men­tal lazi­ness. The one I keep com­ing back to, over and over, is the sub­tleties of lex­i­cal scop­ing ver­sus dynamic scop­ing. Inspired by this post, I thought I’d take a stab at explain­ing it in a slightly dif­fer­ent way. You can never have too many expla­na­tions. This is my under­stand­ing as it exists today. It is likely some­what wrong. It might even be com­pletely and totally wrong.

The first thing to do is replace the word “lex­i­cal” with the word “sta­tic”. Now we’re using orthog­o­nal vocab­u­lary. It also makes some things a bit more obvi­ous. Before we dig into scop­ing (sta­tic v. dynamic), we first need to be care­ful to define what we’re talk­ing about: bind­ings. The idea of a “vari­able” doesn’t really exist in the tra­di­tional sense. You have sym­bols, val­ues and bind­ings. Take the fol­low­ing code snip­pet for example:

user=> (def foo)
#'user/foo

Here, we have def, which is a spe­cial form that cre­ates a Var, and foo, which is a sym­bol that is used to iden­tify the Var in the future. At this point, foo has no “value”, and so we can’t refer to it:

user=> foo
java.lang.IllegalStateException: Var user/foo is unbound.

So why in the world would you want to do this? Just you wait and see. More typ­i­cally, you also cre­ate the ini­tial bind­ing to a value:

user=> (def foo 100)
#'user/foo

Once again, we have the sym­bol foo, but it’s now bound to the value 100. The bind­ing is that “invis­i­ble” link between the Var and the value. To quote the Clo­jure “man­ual”:

Every Var can (but needn’t) have a root bind­ing, which is a bind­ing that is shared by all threads that do not have a per-thread bind­ing. Thus, the value of a Var is the value of its per-thread bind­ing, or, if it is not bound in the thread request­ing the value, the value of the root bind­ing, if any.

That seems easy enough, right? Right. So, if you saw the fol­low­ing code snip­pet, what would you think we would get in response?

user=> (println "The value of foo is:" foo)

Easy, right? The value of foo is: 100. Sim­ple, and no strange behav­ior, right? Right. Next up, sta­tic (lex­i­cal) scop­ing. This is best typ­i­fied by the let spe­cial form.

user=> (let [foo 50] (println "The value of foo is:" foo))

Also, not too hard, I sup­pose. We get The value of foo is: 50, which is pretty pre­dictable. So what’s the fuss? Well, take a look at the next code sam­ple and see if you can fig­ure out what the out­put is.

(def foo 100)
 
(defn print-foo []
  (println "The value of foo is:" foo))
 
(let [foo 50]
  (print-foo))

Did you answer 50? That’s unfor­tu­nately wrong. The answer is 100, but why? Let’s return to Wikipedia:

With lex­i­cal scope, a name always refers to its (more or less) local lex­i­cal envi­ron­ment. This is a prop­erty of the pro­gram text and is made inde­pen­dent of the run­time call stack by the lan­guage implementation.

When Clo­jure eval­u­ated the defn form, it used the Var that was avail­able at-the-time. When you use the let spe­cial form, you “shadow” the orig­i­nal Var — cre­at­ing a hier­ar­chy of bind­ings — but you don’t change the bind­ing that print-foo was refer­ring to. Hence, 100. In the other exam­ple, we don’t refer to the Var until inside the let spe­cial form.

Another bit about let that some poe­ple might not pick up is that once assigned, the value of the local can never change. The bind­ing is fixed. For example:

user=> (let [foo 100]
         (println "The value of foo is:" foo)
         (def foo 200)
         (println "The value of foo is:" foo))
The value of foo is: 100
The value of foo is: 100

Wait, what? As I said, you can’t rebind foo inside the let form. It is a “con­stant”. Another behav­ior of let is that it per­forms its bind­ings sequentially.

user=> (let
           [foo 100
            bar foo]
         (println "The value of foo is:" foo)
         (println "The value of bar is:" bar))
The value of foo is: 100
The value of bar is: 100

So that cov­ers the basics of the let spe­cial form.

Next, we have the binding spe­cial form, and it’s a tad more com­pli­cated to under­stand. So let’s first take the API doc­u­men­ta­tion:

Cre­ates new bind­ings for the (already-existing) vars, with the sup­plied ini­tial val­ues, exe­cutes the exprs in an implicit do, then re-establishes the bind­ings that existed before. The new bind­ings are made in par­al­lel (unlike let); all init-exprs are eval­u­ated before the vars are bound to their new values.

So, first, the Var must already exist. You can’t do this:

user=> (binding [bar 100] (println "The value of bar is:" bar))
java.lang.Exception: Unable to resolve var: bar in this context

So how might we solve this? Well, if we don’t care about the root bind­ing, but only the bind­ing that exists within the binding spe­cial form, then we could do some­thing like this:

user=> (def bar)
#'user/bar
user=> (binding [bar 100] (println "The value of bar is:" bar))
The value of bar is: 100

Recall, that def with­out a value just cre­ates an unbound Var. If we had tried to refer to it prior to binding it, then we would have got­ten an excep­tion. I’m not sure this is a wise way to use things, but in my mind binding is pri­mar­ily some­thing to be used in a con­cur­rent envi­ron­ment since any binding is always thread-local.

So what does binding do? It replaces the root bind­ing of the Var with a thread-local bind­ing for any­thing inside the form. So, for example:

user=> (def bar 10)
#'user/bar
user=> bar
10
user=> (binding [bar 100]
  (println "The value of bar is:" bar))
The value of bar is: 100
user=> bar
10

As you can see, bar returns back to it’s orig­i­nal value out­side the form. Now, let’s play with it in a sim­i­lar way to how we did with let earlier.

user=> (defn print-bar [] (println "The value of bar is:" bar))
#'user/print-bar
user=> (print-bar)
The value of bar is: 10
user=> (binding [bar 100] (print-bar))
The value of bar is: 100

In this case, we’ve swapped the root bind­ing out from under the orig­i­nal one and we’re not actu­ally shad­ow­ing it as we would with let. Another dif­fer­ence is that binding per­forms its ini­tial bind­ings in par­al­lel, which means you can’t have depen­den­cies on ear­lier bind­ings made in the same form.

user=> (def baz 20)
#'user/baz
user=> (defn print-baz [] (println "The value of baz is:" baz))
#'user/print-baz
user=> (binding [bar 100 baz bar]
         (print-bar)
         (print-baz))
The value of bar is: 100
The value of baz is: 10

There’s one last lit­tle piece, and that’s the behav­ior of anony­mous func­tions inside let ver­sus their behav­ior inside of binding.

user=> (binding [print-bar print-baz] #(print-bar))
#<user$eval__1620$fn__1621 user$eval__1620$fn__1621@144683c2>

Wait, what just hap­pened? Why didn’t it print some­thing? That’s sim­ple. The cre­ation of the anony­mous func­tion doesn’t call the func­tion, so the result of the form is the func­tion, not the value of call­ing the func­tion. So that’s point one. Point two is that if you look at what was returned, say by cap­tur­ing it in another Var, you’ll see that it’s the orig­i­nal print-bar, not the rebound one.

user=> (def print-what (binding [print-bar print-baz] #(print-bar)))
#'user/print-what
user=> (print-what)
The value of bar is: 10

Why might that be? It’s because the anony­mous func­tion was actu­ally eval­u­ated “out­side” the bind­ing. This is because it’s actu­ally cre­ated by the reader macros. So, what do you do if you want the binding to apply?

user=> (binding [print-bar print-baz] (#(print-bar)))
The value of baz is: 20

In this sit­u­a­tion, the extra set of paren­the­sis force the eval­u­a­tion of the anony­mous func­tion inside the binding.

Hope­fully, this has been of some use to some peo­ple. It’s been use­ful for me in think­ing through the dif­fer­ences and the uses of the two forms. As with the orig­i­nal post, I come away with a few spe­cific guidelines.

1. Use let unless you really know you need binding.
2. Use binding when you need a thread-local ver­sion of a Var.
3. let is assigns sequen­tially, but binding is parallel.

If I’ve made any mis­take, no mat­ter how minor, please cor­rect me in the com­ments, and I’ll update this to reflect it. This is an excep­tion­ally impor­tant, but also sub­tle, area of languages.

Tagged: clojure, lisp