Clojure

Programming at the REPL: Enhancing your REPL workflow

At this point, you know enough to understand how the REPL works; we’ll now focus on giving you a more streamlined development experience with the REPL. There are a variety of things you may want to improve:

Switching between my editor and the REPL is tedious.

Most Clojure programmers don’t use the terminal-based REPL for everyday development: instead, they use a REPL integration in their editor, which enables them to write expressions directly in their editor buffer and have them evaluated in the REPL with one hotkey. See the Editor integrations section below for more details.

I want to do small experiments in Clojure, but it’s painful to write code in the default CLI tool.

As mentioned above, one solution could be to use an editor integration. Note that some editors such as Nightcode are designed specifically to provide a 'batteries-included' experience for Clojure.

However, if setting up an editor is too heavy for your taste, there also exist more ergonomic terminal REPL clients:

I need to debug a program I’m running from the REPL.

The REPL can definitely help you do that: see the Debugging Tools and Techniques section below.

I find myself repeating a lot of manual steps at the REPL for running my development environment.

Consider creating a 'dev' namespace in your project (e.g myproject.dev) in which you define functions for automating common development tasks (for example: starting a local web server, running a database query, turning on/off email sending, etc.)

When I make changes to my code, it’s often difficult to reflect that change in my running program: I have to do a lot of manual work in the REPL to make it happen..

Depending on the choices you make when writing your programs, interacting with them at the REPL will become more or less practical. See the Writing REPL-friendly programs section below.

I would like to save my REPL sessions in a 'notebook' format.

Gorilla REPL was made for this very purpose.

I want better data visualization than what the REPL provides.

You may get improved visualization features from a specialized Clojure editor: see the Editor Integrations section below.

Having said that, keep in mind that the REPL is a full-feature execution environment: in particular, you can use it to launch special-purpose visualization tools (including ones that you develop yourself). For instance:

  • oz is a Clojure library for displaying numerical charts

  • datawalk is a Clojure library for interactively exploring complex Clojure data structures

  • system-viz is a Clojure library for visualizing the components of a running Clojure system

I want to customize my REPL.

You can usually customize how your REPL reads, evaluates and prints, but the method to do it depends on your toolchain. For instance:

  • when using a REPL started from clojure.main (as is the case when using the clj tool) you can customize the REPL by launching a 'sub-REPL': see clojure.main/repl.

  • when using nREPL.[1], this can be done by writing custom middleware.

I want to use the REPL to connect to a live production system.

The Clojure socket server feature can be used to that end. Tools like tools.nrepl and unrepl can be used to provide a richer experience.

NOTE: You may not need all of this!

Depending on your projects and personal taste, you will most likely use only a fraction of the tools and techniques presented in this section. It’s important to know that these options exist, but don’t try to adopt all of them at once!

Editor integrations

All major Clojure editors support a way of evaluating code in the REPL without leaving the current code buffer, which reduces the amount of context switching a programmer has to do. Here’s what it looks like (the editor used in this example is Cursive):

Editor REPL integration

TIP: You can wrap some expressions in a (comment …​) block so that they don’t get evaluated by accident when the file is loaded:

;; you would NOT want this function to get called by accident.
(defn transfer-money!
  [from-accnt to-accnt amount]
  ...)

(comment
  (transfer-money! "accnt243251" "accnt324222" 12000)
  )

What to expect from an in-editor REPL integration?

Here are some common editor commands provided by REPL integrations. All major Clojure editors support a majority of them:

  • Send form before caret to REPL: evaluate the expression before the cursor in the REPL, in the namespace of the current file. Useful for experimenting in the context of the current namespace.

  • Send top-level form to REPL: evaluate the biggest expression in which the cursor is currently contained -typically a (defn …​) or (def …​) expression-in the namespace of the current file. Useful for defining or re-defining Vars in a namespace.

  • Load the current file in the REPL. Useful to avoid loading libs manually.

  • Switch the REPL’s namespace to current file: useful to avoid typing (in-ns '…​).

  • Show evaluation inline: displays the evaluation of the current expression next to it.

  • Replace expression with its evaluation: replaces the current expression in the editor with its evaluation (as printed by the REPL).

Debugging tools and techniques

While traditional debuggers can be used with Clojure, the REPL itself is a powerful debugging environment, because it lets you inspect and alter the flow of a running program. In this section, we’ll study some tools and techniques to leverage the REPL for debugging.

Printing in-flight values with prn

(prn …​) expressions can be added in strategic places in your code to print intermediary values:

(defn average
  "a buggy function for computing the average of some numbers."
  [numbers]
  (let [sum (first numbers)
        n (count numbers)]
    (prn sum) ;; HERE printing an intermediary value
    (/ sum n)))
#'user/average
user=> (average [12 14])
12 ## HERE
6

TIP: you can combine prn with the (doto …​) macro, i.e (doto MY-EXPR prn), to make adding prn calls less invasive:

(defn average
  "a buggy function for computing the average of some numbers."
  [numbers]
  (let [sum (first numbers)
        n (count numbers)]
    (/
      (doto sum prn) ;; HERE
      n)))

Going further: 'spying' macros

Some Clojure libraries provide 'enhanced' versions of prn that are more informative, by also printing information about the wrapped expression. For example:

  • the tools.logging logging library provides a spy macro to log an expression’s code along with its value

  • the spyscope library lets you to insert these printing calls with very lightweight syntax.

Going further: tracing libraries

Tracing libraries such as tools.trace and Sayid can help you instrument larger portions of your code, for example by automatically printing all the function calls in a given namespace, or all intermediary values in a given expression.

Intercepting and saving values on-the-fly

Sometimes you want to do more with intermediary values than just print them: you want to save them to conduct further experiments on them at the REPL. This can be done by inserting a (def …​) call inside the expression where the value appears:

(defn average
  [numbers]
  (let [sum (apply + numbers)
        n (count numbers)]
    (def n n) ;; FIXME remove when you're done debugging
    (/ sum n)))
user=> (average [1 2 3])
2
user=> n
3

This 'inline-def' technique is described in more depth in this blog post by Michiel Borkent.

Reproducing the context of an expression

When debugging at the REPL, we often want to reproduce manually something that our program did automatically, that is evaluating some expressions inside a function body. To do that, we need to recreate the context of the expressions of interest: one trick to achieve that is to define Vars (using def) with the same names and values as the locals used by the expressions. The 'physics' example below illustrates this approach:

(def G 6.67408e-11)
(def earth-radius 6.371e6)
(def earth-mass 5.972e24)

(defn earth-gravitational-force
  "Computes (an approximation of) the gravitational force between Earth and an object
  of mass `m`, at distance `r` of Earth's center."
  [m r]
  (/
    (*
      G
      m
      (if (>= r earth-radius)
        earth-mass
        (*
          earth-mass
          (Math/pow (/ r earth-radius) 3.0))))
    (* r r)))

;;;; calling our function for an object of 80kg at distance 5000km.
(earth-gravitational-force 80 5e6) ; => 616.5217226636292

;;;; recreating the context of our call
(def m 80)
(def r 5e6)
;; note: the same effect could be achieved using the 'inline-def' technique described in the previous section.

;;;; we can now directly evaluate any expression in the function body:
(* r r) ; => 2.5E13
(>= r earth-radius) ; => false
(Math/pow (/ r earth-radius) 3.0) ; => 0.48337835316173317

This technique is described in more depth in Stuart Halloway’s article REPL Debugging: No Stacktrace Required. The scope-capture library was made to automate the manual task of saving and re-creating the context of an expression.

Community resources about REPL debugging

Writing REPL-friendly programs

While interactive development at the REPL gives a lot of power to programmers, it also adds new challenges: programs must be designed so that they lend themselves well to REPL interaction, which is a new constraint to be vigilant of when writing code.[2]

Covering this topic extensively would take us too far for the scope of this guide, so we will merely provide some tips and resources to guide your own research and problem-solving.

REPL-friendly code can be re-defined. Code is more easily redefined when it is called via a Var (defined e.g via (def …​) or (defn …​)), because a Var can be redefined without touching the code that calls it. This is illustrated in the following example, which prints some numbers at a regular time interval:

;; Each of these 4 code examples start a loop in another thread
;; which prints numbers at a regular time interval.

;;;; 1. NOT REPL-friendly
;; We won't be able to change the way numbers are printed without restarting the REPL.
(future
  (run!
    (fn [i]
      (println i "green bottles, standing on the wall. ♫")
      (Thread/sleep 1000))
    (range)))

;;;; 2. REPL-friendly
;; We can easily change the way numbers are printed by re-defining print-number-and-wait.
;; We can even stop the loop by having print-number-and-wait throw an Exception.
(defn print-number-and-wait
  [i]
  (println i "green bottles, standing on the wall. ♫")
  (Thread/sleep 1000))

(future
  (run!
    (fn [i] (print-number-and-wait i))
    (range)))

;;;; 3. NOT REPL-friendly
;; Unlike the above example, the loop can't be altered by re-defining print-number-and-wait,
;; because the loop uses the value of print-number-and-wait, not the #'print-number-and-wait Var.
(defn print-number-and-wait
  [i]
  (println i "green bottles, standing on the wall. ♫")
  (Thread/sleep 1000))

(future
  (run!
    print-number-and-wait
    (range)))

;;;; 4. REPL-friendly
;; The following works because a Clojure Var is (conveniently) also a function,
;; which consist of looking up its value (presumably a function) and calling it.
(defn print-number-and-wait
  [i]
  (println i "green bottles, standing on the wall. ♫")
  (Thread/sleep 1000))

(future
  (run!
    #'print-number-and-wait ;; mind the #' - the expression evaluates to the #'print-number-and-wait Var, not its value.
    (range)))

Beware of derived Vars. If Var b is defined in terms of the value of Var a, then you will need to re-define b each time you re-define a; it may be better to define b as a 0-arity function which uses a. Example:

;;; NOT REPL-friendly
;; if you re-define `solar-system-planets`, you have to think of re-defining `n-planets` too.
(def solar-system-planets
  "The set of planets which orbit the Sun."
  #{"Mercury" "Venus" "Earth" "Mars" "Jupiter" "Saturn" "Uranus" "Neptune"})

(def n-planets
  "The number of planets in the solar system"
  (count solar-system-planets))


;;;; REPL-friendly
;; if you re-define `solar-system-planets`, the behaviour of `n-planets` will change accordingly.
(def solar-system-planets
  "The set of planets which orbit the Sun."
  #{"Mercury" "Venus" "Earth" "Mars" "Jupiter" "Saturn" "Uranus" "Neptune"})

(defn n-planets
  "The number of planets in the solar system"
  []
  (count solar-system-planets))

REPL-friendly code can be reloaded. Make sure that reloading a namespace will not alter the behaviour of the running program. If a Var needs to be defined exactly once (which should be very rare), consider defining it with defonce.

When dealing with a codebase with many namespaces, reloading the appropriate namespaces in the correct order can become difficult: the tools.namespace library was made to assist the programmer in this task.

Program state and source code should be kept in sync. You usually want to make sure that your program state reflects your source code and vice-versa, but this is not automatic. Reloading the code is often not enough: you also need to transform the program state accordingly. Stuart Sierra has expounded on this problem in his article My Clojure Workflow, Reloaded and his talk Components Just Enough Structure.

This has motivated the creation of state management libraries:

  • Component, which promotes a representation of program state as a managed map of Clojure records called a system.

  • System is a library on top of Component which provides a set of ready-made components.

  • Mount takes a radically different approach as Component, choosing to use Vars and namespaces as the supporting infrastructure for state.[3]

  • Integrant is a more recent library which shares Component’s approach while addressing some of its perceived limitations.


1 At the time of writing (March 2018), nREPL is the most popular toolchain for REPL-editor integration
2 A similar phenomenon happens with the well-known technique of automated testing: while testing can bring a lot of value to programmers, it requires extra care to write code that is 'testable'. Just like tests, the REPL should not be an afterthought when writing Clojure code.
3 At the time of writing, there is controversy in the Clojure community regarding the relative merits of both approaches.