Printing Seqs

In my previous post I referred to how executable code was "list like", and we explored the property of being executable was defined by the ISeq interface. But one element that I didn't follow up on was that the REPL prints these seqs as a list with parentheses. How is this done, and does it exactly align with seqs?

Clojure has been around since before interface methods in Java, and if we look in the ISeq interface we can confirm that there is nothing there. However, two of the primary seqs are derived from the abstract class ASeq, so this looks like a reasonable place to start:

public String toString(){
  return RT.printString(this);
}

So this is jumping immediately to the static method clojure.lang.RT.printString(Object). This captures the output of the print(Object,Writer) method and returns the generated string:

static public String printString(Object x){
  try {
    StringWriter sw = new StringWriter();
    print(x, sw);
    return sw.toString();
  }
  catch(Exception e) {
    throw Util.sneakyThrow(e);
  }
}

The print(Object,Writer) method on line 1881 is quite long, so I'll jump ahead to the relevant section on line 1902:

    if(x == null)
      w.write("nil");
    else if(x instanceof ISeq || x instanceof IPersistentList) {
      w.write('(');
      printInnerSeq(seq(x), w);
      w.write(')');
    }

So we can see that all ISeq objects are printed in a parenthesized list, but so too are instances of IPersistentList, which do not inherit from ISeq. So my first question is, does anything implement IPersistentList that isn't also an ISeq? The answer is yes:

$ grep IPersistentList *.java | grep implements
PersistentList.java:public class PersistentList extends ASeq implements IPersistentList, IReduce, List, Counted {
PersistentList.java:    static class EmptyList extends Obj implements IPersistentList, List, ISeq, Counted, IHashEq{
PersistentQueue.java:public class PersistentQueue extends Obj implements IPersistentList, Collection, Counted, IHashEq{

This isn't a definitive way to find everything that implements an interface, and it's not exactly how I looked, but it keeps the output small for a blog post.

This shows the only type in Clojure that is a PersistentList but not an ISeq: PersistentQueue. This is used internally by agents, but nothing else. This class is briefly discussed in Michael Fogus and Chris Houser's excellent book "The Joy of Clojure". While there are no functions for creating these queues, an empty queue can be accessed directly through clojure.lang.PersistentQueue/EMPTY.

#_=> (def q (into clojure.lang.PersistentQueue/EMPTY [:a :b :c :d]))
#'user/q

#_=> q
#object[clojure.lang.PersistentQueue 0x771db12c "clojure.lang.PersistentQueue@298384c8"]

#_=> (seq q)
(:a :b :c :d)

Unlike other lists, these should only evaluate to themselves, and not execute anything:

#_=> (def q (into clojure.lang.PersistentQueue/EMPTY ['+ 2 3]))
#'user/q

#_=> q
#object[clojure.lang.PersistentQueue 0x3be4f71 "clojure.lang.PersistentQueue@266bfe13"]

#_=> (eval q)
Syntax error (ClassCastException) compiling fn* at (REPL:1:1).
class clojure.lang.PersistentQueue cannot be cast to class java.util.List (clojure.lang.PersistentQueue is in unnamed module of loader 'app'; java.util.List is in module java.base of loader 'bootstrap')

So we found an object that implements IPersistentList but is not a seq and cannot be evaluated as one. However, just like other seqable types, it can be evaluated by first converting it to a seq:

#_=> (eval (seq q))
5

Interestingly, the value of the PersistentQueue is shown as a raw object, and not as a list as we expected from the code in clojure.lang.RT/print(Object,Writer). That's because this code is called from ASeq which is not a parent of PersistentQueue. It's a little odd, since the print function we saw above is explicitly looking for this interface, and nothing built into Clojure implements this interface except PersistentQueue.

Printing

For fun, I thought it would be nice to use the RT.print method to print the queue:

#_=> (clojure.lang.RT/printString q)
"#object[clojure.lang.PersistentQueue 0x3be4f71 \"clojure.lang.PersistentQueue@266bfe13\"]"

This was unexpected. It was printed as an object, despite meeting the IPersistentList interface.

The answer was hidden in plain sight right at the top of this function:

static public void print(Object x, Writer w) throws IOException{
  //call multimethod
  if(PRINT_INITIALIZED.isBound() && RT.booleanCast(PRINT_INITIALIZED.deref()))
    PR_ON.invoke(x, w);
//*
  else {

Oops. 😳

OK, so clojure.core/print-initialized is in fact bound, and set to true. I should have paid better attention.

At the top of clojure.lang.RT we can see references to lots of definitions, including PRINT_INITIALIZED and PR_ON (lines 237,238) both of which are in clojure.core:

static final Var PRINT_INITIALIZED = Var.intern(CLOJURE_NS, Symbol.intern("print-initialized"));
static final Var PR_ON = Var.intern(CLOJURE_NS, Symbol.intern("pr-on"));

Looking in clojure.core starting at line 3663 we find:

(def ^:dynamic ^{:private true} print-initialized false)

(defmulti print-method (fn [x writer]
                         (let [t (get (meta x) :type)]
                           (if (keyword? t) t (class x)))))
(defmulti print-dup (fn [x writer] (class x)))

(defn pr-on
  {:private true
   :static true}
  [x w]
  (if *print-dup*
    (print-dup x w)
    (print-method x w))
  nil)

So print-initialized is false, but it's also dynamic, so we know it can change (and probably has). Also, we see the pr-on function that was invoked in clojure.lang.RT.print(Object,Writer), and it dispatches to either print-dup or print-method, depending on the value of *print-dup*. This is documented in clojure.core:

When set to logical true, objects will be printed in a way that preserves
their type when read in later.
Defaults to false.

We're investigating what things look like at the REPL, so we can just go looking for defmethods for print-method. We find these in the file core_print.clj. But interestingly, it's not a different namespace. Instead, it expands the clojure.core namespace by starting the file with:

(in-ns 'clojure.core)

So remember… just because something doesn't appear in a Clojure file doesn't mean that it's not in that namespace!

Tweaking core-print

Line 174 of core_print.clj defines a print-method for an ISeq:

(defmethod print-method clojure.lang.ISeq [o, ^Writer w]
  (print-meta o w)
  (print-sequential "(" pr-on " " ")" o w))

The print-meta function (line 72) only prints anything if there is metadata and the *print-meta* var is turned on (which it isn't). So it all comes down to calling print-sequential with arguments for:

• The leading string.
• The function to print elements of the sequence.
• The separator string.
• The trailing string.
• The object to print.
• The java.io.Writer to write to.

Multimethods can be redefined, so let's try changing how seqs get printed. First of all, we want to get access to some of these private functions in clojure.core so we can reuse them in our own implementation. We can just save them in similarly named vars:

#_=> (def print-sequential* (deref #'clojure.core/print-sequential))
#'user/print-sequential*

#_=> (def pr-on* (deref #'clojure.core/pr-on))
#'user/pr-on*

#_=> (defmethod clojure.core/print-method clojure.lang.ISeq [o, ^Writer w]
       (print-sequential* "<<<" pr-on* ", " ">>>" o w))
#object[clojure.lang.MultiFn 0x27068a50 "clojure.lang.MultiFn@27068a50"]

#_=> (str '(1 2 3))
"<<<1, 2, 3>>>"

Yay, it worked!

This should also be the mechanism used by the REPL to print the value of an ISeq:

#_=> '(1 2 3)
<<<1, 2, 3>>>

This shows that we definitely have the correct code path for printing things at the REPL.

Now we can try going back to use the existing clojure.lang.RT.printString code on the PersistentQueue. This requires rebinding the print-initialized flag to false. This should also print the list the original way again:

#_=> (clojure.lang.RT/printString '(1 2 3))
"<<<1, 2, 3>>>"

#_=> (binding [clojure.core/print-initialized false]
       (clojure.lang.RT/printString '(1 2 3)))
"(1 2 3)"

#_=> (binding [clojure.core/print-initialized false]
       (clojure.lang.RT/printString q))
"(+ 2 3)"

It's a very round-a-bout way of getting to some code that will print a PersistenQueue without wrapping it in a seq first, but it can be done. More interesting was learning a bit more about the mechanisms for printing.