Friday, June 05, 2009

Adding a Remotely Accessible REPL to a Clojure Application

One of the distinguishing features of Lisp systems, including Clojure, is that the components that parse and compile code are accessible to user programs, even at runtime (i.e. "the whole language always available"). This enables sophisticated language features like macros, and is very different than the traditional Java worldview of distinct compile- and run-times.

The most familiar application of this concept is the interactive REPL that comes with Clojure.
Because the REPL is written in Clojure itself, it is available to any user program as the function clojure.main/repl. We can use this in conjunction with the networking libraries from the JDK to add a REPL interface to our application that is accessible over a socket connection. This gives us the ability to interact with and even modify a running application (e.g. by adding new code or changing data values) simply by connecting to the REPL and evaluating Clojure expressions. Moreover, since none of these activities requires an application restart or a lengthy rebuild process, it can be a huge time saver during the development process. 

There are only a few modifications to our skeletal web app to make this happen. First, I'll make a slight modification to the ClojureContextListener so that it accepts a whitespace-separated list of files to evaluate on startup, rather than just a single file:
public void contextInitialized(ServletContextEvent sce) {
try {
ServletContext sc = sce.getServletContext();
String evalOnContextInitialized = sc.getInitParameter("evalOnContextInitialized");
String[] scripts = evalOnContextInitialized.split("\\s+");
for (String scripts : scripts) {
RT.loadResourceScript(script);
}
}
catch (Exception e) {
// log an error message here ...
}
}
Then we'll create the file myapp/repl.clj, in the WEB-INF/classes directory of our application that provides the functions we need to start and run the REPL. Our REPL will run on a separate thread and serve a single client at a time. Because the built-in repl function expects to send and receive data from *in*, *out* and *err*, we just need to rebind them to the input and output streams associated with our socket connection:
(ns myapp.repl
(:require clojure.main)
(:import (java.io InputStreamReader PrintWriter)
(java.net ServerSocket Socket)
(clojure.lang LineNumberingPushbackReader)))

(defn do-on-thread
"Create a new thread and run function f on it. Returns the thread object that
was created."
[f]
(let [thread (new Thread f)]
(.start thread)
thread))

(defn socket-repl
"Start a new REPL that is connected to the input/output streams of
socket."
[socket]
(let [socket-in (new LineNumberingPushbackReader
(new InputStreamReader
(.getInputStream socket)))
socket-out (new PrintWriter
(.getOutputStream socket) true)]
(binding [*in* socket-in
*out* socket-out
*err* socket-out]
(clojure.main/repl))))

(defn start-repl-server
"Creates a new thread and starts a REPL server on listening on port. Returns
the server socket that was just created."
[port]
(let [server-socket (new ServerSocket port 0)]
(do-on-thread #(while true (socket-repl (.accept server-socket))))
server-socket))
And we'll add at the end of the file the following snippet that starts the REPL server, if the system property replPort has been defined:
(let [repl-port (System/getProperty replPort)]
(if (not (null? repl-port))
(def *repl-server* (start-repl-server (Integer/parseInt repl-port)))))
This will allow us to disable the remote REPL (for example, in production) by simply omitting the replPort property. Lastly we'll add this to web.xml
   <context-param>
<param-name>evalOnContextInitialized</param-name>
<param-value>myapp/repl.clj myapp/web.clj</param-value>
</context-param>
Assuming that we have started our application with -DreplPort=12345 option, at this point we can any generic client (such as netcat) to connect to our application:
tinman:~ sean$ nc localhost 12345
clojure.core=>
 



UPDATE:It turns out that a remote REPL capability is available in the clojure.contrib.server-socket package. I found it accidentally when my Emacs tags file led me to its socket-repl function instead of my own. Interestingly, the two implementations are very similar.

1 comment:

mst said...

Great article, thanks!

Just one comment, I think you could replace your `do-on-thread' function with Clojure's built-in `pcalls'.