Friday, June 05, 2009

Adding a Remotely Accessible REPL to a Clojure Application

One of the distinguishing features of Lisp systems, including Clojure, is that the components that parse and compile code are accessible to user programs, even at runtime (i.e. "the whole language always available"). This enables sophisticated language features like macros, and is very different than the traditional Java worldview of distinct compile- and run-times.

The most familiar application of this concept is the interactive REPL that comes with Clojure.
Because the REPL is written in Clojure itself, it is available to any user program as the function clojure.main/repl. We can use this in conjunction with the networking libraries from the JDK to add a REPL interface to our application that is accessible over a socket connection. This gives us the ability to interact with and even modify a running application (e.g. by adding new code or changing data values) simply by connecting to the REPL and evaluating Clojure expressions. Moreover, since none of these activities requires an application restart or a lengthy rebuild process, it can be a huge time saver during the development process. 

There are only a few modifications to our skeletal web app to make this happen. First, I'll make a slight modification to the ClojureContextListener so that it accepts a whitespace-separated list of files to evaluate on startup, rather than just a single file:
public void contextInitialized(ServletContextEvent sce) {
try {
ServletContext sc = sce.getServletContext();
String evalOnContextInitialized = sc.getInitParameter("evalOnContextInitialized");
String[] scripts = evalOnContextInitialized.split("\\s+");
for (String scripts : scripts) {
RT.loadResourceScript(script);
}
}
catch (Exception e) {
// log an error message here ...
}
}
Then we'll create the file myapp/repl.clj, in the WEB-INF/classes directory of our application that provides the functions we need to start and run the REPL. Our REPL will run on a separate thread and serve a single client at a time. Because the built-in repl function expects to send and receive data from *in*, *out* and *err*, we just need to rebind them to the input and output streams associated with our socket connection:
(ns myapp.repl
(:require clojure.main)
(:import (java.io InputStreamReader PrintWriter)
(java.net ServerSocket Socket)
(clojure.lang LineNumberingPushbackReader)))

(defn do-on-thread
"Create a new thread and run function f on it. Returns the thread object that
was created."
[f]
(let [thread (new Thread f)]
(.start thread)
thread))

(defn socket-repl
"Start a new REPL that is connected to the input/output streams of
socket."
[socket]
(let [socket-in (new LineNumberingPushbackReader
(new InputStreamReader
(.getInputStream socket)))
socket-out (new PrintWriter
(.getOutputStream socket) true)]
(binding [*in* socket-in
*out* socket-out
*err* socket-out]
(clojure.main/repl))))

(defn start-repl-server
"Creates a new thread and starts a REPL server on listening on port. Returns
the server socket that was just created."
[port]
(let [server-socket (new ServerSocket port 0)]
(do-on-thread #(while true (socket-repl (.accept server-socket))))
server-socket))
And we'll add at the end of the file the following snippet that starts the REPL server, if the system property replPort has been defined:
(let [repl-port (System/getProperty replPort)]
(if (not (null? repl-port))
(def *repl-server* (start-repl-server (Integer/parseInt repl-port)))))
This will allow us to disable the remote REPL (for example, in production) by simply omitting the replPort property. Lastly we'll add this to web.xml
   <context-param>
<param-name>evalOnContextInitialized</param-name>
<param-value>myapp/repl.clj myapp/web.clj</param-value>
</context-param>
Assuming that we have started our application with -DreplPort=12345 option, at this point we can any generic client (such as netcat) to connect to our application:
tinman:~ sean$ nc localhost 12345
clojure.core=>
 



UPDATE:It turns out that a remote REPL capability is available in the clojure.contrib.server-socket package. I found it accidentally when my Emacs tags file led me to its socket-repl function instead of my own. Interestingly, the two implementations are very similar.

Wednesday, June 03, 2009

Setting up a Clojure development environment is is fairly straightforward, although the tool support is fairly limited at present. The choices are:
  1. An Emacs clojure-mode, and that provides some support for the SLIME Lisp development environment;
  2. An Eclipse plugin, Clojure-dev;
  3. A NetBeans plugin, Enclojure.
All of these are fairly rudimentary. They each support syntax highlighting and formatting and some degree of REPL integration. The Eclipse and NetBeans plugins provide the project management capabilities of their environments, but the more advanced features (refactoring, automatic symbol cross-referencing, debugging) aren't available. Hopefully they will be added as the tools mature.

At this point, the most advanced version is Enclojure. Having been an Eclipse user for the past few years, learning a new tool has been somewhat annoying, but most of the features are present in NetBeans. The key advantage of Enclojure (over Clojure-dev) at this point is the ability to connect to a remote REPL, which is a very cool feature that I'll talk more about in a future post.

Monday, June 01, 2009

Because Clojure has such tight integration with the JVM, embedding it into a Java application is a snap. In particular, writing Clojure code that runs inside of a J2EE web container and handles web requests is trivial.

We just need to create two simple pieces of scaffolding code in Java to help the container find and dispatch web requests to our Clojure functions.

The first item is a ServletContextListener that initializes the Clojure runtime and evaluates our code when the web application is first loaded by the container:

package myapp.servlet;

import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;

import clojure.lang.RT;

public class ClojureContextListener implements ServletContextListener {

public void contextInitialized(ServletContextEvent sce) {
try {
ServletContext sc = sce.getServletContext();
String script = sc.getInitParameter("evalOnContextInitialized");
RT.loadResourceScript(script);
}
catch (Exception e) {
// log an error message ...
}
}

public void contextDestroyed(ServletContextEvent sce) {
// nothing to do here ...
}

}

We'll expect a<context-param> "evalOnContextInitialized" from the application's deployment descriptor to contain the name of a file containing our functions.

The second bit of helper code is a simple HttpServlet that delegates calls to its doService method to a Clojure function of our choice:

package myapp.servlet;

import javax.servlet.ServletConfig;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import clojure.lang.RT;
import clojure.lang.Var;

public class ClojureServlet extends HttpServlet {

private Var entryPoint;

@Override
public void init(ServletConfig sc) throws ServletException {
String entryPointString = sc.getInitParameter("entryPoint");
if (entryPointString != null) {
String[] nsAndName = entryPointString.split("/", 2);
entryPoint = RT.var(nsAndName[0], nsAndName[1]);
}
else {
throw new ServletException("Servlet " + sc.getServletName() +
" missing 'entryPoint' parameter!");
}
}

@Override
protected void service(HttpServletRequest req, HttpServletResponse resp)
throws ServletException {
try {
entryPoint.invoke(req, resp);
}
catch (Exception e) {
throw new ServletException(e);
}
}

}

The name of the Clojure function will be specified as an <init-param> in the servlet's configuration. We will follow the Clojure convention of namespace/name convention for identifying our function.

The final (and somewhat anti-climactic) piece of the puzzle is to create the Clojure code that actually handles the web request. This will be be a function of two arguments that accepts the javax.servlet.http.HttpServletRequest and javax.servlet.http.HttpServletResponse objects provided by the web container. We'll put in a simple "Hello world" handler to demonstrate.

(ns myapp.web)

(defn hello
"Hello world"
[req resp]
(.. resp (getWriter) (println) (str "<html><body>" "Hello world" "</html><body>"))

Since our hello function is in the the "myapp.web" namespace, it should be contained in a file named myapp/web.clj, that is on the application's CLASSPATH. In our case, this means in a directory beneath the WEB-INF/classes of the WAR file.

Accordingly, our web.xml file should look like:
<web-app version="2.4"
xmlns="http://java.sun.com/xml/ns/j2ee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd">

<context-param>
<param-name>evalOnContextInitialized</param-name>
<param-value>myapp/web.clj</param-value>
</context-param>

<listener>
<listener-class>myapp.servlet.ClojureContextListener</listener-class>
</listener>

<servlet>
<servlet-name>ClojureServlet</servlet-name>
<servlet-class>myapp.servlet.ClojureServlet</servlet-class>
<init-param>
<param-name>entryPoint</param-name>
<param-value>myapp.web/hello</param-value>
</init-param>
</servlet>

<servlet-mapping>
<servlet-name>ClojureServlet</servlet-name>
<url-pattern>/hello</url-pattern>
</servlet-mapping>

</web-app>

And the resulting web application will be laid out as follows:

myapp.war/
WEB-INF/
web.xml
classes/
myapp/
web.clj
servlet/
ClojureServlet.class
ClojureContextListener.class
lib/
clojure-1.0.0.jar


Note that the single clojure-1.0.0.jar file is the only required dependency. That's all there is to it!