Lisplets: Bridging the Gap Between Lisp and Java Servlets

In the early 2000s, the challenge for Lisp developers wanting to build web applications was often a choice between reinventing the wheel—writing their own HTTP servers and SSL handlers—or compromising on the language's flexibility. Enter Lisplets, a project created by Rich Hickey in 2004, designed to provide a Lisp-friendly interface to Java Servlets.

Lisplets acts as a bridge, allowing developers to use the industrial-strength capabilities of Java-based web environments (like Tomcat, Jetty, or Resin) while writing the actual application logic in Common Lisp or Scheme. By offloading the "plumbing" of the web to Java, Lisp developers can focus on the expressive power of their language without sacrificing scalability or security.

How Lisplets Works

At its core, Lisplets consists of a single Java class that implements HttpServlet. Instead of processing the request internally, the Lisplet servlet forwards the request and gathers the response using s-expressions over TCP sockets.

The Request Flow

1. Interception: The Java servlet container receives an HTTP request.
2. Packaging: The Lisplet servlet packages the request header, parameters, cookies, session data, application state, and user information into a single s-expression.
3. Transmission: This s-expression is sent over a TCP socket to a listening Lisp process.
4. Processing: The Lisp process reads the s-expression using a standard read function, processes the logic, and generates a response.
5. Return: The Lisp process sends back an s-expression containing the status, headers, and session state, followed by the raw document content.

Key Advantages for Lisp Developers

By utilizing a Java servlet container as a front-end, Lisplets provides several immediate benefits:

• Infrastructure for Free: Developers gain access to professional HTTP server implementations, SSL support, and integration with industry-standard web servers without writing a line of Java code.
• Enterprise Features: The Lisplet API reflects the full power of the Servlet API, including session management, authentication, access control, user roles, and declarative configuration.
• Hybrid Applications: Because it operates within a standard Java web application, Lisp-generated content can coexist with Java Server Pages (JSP) or other Java Servlets in a single application.
• Flexible Deployment: It works with any servlet container, meaning developers can choose their environment based on cost, embed-ability, or platform support.

Technical Deep Dive: The API

Configuration via web.xml

Lisplets is configured through the standard web.xml file. While lisp-host and lisp-port are required to locate the Lisp process, several optional parameters allow for fine-tuning:

• lisp-tag-prefix: Allows developers to prepend a string (e.g., :) to tags, effectively turning them into keywords in Common Lisp.
• lisp-create-session: A boolean that determines if a session should be automatically created if one doesn't exist.
• lisp-roles: A comma-separated list used to inquire about a user's membership in specific roles, integrating Lisp logic with Java's authentication system.

The Request and Response Format

Requests are sent as a single list of lists. Each sub-list begins with a tag (e.g., :method, :uri, :headers, :session). This structure allows the Lisp process to easily parse the request using standard list processing techniques.

Responses are similarly structured. The Lisp process must send at least an empty list () to the Java side. The response s-expression can control:

• HTTP Status: Setting the status (e.g., 200 OK) or triggering a redirect.
• Cookies: Defining name, value, domain, and path.
• Session/Context Attributes: Updating dynamic attributes on the Java side, which are then persisted by the servlet container.
• Logging: Sending strings directly to the ServletContext log.

Implementation Example

Rich Hickey provided a sample server written for Xanalys (LispWorks) to demonstrate the simplicity of the integration. The server listens on a port, reads the s-expression from the stream, and echoes the request back to the browser as the response body:

(defun handle-lisplet-on-stream (stream)
  (unwind-protect
      (let ((req (read stream)))
        (format stream "~S"
                `((:status 200)
                  (:headers (:content-type "text/plain"))
                  (:log "Hello from Lisp")))
        (pprint req stream)
        (force-output stream)
        (close stream))))

Summary

Lisplets represents an elegant solution to a classic problem: how to use a high-level, flexible language for logic while leveraging a stable, performant ecosystem for infrastructure. While the project dates back to 2004, its philosophy of using a simple, language-agnostic communication format (s-expressions) to bridge two different runtime environments remains a timeless architectural pattern.