A User Interface Definition Language in Common Lisp

Wednesday, February 13, 2008

Introduction

I like HTML in the same way I like PDF - as a document serialization format that does a reasonable job and that I never want to modify by hand. This is one of the main reasons I started Weblocks framework - I never wanted to write a line of HTML again. Ironically, I ended up writing a lot of HTML and learning more about its quirks, accessibility issues, and CSS hooks than I ever wanted to, but I finally ended up with a high level user interface definition language embedded into Common Lisp. Finally, HTML is out of my life, for good. Being lazy does pay off.

Defining a User Interface

So how does it work? From the perspective of a web application developer it's very simple. You just specify the user interface in a declarative manner and the engine figures out how to convert it to HTML:

(defview employee-form (:type form :default-method :post)
  first-name
  (last-name :requiredp t
             :label "Family Name")
  (contract :present-as (radio :choices
                          '(:full-time :part-time :consultant :intern))
            :parse-as keyword)
  (age :present-as (input :max-length 3)
       :parse-as integer))

Rendering the object into HTML is then as simple as typing the following line:

(render-object-view some-employee 'employee-form)

It's up to the engine to render the form, labels, input fields, radio buttons, etc. Reasonable defaults are used whenever possible, and when the defaults aren't sufficient the developer can declaratively specify how a field is to be rendered, how it is to be parsed, how the value of each field is to be acquired and stored, etc. The declarative language mirrors a set of extensible data structures that hold the rendering information, so when new renderers need to be provided for an obscure HTML construct I didn't think of, the developer can extend the engine with the construct once, and forget it ever existed.

The language is very flexible. For example, to render the contract choices as a dropdown and to obtain them dynamically we could declare the contract field like this:

(contract :present-as (dropdown :choices
                        (lambda (obj)
                          (declare (ignore obj))
                          (list :full-time :part-time :consultant :intern))
          :parse-as keyword))

Here is an example that defines an interface which simply renders textual information about an employee:

(defview employee-data-view ()
  first-name
  last-name
  contract
  age
  company)

The type of the view isn't specified because text view is the default, nor are there any special options for the fields.

This works very nicely, but what about reusing views when parts of the website require simple deviations?

Inheritance And Composition

So far we defined a simple UI for an imaginary employee object. However, real life web development is rarely that simple. Most user interfaces involve presenting complex, interrelated objects in various ways. To allow doing this, views provide two abstraction facilities - inheritance and composition.

Inheritance

Inheritance allows one view to inherit fields from another and to selectively override necessary fields. Consider presenting the employee - in case we're on a particular company's directory, we don't want to show which company each employee works for - we already know which directory we're in. However, if we're building a directory of all contractors that work for the government and we stumble on a give employee, we want to know which contractor the employee works for.

We could copy-paste the employee-data-view and remove the company field for the company directory view, but a much more elegant way to solve the problem is to have the new view inherit from the old one:

(defview employee-company-directory-view (:inherit-from 'employee-data-view)
  (company :hidep t))

Now employee-company-directory-view has all the fields in employee-data-view, except we override the company field and hide it. We can also add new fields, if the need arises.

Composition

Composition allows mixing in views. Consider a view to present an address:

(defview address-form-view (:type form)
  street
  city
  (state :present-as us-state)
  zip)

It can be used in a large number of settings - as part of the employee profile, company information, credit card payment form, etc. We could have other views inherit from the address view, but we wouldn't get positioning flexibility, wouldn't be able to mix in multiple fields, and would prevent the view from inheriting from other, more relevant views. Composition solves these problems:

(defview credit-card-form (:type form)
  number
  expiration
  (address :type mixin
           :view 'address-form-view)
  (street :hidep t))

This way the address view is mixed in. The fields of the address are presented inline in the credit card view. We can mix in as many fields as we wish. Additionally, we can override the way address fields are presented in the credit card view. In this example the street is hidden, as it may not be relevant to credit card validation.

View inheritance and composition provide powerful means to define, structure, and customize user interfaces. But can things get better? I think they can.

Scaffolding

When I heard about a new Ruby framework called "Rails" that automatically generates interfaces from class definitions I was ecstatic. Finally, someone wrote a framework I always wanted to write! A class definition, in most languages, is fairly close to a UI definition language (albeit not incredibly expressive). By default, a tremendous amount of valuable information can be obtained from member names and types, and this information can be used to generate default user interfaces!

I downloaded Rails first chance I got. Unfortunately I was in for a disappointment. The framework converted class definitions into HTML. It was a one way script, not a true abstraction. If I modified the generated code, I couldn't update the model and regenerate the interface - my modifications would be lost.

The proper way to implement scaffolding, the way that's actually useful throughout the entire project, not merely at the beginning, is to introspect class definitions and generate high level user interface definition data structures that can later be selectively overridden in a declarative manner. We can do that by taking advantage of the inheritance mechanism described above. Consider the following snippet:

(defview employee-data-view (:inherit-from '(:scaffold employee)))

Now our view inherits from a scaffold view. The scaffold view generates a sensible interface from the CLOS object in question. It turns out that Common Lisp class definitions are so expressive, very good interfaces can be generated by default. Consider the following class definition:

(defclass employee ()
  ((first-name :reader employee-first-name)
   (last-name :accessor employee-last-name
              :type string)
   (contract :accessor employee-contract
             :type (member :full-time :part-time :consultant :intern))
   (age :accessor employee-age
        :type (or null integer))))

We can gain an incredible amount of information from this definition. We know that we have four fields: first name, last name, contract, and age. We know that first name should be read-only, as only a reader is defined for the corresponding slot. We know that last name and contract should be required fields, because their type signature doesn't permit for nulls. Similarly we know that age should be optional, as null is permitted, and that it must be an integer. We also know that contract can be one of four values.

The class definition is so succinct, one may wonder why not use CLOS classes in the first place without defining a custom language. In fact, this is the route I took initially, only to find out later that it doesn't work. The same object is often rendered in different ways, and people aren't very fond of changing their data model to change the way a field is rendered. I also couldn't define a custom metaclass so that specialized rendering information for each slot could be put in place. Firstly, more often than not model classes already have a custom metaclass to allow for mapping to a backend store, and secondly this wouldn't easily accommodate creating multiple different views of the same object.

Every field we define in employee-data-view overrides the fields automatically defined by the scaffold view. We can hide fields defined by the scaffold by setting :hidep option to true, set custom presentations and parsers, and change every other parameter. Of course interfaces can be defined without the use of scaffolding. But why would anyone want to do a thing like that?

What's Next?

So, does any of this work? You bet it does! And coupled with closures-based actions, continuations-based control flow, stateful widgets, and a shiny new CLSQL backend store1, it may very well be the web development nirvana2. Download Weblocks, and see for yourself.

Comments?

If you have any questions, comments, or suggestions, please drop a note at coffeemug@gmail.com. I'll be glad to hear your feedback.

1You can also use a built-in, zero configuration cl-prevalence backend. Or you can roll your own.

2Weblocks is relatively young, so there still may be some issues that need to be ironed out. Proceed with caution!