Programming Clojure, Smalltalk-style
This is the first post about a system I call cloxp, a development environment for Clojure that facilitates interactivity and exploration. In the following I want to explain the motivation of the project.
Apart from this introduction I plan two more posts, one explaining some of the features of cloxp and one describing change handling and persistency. Please note that cloxp is a project in its very early stages. There are a lot of rough edges and missing features. However, I already enjoy working with it and I would love to get constructive feedback about how to improve it and how to drive the development forward.
Cloxp is open source and you will be able to use it both on the web without installation or sign on and locally by installing it on your system. I'll be happy about any help and feedback that you might be able to offer!
I'll post more information about how to try out the system in the next few days. Here is a short sneek peak for now:
Interactive programming
Clojure comes with a set of qualities that shape how programmers can interact with the language and runtime environment. To a large part this is derived from the powerful yet simple ideas and concepts Clojure builds on. Properties such as homoiconicity (your code is nothing special, "just" data) enable several versatile features: Meta-programming based on code transformations (macros). Documentation and other meta-data that becomes part of the actual system. Structural editing and inspection because, well, code is just data!
Clojure is entirely dynamic; the runtime brings all that is needed to execute code even "after" the program was created. From the standpoint of a programmer this means that there is no real difference between "compile time" and "runtime". Programming becomes much more of a flow of changing the code structure a little or trying something out and immediately seeing the effect. The program does not need to be stopped to reset program state.
It is not just about saving time while programming. One important aspect is not to interrupt the flow of creation. As a result the system supports a workflow that is more like a conversation, a back and forth between programmer and system. Actually, some of the earliest interactive computer systems such as Ivan Sutherland's Sketchpad and Doug Engelbart's oN-Line System were focused on exactly this aspect: How can computers support and interact with us in order to "augment" and extend our abilities? I stand in awe realizing the depth of those ideas and visions. Taking those and comparing them to something as tiny and rough as cloxp now is seems naïve and blasphemous. When discussing tools and languages it is easy to get lost in the nitty-gritty details and keeping in mind the "bigger picture" can help when thinking about why we want such tools in the first place and what kind of features and workflows are desirable.
Even though a lot of the available tooling for Clojure is really good when compared to other environments, with such a great underlying system the tooling and workflows around the language deserve to be made even better, especially if you keep the original visions of those pioneering systems mentioned above in mind.
I have really enjoyed working with Cider, Cursive, and LightTable. These are all good tools for Clojure development and can already support an interactive workflow via their repl integrations. If you want to use Clojure in production today these tools will be a much better fit than cloxp. With cloxp, however, it should be possible to explore alternative workflows that embrace more the dynamic "liveness" that a Clojure runtime system can provide and that are less file-centric and configuration-driven than current tooling.
Smalltalk, Self, and the power of directness and liveness
To go forward, it might be worth looking back some more. Besides Lisp, Smalltalk is another family of systems that pioneered various aspects of computer interaction and programming. Even more than Lisp, Smalltalk was focused on interactive and exploratory programming from its earliest version on. There is at least one major difference between the two languages/environments: Smalltalk's tooling provides a rich set of widgets and graphical representation of objects, besides text. Those objects can be used to visualize data, and can capture input and gestures from users. Since a Smalltalk system is as integrated as a Lisp system, those graphical facilities and their primitives can not only provide a tooling layer but can also be used by programmers to create rich graphical applications themselves*.
Many people favor their vim/emacs/zsh setup over any graphical programming environment. This is understandable: text is an extremely powerful representation of data, and it is mostly independent of a particular encoding or format. Tooling to process text exists in various forms and across systems. However, I'll argue that it is possible to combine both worlds effectively and hope that cloxp can be a proof-of-concept. Keep in mind that both graphical and textual representations are just that — representations. Even though a data structure can be made explorable and modifiable in a graphical dropdown inspector doesn't mean that it couldn't be modified (maybe even at the same time) in a textual representation. om is a beautiful example how a functional style system can make it easy to render and synchronize between different views of data structures. In the same way, I think, it is possible to bring more richness and power into our daily tools.
Self is a system that proved how a more direct and interactive workflow can simplify working with software systems. Software, by definition, is about creating abstractions. If a tool can make those abstractions explorable, "touchable", the way we interact and think about systems while programing can be fundamentally improved. Instead of just standing outside the system and trying to reason about some abstract algorithm, Self enabled the programmer to go "into the objects" and explore the system from a "first-person" viewpoint. Self's UI, Morphic shows how these ideas can form a conceptually and technically simple set of rules that is applicable to other systems as well.
How does this fit in with Clojure and cloxp? Clojure provides the runtime system that makes it possible to implement such a "first-person viewpoint". The data and the objects that you work with are accessible and explorable, and being able to explore them using different representations can be very helpful while learning, debugging, and reasoning about the systems you create or explore. Since, again, code is just data, this automatically covers the entire system. A long term goal is to integrate directness and rich representations into cloxp while not forgetting about the importance of simplicity that comes with text and "plain data".
*There are a lot of great resources to learn more about Smalltalk and the ideas behind it: Squeak, for example, is a current and open source Smalltalk environment, very much in tradition with the original ideas.
System-centric development
Clojure's runtime and meta-programming facilities enable one to view the system "as a whole". Once the system is running, the static structure, such as files and configurations that so many of today's tools and workflows emphasize, become less important. By interacting with the live system, you can discover, for example by tracing or step-wise code execution, how system parts play together and thus understand relationships much more rapidly.
Since code entities are still known at runtime, you can refer to them directly, ask for their documentation and see where they are used. Instead of referring to code by line and column number and changes by diffs of files, you can track code entities directly. It becomes possible to view and explore programs and their parts outside of their original file context in changeable and dynamic view compositions. Even implementing cool features such as a code-centric chat becomes possible.
Most tooling for Clojure is focused around a file- and project-specific workflow. To a certain degree this is necessary to fit in with the rest of the world and create software usable outside of a Clojure context. However, while programming, learning and exploring, programming tools should allow us to see the runtime system. One of the main contributions of cloxp at this point is a Smalltalk-style system browser that allows you to browse and search namespaces and methods per-entity. It is possible to jump to the definitions of functions and vars you work with, even if they are outside of your own project. Going forward, I want to emphasize these code navigation and exploration features that make Smalltalk environments so much fun to use. Since documentation is by default part of the code in Clojure, this facility usually allows one to avoid looking at the javadocs at all.
Why Clojure?
The main motivation of cloxp is to provide tool support around live and interactive programming workflows. It is based on Clojure because Clojure comes with a set of attractive properties. The prerequisite of a reflective runtime environment and well defined abstraction for system-level objects around namespaces and vars is one. Clojure also combines an exciting mindset around simplicity with enough pragmatism to make it a successful system for real-world projects. This sounds like a contradiction, yet Clojure's success is proof enough that this approach actually works. Additionally, Clojure's strong view on how state is managed, the distinction between identities and values and the focus on immutability all make it easier to deal with the complexities that come with "live programming". Having a strong concept of what state is and how state transitions occur makes programming tools simpler.
Even though the system currently only implements a small set of the ideas presented here, I hope that this will be the scope for the features yet to come. I wish to thank my employer CDG Labs without whose support I would not be able to work on such cool things :)