Clojure Projects

I don't really know how to start this other than to look at various ways you can create functionality in a clojure project and attempt to categorize them in some productive way.

The first I want to talk about is the repl. Hopefully you have had the pleasure of working with a good repl setup (SLIME's is the best I have ever used, have you seen better?). REPL stands for read-eval-print-loop. It looks like a command prompt and you type in some stuff and immediately see what the result is. This is similar to when you break in a debugger and you can both analyze values *and* do edit and continue.

I am going to out on a limb here and state that the REPL is the fastest way to add new functionality. You type in a couple statements and you see, right after you type them if it worked or not. This is like Christmas. It is a fucking blast if you have never used it; trust me. It feels weird at first but you just have to use it for a couple days and you will be amazed at how much stuff you get to work. The rate you can add functionality to a new system is directly related to the rate that you can get feedback as to the correctness of this functionality. Typed languages use the static typing system to give you a little faster feedback but nothing means anything until the bits are actually moving; thus the repl is the best. It is more fun than diagnosing hindley-milner errors anyway.

OK. The problem with the repl is that you would have to type in your program all the time. So you have files where you can save the information that you can dynamically load into the repl. You can use either (load-file "fname") or you can use slime-compaile-and-load-file from emacs. This is almost as much fun as the repl and a lot more repeatable. The file is still dynamically loaded but at least you can replace sets of functionality at one, easily.

Next we can compile a file and put it into a jar file; or more generally some compilation unit. These are *not* dynamically loadable to my knowledge; thus if you want a program to run for a long time you will load the jar files once and then you are off. Hopefully the program is completely correct and never needs runtime updates.

Jar files are more easily reusable and distributable than stand alone text files. They also protect your IP to some extent although I really don't give a shit about that.

So we have three levels of adding code to the system, in order of ease of mutability. Repl, repl-loaded, and jar'd. So how do we set up a system that makes all this cool?

Well, lets say we have a set of .clj files. We want an ant project that will compile and jar up all of them. I will set this up in a second.

Now, we have to have this jar file path in our classpath somehow. You can either add it via (set-classpath) or you can add it to the java "cp" startup argument.

Now we find some function is wrong in this jar file. Can we dynamically replace this function in a running system? Well, assuming it is reference via 1 level on indirection then yes, we can. This is what the whole environment and "defn" system gets you; I am assuming. But this requires a test.

Basically, if I can create a jar file and then replace some of its functionality in the running system then I am golden; I can have fast, precompiled code *and* I can update the system and try out fixes and new ideas dynamically. I believe, due to the way that clojure's Vars work that I can get this stuff running. But, hope is not a strategy so lets try some of this out.

The first step is to make a jar file out of some clojure file. In the simplest case, I have a clojure file and has a single function in some namespace, and then I jar it up using ant. Next I add it to my classpath and load it into the repl, checking that it works. Finally I replace said function somehow using the repl.

There is one guaranteed caveat that was there in common lisp and it is here too. If you create a closure dynamically this won't be replaced if you load a new file. Thus you need to figure out what level of dynamic functionality you want and avoid closures where necessary, or create closures that immediately call into the namespace vars.

So here we go. Into the lambinator directory and create a subdirectory called src/experiment

OK, copied clojure-contrib's jar file (after actually getting it to produce useful output, you have to tell ant where clojure.jar is). Got the dynjar.clj file compiling into a class directory and having classes shoved into a jar file. At this point I am now as good at clojure as I ever was at java.

I have to update my ~/bin/clojure script to add this jar file to my classpath. I know this will change per project but emacs needs this command; I could perhaps have project-specific .emacs files but I am really not that ambitious.

In my test file, I have one function that returns an integer, and another function that returns the result of the first. Now the test is that if I load the jar file and mess around with the repl I should be able to get the second function to return a new value by replacing the first.

So:
a -> 5
b -> a -> 5

In repl, I will change a to return anything but 5. If this works, then I can dynamically update my running program with new stuff!

Lets see what happens:

Hell happened. Jar hell.

load-file does the equivalent of import and require on all the objects in the file. This was not quite what I expected.

I will update this post later when I have much more information.