Clon version 1.0b19 has just been released. This version comes with a couple of fixes related to error handling and a switch from the GPL to the BSD license. Grab it here.
Didier Verna's scientific blog: Lisp, Emacs, LaTeX and random stuff. |
Wednesday, February 9 2011
By Didier Verna on Wednesday, February 9 2011, 17:02 - Lisp
Clon version 1.0b19 has just been released. This version comes with a couple of fixes related to error handling and a switch from the GPL to the BSD license. Grab it here.
Tuesday, February 8 2011
By Didier Verna on Tuesday, February 8 2011, 10:11 - Lisp
It's been a long time since I last thought about this. Software licensing is probably the most dreadful computer science topic to me. So out of boredom, laziness, but also out of a feeling of adherence to the ideas behind the FSF, I used to blindly stick the GNU GPL to most of my code. A couple of things made me rethink about that passive attitude recently though, notably the fact that the viral nature of the GPL is not something that I always want.
Even in Lisp, you can make a difference between (more-or-less) standalone applications and actual language extensions or libraries that other people might want to use in their own code. While I still think that the GPL is okay for my applications (as Pascal Bourguignon would turn it: I want to see your code too!
), I find it too restrictive for my libraries. I sympathize with the FSF concerns, and we probably need something so idealistic to make the open source culture progress, but I can also understand why people wanting to make a living out of their development need to close their source. For example, I consider the few proprietary Common Lisp vendors as my friends, and I would be glad to see them use my libraries in their own products.
The other thing with the GPL is that it is long, difficult to read and gives me a feeling of over-complication. Probably just me and my allergy to software licensing issues, but on the other hand, it is notorious that the GPL doesn't fit very well with Lisp, for the same reasons that it doesn't fit very well with C++ code constituted mostly of templates. In those environments, the distinction between static or dynamic linking, compile-time or run-time etc. is blurred to a considerable extend, to the point that it doesn't make much sense anymore.
So I was ready to spend, say, 30 minutes looking for a non-viral alternative to the GPL. The first one that comes to mind is of course the LGPL, but it looks even more complicated than the GPL itself, and as misfitted to Lisp as its mother. To the point that some people at Franz Inc. created the Lisp Lesser GNU General Public License
, a.k.a. the LLGPL. This license is in fact constituted with the original LGPL plus a preamble aimed at clarifying or precising some terms used in the license itself, in the context of Lisp. This preamble left me quite sceptical. While it seems to indeed clarify a few points, I think it also adds to the confusion. For instance, it says at some point:
Since Lisp only offers one choice, which is to link the Library into an executable at build time ...
which I think is confusing. What about a Lisp application that dynamically loads an ASDF system. Shouldn't this be considered as very close to dynamic linking? So no. I want something simpler, widely used and accepted. This leaves me basically with MIT, or BSD style, plus Boost that Daniel Herring suggested on comp.lang.lisp.
The MIT license is probably too liberal for me. It is essentially a big disclaimer but that's all. Also, some words are frightening. For instance, what does sublicensing
exactly mean in this context? The Boost one is nice, apart from one thing: it makes acknowledgments optional for binary distribution, and I don't like that. It seems to me that the least you can do when you're using a software component written by someone else is to mention it. It will also help people interested by the feature in question to find its source.
So that leaves me with BSD which I decided to use with a bit of rewording and one specific part removed: the All rights reserved
part. I think it doesn't make any sense since the purpose of the license itself is to grant you some rights.
Eventhough I find that acknowledgments are important, I don't want it to be a burden on the shoulders of my clicents
. One simple way to ease their life is to include the complete license as a header in every single file of your library. The BSD license is so short that it fits nicely in headers (actually, the whole license doesn't take much more space that the GNU GPL header alone). Besides, I use a (never released) XEmacs package for maintaining static file contents automatically, which means that I never have to deal with license contents by hand, not even in new files. For binary distribution, I also include a LICENSE file in my libraries, so that people can just copy it into their own packages and be compliant.
As for Texinfo files, the copyright material needs some adjustment with respect to the original GNU recommendation. Most notably, I don't need the FSF to approve translations of the copyright notice anymore. I find that the following template fits my needs nicely:
@copying @quotation Copyright @copyright{} @value{COPYRIGHT_DATE} Didier Verna Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies. @ignore Permission is granted to process this file through TeX and print the results, provided the printed document carries a copying permission notice identical to this one except for the removal of this paragraph (this paragraph not being relevant to the printed manual). @end ignore Permission is granted to copy and distribute modified versions of this manual under the conditions for verbatim copying, provided also that the section entitled ``Copying'' is included exactly as in the original. Permission is granted to copy and distribute translations of this manual into another language, under the above conditions for modified versions, except that this permission notice may be translated as well. @end quotation @end copying
So that's it. You can expect most of my Lisp libraries to be released under a BSD style license now.
Tuesday, January 25 2011
By Didier Verna on Tuesday, January 25 2011, 12:31 - Lisp
Clon v1.0b18 is now out. Compared to the previous version, the only small bit of change is the fact that the CLISP implementation now depends only optionally on cffi, whereas that dependency was mandatory before. Doing this is actually quite simple but raised the question of optional ASDF system dependencies, a topic on which there's quite a bit to be said.
A bit of context first. Clon has a terminal autodetection feature which allows it to automatically highlight its output on a tty (see --clon-highlight=auto
) and also compute the output line width in case the COLUNMS environment variable is not set. This feature requires an ioctl
call which is beyond Lisp. Since one of my goals is to have as few dependencies as possible (zero being the ideal number), I looked at compiler-specific solutions to this problem. The result is that SBCL, CMU-CL, CCL and ECL provide solutions almost out-of-the-box: SBCL has a grovel contributed module, CMU-CL and CCL already have system bindings and ECL has its own ffi
interface. The ABCL port doesn't support terminal autodetection yet, but that's another (Java) story. The only black sheep in the picture is CLISP which, as far as I can tell, neither has a native binding for ioctl
, nore any built-in grovel facility, hence the need for cffi
.
Previously, my system definition file was like that:
#+clisp (asdf:operate 'asdf:load-op :cffi-grovel) (asdf:defsystem :com.dvlsoft.clon #| ... |# :depends-on (#+clisp :cffi) :components ((:file "package") #+clisp (:module "clisp" :depends-on ("package") :components ((cffi-grovel:grovel-file "constants"))) (module "src" :depends-on ("package" #+clisp "clisp") :components #| ... |#)))
And then later, I had a file with the concerned utility function:
(defun stream-line-width (stream) #+clisp (#| CLISP implementation |#) #| etc. |#)
After that, I started looking at ways to make the dependency on cffi optional. After all, it makes sense to avoid that dependency at the cost of not being able to autodetect terminals.
One thing I fell on almost by accident is ASDF's :weakly-depends-on
keyword. Here is an interesting thread about it. It was an accident because that feature is not documented :-( People, however, have very mitigated feelings about it, as shown in this other thread (syntax and semantics both appear somewhat shaky and underspecified). Besides, the aparent behavior is that if A weakly depends on B and B is not present, then A is operated anyway. So I could probably have my "src" module weakly depend on the "clisp" one, but that doesn't change the fact the "clisp" module should not be included at all if cffi is not there.
In the same thread, somebody proposes another kind of dependency called :contigent-on
which looks closer to what I would need for the "clisp" module: a module the contingency of which is not present will not be processed. This new kind of dependency doesn't seem to be implemented yet, however.
So, all of this seems a bit frightening. Too borderline for my taste. Fortunately, my solution is much simpler, although probably not universal.
This first thing to do is only attempt to load cffi-grovel:
(eval-when (:load-toplevel :execute) #+clisp (handler-case (asdf:operate 'asdf:load-op :cffi-grovel) (asdf:missing-component () (format *error-output* "~ ********************************************************************* * WARNING: ASDF component CFFI-GROVEL not found. * Clon will be loaded without support for terminal autodetection. * See section A.1 of the user manual for more information. *********************************************************************"))))
After that, if loading it were successful, we end up with cffi
as a feature, so we can just conditionalize on that:
(asdf:defsystem :com.dvlsoft.clon #| ... |# :depends-on (#+(and clisp cffi) :cffi) :components ((:file "package") #+(and clisp cffi) (:module "clisp" :depends-on ("package") :serial t :components ((cffi-grovel:grovel-file "constants"))) (module "src" :depends-on ("package" #+(and clisp cffi) "clisp") :components #| ... |#)))
One last problem remains however: what to do in the source code, for the feature-dependent parts. Conditionalizing an ASDF system may indeed lead to trouble: for instance, what would happen if the function stream-line-width
was compiled with cffi around, and later used in a context where it is not? To be on the safe side, what you really need is to dynamically check for the feature. One possible solution is this:
fbound
.In my specific case, what I did was to implement a CLISP-specific version of stream-line-width
, called clisp/stream-line-width
and put it in a new file in the "clisp" module, now defined as follows:
#+(and clisp cffi) (:module "clisp" :depends-on ("package") :serial t :components ((cffi-grovel:grovel-file "constants") (:file "util")))
Then, the original function is rewritten like this:
(defun stream-line-width (stream) #+clisp (when (fboundp 'clisp/stream-line-width) (clisp/stream-line-width stream)) #| etc. |#)
So now I think I'm on the safe side, and Clon has zero mandatory dependency again...
Monday, January 24 2011
By Didier Verna on Monday, January 24 2011, 18:06 - Lisp
I'm happy to announce the release of Clon version 1.0b17. This version notably introduces support for ABCL and expands the documentation in the portability section. Grab it here.
Saturday, January 22 2011
By Didier Verna on Saturday, January 22 2011, 11:04 - Lisp
UPDATE: ABCL now supports the POSIX-compliant use of --
, so I've modified this blog to reflect that. I've also added a final trick to emulate the existence of a proper argv[0]
.
Creating standalone executables in Common Lisp is already a portability minefield, as the standard doesn't specify anything in that regard. The situation is even worse with ABCL because it is a Java-based implementation of Common Lisp, and Java has no notion of standalone executable at all. In this article, we will try to figure out how far we can go in that direction nevertheless.
This work comes from my wish to port Clon to ABCL. In order to make it work, ABCL itself needed some changes, so if you want to try out the code provided below, beware that you need at least revision 13156 from the ABCL Subversion trunk (or version 0.24.0).
Let's start by writing a great Common Lisp application for ABCL, and call it greet.lisp
:
(defun main () (format t "Hello,~{ ~A~}!~%" extensions:*command-line-argument-list*) (extensions:exit :status 0)) (main)
As you guessed, extensions:*command-line-argument-list*
stores the list of command-line options passed to the program. Now, let's try it out:
$ abcl --noinform --noinit --nosystem --load greet.lisp John Doe Hello, John Doe!
So far so good: ABCL-specific options were eaten by ABCL. Not very standalone however. Let's create a wrapper script around that, and call it greet
:
#! /bin/sh exec abcl --noinform --noinit --nosystem --load greet.lisp ${1+$@}
In fact, we have a problem:
$ ./greet --help me Parameters: --help displays this help --noinform suppresses the printing of version info --eval <form> evaluates the <form> before initializing REPL --load <file> loads the file <file> before initializing REPL --load-system-file <file> loads the system file <file> before initializing REPL --batch enables batch mode. The --load, --load-system-file and --eval parameters are handled, and abcl exits without entering REPL --noinit suppresses loading a .abclrc startup file --nosystem suppresses loading the system startup file
Woops! We were expecting to get Hello, --help me!
weren't we? In other words, our own command-line options may conflict with ABCL's which is not nice. One thing we can do with a recent ABCL version is to use --
to separate ABCL's own command-line arguments from ours. Our script becomes:
#! /bin/sh exec abcl --noinform --noinit --nosystem --load greet.lisp -- ${1+$@}
And we now get this:
$ ./greet --help me Hello, --help me!
This is better, although we're not actually very standalone yet: our Lisp file is a separate beast, and the abcl
thing is also just a wrapper script (see below). If we want to be more standalone, we will need to go the Java way.
The first thing to understand is that there's actually no such thing as an ABCL executable. ABCL is packed into a JAR (Java archive) file, a zip file, really, and what the wrapper does is simply to call java -jar /path/to/abcl/abcl.jar options...
The JAR file contains a Main
class with a main
function the purpose of which is to launch a Lisp interpreter. This is done by calling the method Interpreter.createDefaultInstance
, which, amongst other things, handles the command-line options. There is also a method called Interpreter.createInstance
which does not handle the command-line at all, so it seems that this is the one we could use, provided that we find a way to set extensions:*command-line-argument-list*
manually.
The following code does exactly this. Let's call it Greet.java
.
import org.armedbear.lisp.*; public class Greet { public static void main (final String[] argv) { Runnable r = new Runnable () { public void run() { try { LispObject cmdline = Lisp.NIL; for (String arg : argv) cmdline = new Cons (arg, cmdline); cmdline.nreverse (); Lisp._COMMAND_LINE_ARGUMENT_LIST_.setSymbolValue (cmdline); Interpreter.createInstance (); Load.load ("greet.lisp"); } catch (ProcessingTerminated e) { System.exit (e.getStatus ()); } } }; new Thread (null, r, "interpreter", 4194304L).start(); } }
Our Greet class mimics what ABCL's Main class does, only with some tweaks of our own:
extensions:*command-line-argument-list*
manually from the actual argv
passed to the function main
, by consing equivalent Lisp objects and then reversing the list. That way, we completely bypass ABCL's options processing code.Interpreter.createInstance
instead of Interpreter.createDefaultInstance
.Load.load
.Finally, the try/catch
construct intercepts calls to extensions:exit
and effectively leaves the Java environment.
Let's compile this beast now:
$ javac -cp /path/to/abcl/abcl.jar Greet.java
The -cp
option to javac
, the Java compiler, tells it where to find ABCL classes. This command gives us a Greet.class
file, but also a second one named Greet$1.class
. That's because our code creates an "anonymous inner class", whatever that means, in the Java jargon. Now, how do we bypass ABCL's main function and use or own instead? This is actually very simple. The Java command-line lets you specify (read: override) the name of the class in which to find the main
function. So it turns out that we don't even have to modify ABCL itself for this to work. We can simply go:
$ java -cp /path/to/abcl/abcl.jar:. Greet John Doe Hello, John Doe!
and it works. By the way, we can also go:
$ java -cp /path/to/abcl/abcl.jar:. Greet --help me Hello, --help me!
and see that it still works. Since ABCL doesn't process its command-line anymore, there's no question about --
or whatever. Note that this time, we need to add the current directory to the class path option in order for Java to find the files Greet.class
and Greet$1.class
. Note also that with our custom interepreter, there's no more need for all those "standalone" options like --noinit
, --noinform
etc. Wrapping this in a shell script is left as an exercise, although not a difficult one.
Still, we're not quite standalone yet. For example, it's not cool to have these two Greet
class files floating around like this. All other class files are in the ABCL JAR file, so we can think of adding ours in too. If we want to do that, we also need to state that the main class in the JAR file should be our Greet
class, not ABCL's Main
one. This can be done by modifying the so-called JAR manifest.
Let's create a manifest file first and call it Greet.txt
:
Main-Class: Greet
It says what it says: our application's Main
class (where the main
function is) is Greet
. Now let's make a copy of abcl.jar
and add our class file to it:
$ cp /path/to/abcl/abcl.jar greet.jar $ jar umf Greet.txt greet.jar Greet.class Greet$1.class Jan 22, 2011 5:50:32 PM java.util.jar.Attributes read WARNING: Duplicate name in Manifest: Main-Class. Ensure that the manifest does not have duplicate entries, and that blank lines separate individual sections in both your manifest and in the META-INF/MANIFEST.MF entry in the jar file.
The u
stands for "update", the m
stands for "manifest" and the f
stands for target (JAR) "file". The manifest and JAR files must appear in the same order as their corresponding options. Don't worry about the warning. That's because we're changing the Main
class (ABCL had one already).
Let's try it out:
$ java -jar greet.jar John Doe Hello, John Doe!
Cool. But what about the Lisp file? It's still floating around. As a matter of fact, the JAR file may contain anything you want, not just class files. ABCL itself has a bunch of lisp files in its JAR file already. So we can put it in there just as well:
$ jar uf greet.jar greet.lisp
The last question is: how to have ABCL load a lisp file that's in its own JAR? The answer is: dig into the code and figure out that there is a method called Load.loadSystemFile
which does just that. ABCL uses it to load its own boot.lisp
for instance. We hence need to modify our main
function and replace the line that calls Load.load
with this one:
Load.loadSystemFile ("/greet.lisp", false, false, false);
Note the leading slash. It is needed because we have stored greet.lisp
at the root of the JAR file. Let's update our JAR file once again:
$ javac -cp /path/to/abcl/abcl.jar Greet.java $ jar uf greet.jar Greet.class Greet$1.class
And voilà! What you get this time is your application completely packed up in the JAR file. You can move this file around, distribute it... everything's inside.
Here's a final additional trick and I think we're done. As you know, ABCL stores the command-line argument list in extensions:*command-line-argument-list*
. One potential problem is that there's no notion of a program name (argv[0]
) in ABCL (the program is in fact always Java). Since we're going standalone, however, maybe we would like to have that. Something like, say, a variable named extensions:*argv0*
. So here's a way to do it (Clon does this as part of its built-in dump facility): in the Greet
class, replace the line
Interpreter.createInstance ();
with this:
Interpreter interpreter = Interpreter.createInstance (); interpreter.eval ("(defvar extensions::*argv0* \"greet\")"); interpreter.eval ("(export 'extensions::*argv0* 'extensions)");
So that's it I guess. It seems to me that this is as far as you can go in the direction of standalone ABCL executables. Of course, this example is overly simple. We didn't try to incorporate full ASDF systems in the archive, and stuff. But you get the idea... I've attached my custom interpreter class below if you want to play with it.
Have fun!
Thursday, December 16 2010
By Didier Verna on Thursday, December 16 2010, 16:30 - LaTeX
Tuesday, December 14 2010
By Didier Verna on Tuesday, December 14 2010, 16:47 - Lisp
By Didier Verna on Tuesday, December 14 2010, 15:24 - LaTeX
Monday, December 6 2010
By Didier Verna on Monday, December 6 2010, 09:20 - Lisp
Friday, December 3 2010
By Didier Verna on Friday, December 3 2010, 22:00 - LaTeX
Wednesday, December 1 2010
By Didier Verna on Wednesday, December 1 2010, 19:50 - LaTeX
I really enjoy Didier Verna's paper (pp. 162-172). His analogies between LaTeX and microbiology is truly exciting! Being neither a TeXnician nor a (micro) biologist, the paper gives me more insight about LaTeX while at the same time giving me a glimpse to a world beyond my narrow field of knowledge. Please do extend my compliments to the author.
Tuesday, November 30 2010
By Didier Verna on Tuesday, November 30 2010, 13:27 - Lisp
Thursday, November 18 2010
By Didier Verna on Thursday, November 18 2010, 08:20 - Lisp
Thursday, November 11 2010
By Didier Verna on Thursday, November 11 2010, 20:24 - Lisp
Tuesday, November 9 2010
By Didier Verna on Tuesday, November 9 2010, 08:27 - Lisp
Tuesday, October 5 2010
By Didier Verna on Tuesday, October 5 2010, 12:14 - LaTeX
Tuesday, September 21 2010
By Didier Verna on Tuesday, September 21 2010, 08:21 - Lisp
Monday, September 13 2010
By Didier Verna on Monday, September 13 2010, 13:37 - Lisp
Tuesday, August 31 2010
By Didier Verna on Tuesday, August 31 2010, 14:31 - Lisp
Thursday, June 24 2010
By Didier Verna on Thursday, June 24 2010, 15:56 - Miscellaneous
« previous entries - page 6 of 9 - next entries »