Nice little trick du jour
By Didier Verna on Wednesday, June 26 2013, 16:24 - Lisp - Permalink
This morning, I came up with a nice little trick which made my day. Perhaps this is already well known or even idiomatic, but in case it is not, it goes like this.
Sometimes, I have to read a whole Lisp file containing more than just one toplevel form and get the contents as an unprocessed list of expressions. This happens for instance when loading a DSL program that needs post-treatment.
Usually, I end up doing something like this:
(with-open-file (stream filename) (loop :for expr = (read stream nil :eof) :if (eq expr :eof) :return exprs :else :collect expr :into exprs)))
But for some reason, this has always felt clunky to me. I mean, instead of ogling the family jewels of this poor shy file, I should be able to read it as a whole (hint: read-delimited-list) and preserve its intimacy.
And then I realized that the only thing that prevents me from using read-delimited-list is the lack of an ending parenthesis. But as always, Common Lisp immediately comes to the rescue, this time with its rich streams API. The idea is that you can create a string stream that just contains the closing parenthesis, and concatenate this stream at the end of the file one. I can now do something like this instead:
(with-open-file (stream filename) (read-delimited-list #\) (make-concatenated-stream stream (make-string-input-stream ")"))))
Not only it is shorter, it also looks much more elegant to me.
Comments
One possible inconvenient is that an erroneous file with an extra ")" somewhere in the middle isn't detected as erroneous; you just stop reading at that point instead. For example, a file containing:
(foo bar) ) (baz)
is read as if it was just:
(foo bar)
Maybe using some obscure control character rather than #\) as the delimiter would mitigate the problem? Feels a bit hacky though.
Or, since ASDF3, just use
(uiop:read-file-forms filename)
This will work ok as long as :eof is not likely to be a form in the file. The usual trick for avoiding it is to use the stream object as the eof marker, e.g. (read stream nil stream) and (eq expr stream).
The loop version can be made a bit nicer:
(loop
:for expr = (read stream nil stream)
:until (eq expr stream)
:collect expr)
That loop is a bit strange, it'd be less clunky with
(loop ... until (eq expr eof-marker) collect expr)
Here it is in terms of the SERIES package: (collect (scan-file filename))
Here's a way that avoids problems with reading atoms or dotted lists and just looks more sensible:
(defun list-stream (stream)
(make-concatenated-stream
(make-string-input-stream "(")
stream
(make-string-input-stream ")")))
(defun read-forms (filename)
(with-open-file (stream filename)
(read (list-stream stream))))
Of course, you're right. This is even better!