The Return of Segfaults
By Didier Verna on Wednesday, August 5 2015, 17:57 - Lisp - Permalink
I was watching the discussion between Gilad Bracha and Matthias Felleisen on gradual typing this afternoon (it's available on YouTube). This was the last event at the STOP workshop, part of ECOOP 2015 in Prague. I couldn't attend it because I was somewhere else (Curry On) at the time. The discussion is interesting, but if you go all the way, in the last 10 minutes or so, you will notice that Matthias seems to be completely obsessed with what he calls the "Return of the SegFaults".
Basically, the point is the following. Mathias dislikes the optional types in Common Lisp because it's opening the door to unsafety. Initially, any dynamic language has a sound type system (all type errors are caught; at run-time, yes, but they are caught). As soon as you introduce an optional type system ala Lisp, the compiler implementors are "pushed" to use the provided type information for optimization, hence weakening the type system and breaking soundness. It's the "return of segfauts".
Of course, I agree with that, at least on the principle. Yes, Common Lisp's weak, optional type system is an antiquated thing. However, it seems to me that Matthias is forgetting two important things on the practical level:
- by default in many implementations that I know, if not all of them, introducing type declarations doesn't actually break the soundness of the type system but leads to even more type checking. For example,
(defun plus (a b) (+ a b))
works on every possible numerical value, but add(declare (type fixnum a b))
in there and it will suddenly stop working on anything else but integers. It's only if you require the compiler to optimize forspeed
at the expense ofsafety
that you effectively weaken your type system. - In practice, the risk of re-introducing segfaults in your application may be mitigated by the interactive aspect of the development (TDD made easier, simultaneous write / compile / run / test / debug phases etc.).
So my impression is that Matthias is largely exaggerating the problem, but I'm not really qualified to tell. That's why I would like to know from you guys, working with Lisp in the industry, writing large applications, lots of type annotations, and (declaim (optimize (speed 3) (safety 0) (debug 0))
(EDIT: that's exaggerated of course, I really mean breaking safety for performance reasons): how much of a problem the "return of the segfaults" really is in practice ?
As a side note, this reminds me of the dynamic vs. lexical scoping debate. Many people were and still are strongly opinionated against dynamic scoping by default. Of course, I too, at least in principle. But how dangerous dynamic scoping really is in practice (EDIT: I'm not talking about expressiveness, e.g. closures, here. Only unsafety.)? What I can tell you is that in the 15 years I was actively maintaining the XEmacs codebase, I may have run into name clashes due to dynamic scoping... twice.
Comments
Safety 0 in production, globally? No! Why? :-) Even in specific functions, it's probably a bad idea. There are always bugs and flawed assumptions... I've seen some very strange results happening due to declaims like that, notably treating NIL as though it were some random structure and trying grab a value at some offset.
Lexical vs dynamic scoping is not just about name clashes, it's about closures, isn't it? Closures are what I miss when I only have dynamic scoping. (Thankfully Emacs has had lexical scoping for a while now. ;-))
Granted, (safety 0) globally is abusive. I was really talking about the actual risk of breaking type safety with optimization in production code.
As for scoping, you're right, but I was not talking about the expressiveness of the paradigms here, only the dangers, and I think the biggest danger with dynamic scoping is name clashes.
Thanks for your comments, BTW, I'm rephrasing the ambiguous parts.
Another supposedly dangerous thing are non-hygienic macros. But again, in practice, I have never had a symbol-name clash. Just use gensym and you are done.
In speed-critical functions: (declaim (optimize (speed 3) (safety 0) (debug 0)) in production, (declaim (optimize (speed 0) (safety 0) (debug 3)) otherwise...
> Yes, Common Lisp's weak, optional type system is an antiquated thing.
What would be an improvement over it? is that what Racket is doing? I am really curious if there is any research happening on this.
My 2 cents, the future is in the ability to mix dynamic types and strong+explicit+static ones. The whole area of research around "gradual typing" is going in that direction.
More generally, I've always believed that general purpose programming languages should always strive for being as multi-paradigm as possible, which in fact they do even against their original goals or designers, because real life is just like that. In history, I can see 3 major areas of paradigm mutual exclusion: imperative/procedural vs. functional, dynamic vs. static types and explicit vs. automatic memory management.
The first war is over: everybody now understands the importance of the functional paradigm (e.g. both C++ and Java have lambdas).
The second war is almost over: more and more languages are attempting to mix static and dynamic types (e.g. Racket but also F# etc.).
So I predict (ahem) that one of the next big challenges for future programming languages will be to provide the ability to mix garbage collection and explicit memory management.
The door to no safety has always been open with Common Lisp. See CLHS 1.4.4.3. An implementation is not required to detect invalid types at run time unless the operator is specified to do so. A great example is (aref v -1).