This is a read-only archive!

Lisp ramblings

Two posts in one, just because I can.

Rambling #1: One thing that makes a language like Ruby easy to read is that foo.bar always has the same meaning: object.method.

In Common Lisp you don't have that. One of CL's strengths is that everything looks the same, it's all s-expressions. However a cost of this is that you can't tell just by looking whether (foo bar) is:

  • A special operator foo, that does something with bar (maybe assigning it a value, maybe conditionally evaluating it, who knows).
  • A system-defined function call foo, where bar is evaluated and the result is passed in as an argument, and which you can't redefine because it's in the COMMON-LISP system package.
  • A user-defined function call foo, where bar is evaluated and the result is passed in as an argument, and which you can re-define if you want.
  • A method call foo, where bar is possibly an object used to dispatch the method call (or maybe not).
  • A macro foo, which can produce code that does practically anything, or nothing at all, with bar.
  • Other?

Along the same lines, even if you know for sure that (foo bar baz) is a method call, you can't tell by looking whether bar, or baz, or both, are objects which determine how the method call is dispatched. You have to look it up in documentation or inspect the method somehow. As far as I know. (EDIT: Slime has lots of tools for doing this kind of inspection., as was pointed out to me by Ivar.)

This is a good feature of Common Lisp, in terms of flexibility and power; multimethods are great, the consistency and lack of syntax and precedence is great in many ways. But it can be a bad thing in terms of easy readability. It's a bit of a tradeoff.

Ruby code like arr.reverse.sort.join(',') just flows right along. And it's highly predictable. I know reverse is a method call, and it's defined in whatever class arr is an instance of (or a mixin of that class, or a meta-class I guess), and it returns something that has a sort method. And I can redefine any of those things.

Common Lisp non-working pseudo-code: (join (sort (reverse arr)) ","). Not as much fun to read, and oops, in this case SORT is a system-defined standard function, so if I want to write this I'll have to go with MY-SORT or SORT2 to avoid collisions (which you see a lot of in Lisp examples and literature).

Rambling #2: Way back in 2006, I decided to learn Common Lisp because I'd heard such nice things about it. I ended up abandoning that project very quickly (only to try again late in 2007, and stick with it this time). One reason (among many many others) that I gave up was the fact that it took me a good hour or two even to figure out how/where to download and install Common Lisp.

I remember going to a terminal and searching the Portage tree for packages matching the name "common lisp". Zero results. OK, how about just "lisp". One result I got back was clisp, which at that time, based on the name, I assumed was "just another name for the Common Lisp interpreter". However a search of the Gentoo forums revealed that a lot of people were using SBCL, or CMUCL, or many others. And those things were "Common Lisp" too? How confusing.

Little did I know that there is no THE Common Lisp. There are a bunch of implementations of the Common Lisp standard (or varying parts of the standard). To a newbie, this is somewhat confusing. Which one should you pick? What the heck is the difference? If it's a standard, does it even matter which you pick? So before you can even start writing code, you get to treat yourself to a history lesson to learn why there are all these different implementations, and then go on a fact-finding mission to try to find an implementation that's right for you.

It wouldn't matter, if the standard was complete enough that it covered all of what you'd want to do. But the standard is old, and arguably out-of-date. And it doesn't cover things like threads, sockets, POSIX utilities, etc. that you're really likely to use. And if something isn't in the standard, you can bet good money that every implementation does that thing in its own way, or not at all. A lot of sufficiently complex code in Common Lisp will demand that you go to great lengths to get it to run on different implementations, judging by some of the library code I've read.

Consider CL-FAD. A function DELETE-DIRECTORY-AND-FILES is used to (you guessed it) recursively delete a directory and its files. Here are the guts of that function:

(cond ((directory-pathname-p file)
      #+:lispworks (lw:delete-directory file)
      #+:cmu (multiple-value-bind (ok err-number)
                 (unix:unix-rmdir (namestring (truename file)))
               (unless ok
                 (error "Error number ~A when trying to delete ~A"
                        err-number file)))
      #+:scl (multiple-value-bind (ok errno)
                 (unix:unix-rmdir (ext:unix-namestring (truename file)))
               (unless ok
                 (error "~@<Error deleting ~S: ~A~@:>"
                        file (unix:get-unix-error-msg errno))))
      #+:sbcl (sb-posix:rmdir file)
      #+:clisp (ext:delete-dir file)
      #+:openmcl (ccl:delete-directory file)
      #+:cormanlisp (win32:delete-directory file)
      #+:ecl (si:rmdir file)
      #+(or :abcl :digitool) (delete-file file))
     (t (delete-file file)))

What's going on there? All those #+ and #- lines are kind of like the Common Lisp equivalent of C preprocessor macros. #+ and #- hide (or include) a line of code from the view of the compiler, depending on which implementation you're using. Ugh.

So in this case, you have what is a pretty simple function (1-5 lines of code, in each implementation), but you get to have the fun of writing it TEN TIMES, all woven together like spaghetti. There are many other such compatibility layers.

Kudos to people who go to the effort of writing portable code like this, but I don't see why it should be necessary. This strikes me as a senseless and wasteful duplication of effort. Not just on the part of the people writing code to run on all these implementations, but also on the part of the people working on the implementations themselves. Why not join forces and make one big mature official Common Lisp for everyone to use? What benefit is there to having so many implementations? Does the benefit outweigh the amount of work someone has to do to write portable code to share with the community? Maybe there's a good reason for it that I'm not aware of. But it seems to me like it defeats the purpose of having a standard in the first place.

On the other hand if I want to write some Perl, I do an emerge perl (substituting your package manager of choice) and then I have Perl. The same exact Perl (barring version differences, and slight differences between Perls on different OSes) that every other Perler in the world has. If I write a Perl script, any other person with Perl could reasonably expect to be able to run it. Same with PHP, Python, Ruby, and many other languages. (People are working on separate interpreters for Ruby, e.g. JRuby but from what I know, they're going to great lengths to make it run EXACTLY like the "official" Ruby, even down to copying bugs to maintain compatibility.)

I know now that there are things like Lisp in a Box that give you an easily-installable Common Lisp environment. Which is really nice for learning purposes. I wish I'd have found it back in 2006 (if it existed at that time). But it only delays the mess you're going to have to handle someday if you want to write portable code, or if you want to use anything other than CLISP or Allegro.

I have an urge to start porting some of my favorite Ruby gems to Common Lisp. But I am not going to install 10 CL implementations just so I can re-write my code 10 times to get it working on all the Lisps in the world. Very likely my code would be ruthlessly SBCL-specific.

(This is one reason I'm kind of hoping Arc takes off. Hopefully there would only be one Arc implementation.)

February 14, 2008 @ 4:09 PM PST
Cateogory: Programming
Tags: Lisp, Perl, Ruby, Arc, Rant

14 Comments

Jakub
Quoth Jakub on February 14, 2008 @ 10:30 PM PST

>Why not join forces and make one big mature official >Common Lisp for everyone to use?

For the same reasons there is no "one big mature official" Operating System "for everyone to use"? How do you force commercial implementors to open their code, or all open implementations to fold their code into commercial one, and how would that work exactly? Do you also want only one official Linux distro? And which one would that be?

Every Common Lisp implementation is different, serves somewhat different purposes, and so on. Just pick one, perhaps using compatibility layers where applicable, that fits your purposes best.

Leonardo Varuzza
Quoth Leonardo Varuzza on February 14, 2008 @ 10:44 PM PST

About the fragmentation of lisp implementations: Lisp don't have a benevolent dictator to make all the implementations agree in a single set of extensions to common lisp. In the past, the only organization strong enough to make a patronization in lisp is ARPA, but it is not interested in lisp any more.

I think with the maturing of free implementations, one will start to be a standard the facto, and SBCL seen to be the best candidate, only missing a stable windows version to global domination).

numerodix
Quoth numerodix on February 15, 2008 @ 12:59 AM PST

I see the word "common" is somewhat of a misnomer. "not common" would be more to the point.

Brian
Quoth Brian on February 15, 2008 @ 4:34 AM PST

Jakub: It makes sense to have different Linux distros because people do have different needs. Some people want a nice easy-to-use desktop machine, some people want a rock-hard secure server, some people need Linux to fit on a cell phone.

But Common Lisp is an all-purpose tool. I thought one of the whole points of Common Lisp as the "programmable programming language" was that you can customize it however you want. I'd rather there be one implementation that everyone used and tweaked rather than 10 doing the SAME THING for the most part, yet being just different enough to make a hassle for everyone.

I wouldn't want to force everyone to cooperate. I'm sure people have their reasons for not doing so. But it seems like a waste.

Leonardo Varuzza: I hope so. SBCL does seem to be a bit of a leader and I'm happy to consider it a de facto standard myself.

numerodix: Before Common Lisp, things were even worse, from what I read of history. The standard does cover a lot, but when it was written in the 1980s they had much different systems and needs than we do now sadly.

Jakub
Quoth Jakub on February 15, 2008 @ 8:19 AM PST

The language is all purpose, but the implementations are not. For example, light interpreter (CLISP), heavy compiler to native code (SBCL) and embeddable compile-to-C (ECL) serve very different purposes. I don't know what exactly are the differences between SBCL and CMUCL, so I just ignore the latter.

I agree that it would be helpful if everyone agreed on common interface to some extensions, like networking or threading, but then again most other language standards don't cover those either, and compatibility libraries are good enough usually. Or one can just ignore that and write implementation specific code.

she
Quoth she on February 15, 2008 @ 9:53 AM PST

I agree that it would be helpful if everyone agreed on common interface to some extensions, like networking or threading, but then again most other language standards don?t cover those either, and compatibility libraries are good enough usually. Or one can just ignore that and write implementation specific code.

One already has to do this, in other languages too. Ruby has one big advantage: It has matz.

Also, ruby is not influenced as heavily as lisp. But the biggest point, which is the common factor, is that languages such as perl or python are kinda STRAIGHTFORWARD to learn for anyone coming and diving into this world. Perl sucks as far as syntax beauty is concerned so I predict python and ruby will win more and more users over the time. How can lisp compete here? The smarter ones will use scheme or haskell anyway, and lisp can never hope to replace C or any other of the bigger languages, so I feel the strategy for lisp will ultimately fail. Lets look at the situation in two years.

dudestuffs
Quoth dudestuffs on February 15, 2008 @ 2:33 PM PST

Part of this many-implementations mess is the same in the C world, where you have similar choices of compilers (gcc, msvc, intel) and lots of compatibility issues. Basically all those "./configure" scripts are a huge hack to deal with that kind of crap. You're still 100% right that it sucks nuts.

Your discussion of syntax seems to be missing a crucial advantage of s-expressions, code generation.

You can also define your own package so that you can use your own "sort" function, etc., but I get that you wouldn't do that in tutorials.

FWIW.

Galo
Quoth Galo on February 15, 2008 @ 3:04 PM PST

numerodix: The 'Common' does not come from 'common implementation'. The name translates as 'A Lisp language for all lispers'.

she: The smarter ones will not pick on lack of superfluos bullshit and choose to learn/use CL despite it's troubles. No one 'owns' CL like they 'own' Perl, Ruby, Python or Arc (all these languages have their Benevolent Dictator, whatever the hell that is). That's why they are 'straight forward': one guy tells everyone how to do it. That works good if you just started a language, or if you're the original implementor of the original implementation. But if your language was created and approves by a group of people, you're out of luck. Your caracterization of CL competing against Scheme, Haskell and C is stupid.

If not having a common implementation or not having a package with the same name of the language puts you off, CL is not for you. Script kiddies need common implementations so they can copy paste their stuff off the Internet. Programmers learn to deal with it. They don't pretend it isn't there, but they deal with it. If the trade-off is too big, just don't use CL. Just don't.

The day CL has a Dictator, is the day I stop using it.

Ivar Refsdal
Quoth Ivar Refsdal on February 16, 2008 @ 7:45 PM PST

"Along the same lines, even if you know for sure that (foo bar baz) is a method call, you can't tell by looking whether bar, or baz, or both, are objects which determine how the method call is dispatched. You have to look it up in documentation or inspect the method somehow. As far as I know."

And you have a lot of good tools for figuring this out, no?

slime-edit-definition (This one works well for multimethods*, classes, macros and probably more. You will also need it's companion slime-pop-find-definition-stack) slime-inspect slime-macroexpand-1 slime-hyperspec-lookup slime-documentation slime-list-callers slime-list-callees (I use the two first the most, the rest not so much.)

Brian: You've seen the Slime video, so why aren't you mentioning this?

  • I'm not too into the "lingo" so I'm assuming you're meaning multiple defmethods.

Galo: How does anyone 'own' Perl, Ruby, etc? Is there anything stopping anyone from creating their own forks of those languages? (Given that this is a seemingly good thing, according to you.) License?

Brian
Quoth Brian on February 16, 2008 @ 8:12 PM PST

Ivar: My Slime-fu is not that strong yet. Aside from macroexpand and right-click-inspect, I actually don't make much use of those commands you mention. I'm still learning. I've made an edit to my post.

However my real point was only that in a language like Ruby, not even those things are as necessary as in CL. In Ruby it's all objects and methods. We have good tools in CL, but those tools are solving a problem that isn't present in some other languages.

This doesn't condemn CL by any means, it's only an observation. Every language has pluses and minuses. Readability-at-a-glance is one minus for Common Lisp, among its many pluses.

(By multimethod I mean a method which has multiple dispatch. As opposed to single dispatch in e.g. Ruby/Java/etc., where every method belongs to one single class. So CL's defmethod, yeah.)

Ivar Refsdal
Quoth Ivar Refsdal on February 16, 2008 @ 8:54 PM PST

Brian: Allright, allright. That's nice. The two first are great, you should check them out. (And it's convenient to use the built in hyperspec lookup when you need to lookup something in the definition.) There's also a reference for the Slime video for easy access. slime-inspect is very nice for inspecting "live" objects to see what they are like. Thanks for the other link, I'll check that out sometime later.

"However my real point was only that in a language like Ruby, not even those things are as necessary as in CL. In Ruby it?s all objects and methods." Yes, I got your point, and I agree. I'm a bit confused myself if I should be writing 'something, :something or just something. (So I just usually look at what someone else wrote or what I wrote earlier). The asdf:load-op seems to be happy with both ' and :.

Brian
Quoth Brian on February 17, 2008 @ 5:52 AM PST

Yeah, don't forget "something" too. There are functions to convert back and forth between most of those, and if a function accepts two kinds ('package and :package) it's probably doing that conversion itself. I was confused by that myself for a while also. Still am a bit, at times.

Michael Jung
Quoth Michael Jung on February 18, 2008 @ 4:58 PM PST

I'm at the very beginning of learning lisp and have similar problems. Of SLIME/Emacs I use only 5% of their features yet as I'm spoiled by 20 years development on M$-DOS/M$ Windows but I try hard and learn.

The fascinating thing about Lisp is that I can write my own DSL just on top of nearly any implementation with some implementation specific adjustment. And I can write wrapper which make the language more read- and useable for me but not necessarily for anybody else. And not to forget about the power of macros.

Any languages out there serves one or many purposes, at least enjoy the creator :-) For my own I use a language which is best suited to a given problem even if I have to learn a new one. And any language out there has it's pro's and contra's. None is perfect, and none will be as these are developed by imperfect human beeings. That's life and life is fascinating.

Just my 2 cents.

Robert Goldman
Quoth Robert Goldman on February 24, 2008 @ 12:47 AM PST

You write "Common Lisp non-working pseudo-code: (join (sort (reverse arr)) ","). Not as much fun to read, and oops, in this case SORT is a system-defined standard function, so if I want to write this I'll have to go with MY-SORT or SORT2 to avoid collisions (which you see a lot of in Lisp examples and literature)."

You only see my-sort or sort2 in the examples and literature because the examples and literature tend to be short on examples that involve packages (common lisp namespaces).

If you're working on something serious, you can work in a package in which the normal definition of SORT is shadowed by your own definition. On the other hand, if your code is throwaway and you're just playing around, then calling your function my-sort or sort2 is not a high price to pay for having sort provided by the language.