International Lisp Conference 2009
Lisp: The Next 50 Years
NavigationUser login |
Proposal: Lisp distributionsThe most famous are Linux distributions such as Slackware, Debian, RedHat, and SuSE. (La)TeX has TeXLive, MiKTeX, teTeX, etc. ActiveState makes popular distributions for Perl, Python, and Tcl. The list goes on. The beauty of these distributions is that they simplify life for the end user without requiring standardization. Each distribution can focus on the needs of a particular community. They can all push package-specific improvements back to common upstream repositories so code changes benefit everyone. Common Lisp does not need "yet another source repository". CL doesn't need "yet another CPAN" which automates the piecewise downloading of a system. CL needs distributions of packages that have been engineered and tested to function as a whole system. Saying "I run Slackware Linux 12.2" is very powerful. It macroexpands into "I run kernel 2.6.27.7, glibc 2.7, xorg 1.4.2, libpng 1.2.32, emacs 22.3, firefox 3.0.7, ..." This is a power not offered by current CL systems such as asdf-install or clbuild, and it is outside the scope of projects such as cliki and common-lisp.net.
|
Active forum topicsNew forum topicsSearch |
Reuse OS Distribution tools!
* If you are going to do the same job as the OS Distribution tools, why not just reuse them? i.e. automatically produce rpm's for RedHat, deb's for Debian, whatever for Windows, etc.
* If you are going to roll your own, at the very least, do the packaging with a purely functional approach, as in NixOS. All the other systems force you to BREAK YOUR INSTALLATION when you try to upgrade with NO HOPE OF GOING BACK. With a purely functional approach, you always keep all your previous software versions running at all times, until you explicitly mark them as not wanted anymore (and they get garbage-collected).
Cross platform challenges
Hi Fare,
I don't disagree in theory, however there isn't an RPM-like distribution model for Windows or Mac (I believe) that is built in so you need to roll your own. If you use the unix tools, then you need to support the different variants (deb and rpm) which from what I hear has a surprisingly high overhead. I think Daniel made this same arguments for the positions he has taken.
I do like the idea of parallel versions running so we can run multiple libs in parallel and not break packages on upgrade! To accomplish this for common lisp, IMHO, we need an asdf/mudballs system definition model that automates and standardizes the naming of packages according to metadata in the system definition:
1) System definition contains package dependencies + versioning info - default is latest
2) defpackage :use and :import/shadow forms should not be in the usual package.lisp file, but be automatically defined from the system definition. This allows for automated package import renaming based on the library version. Also allows for (:depends (:foo > "2.5") (:bar = "0.1.4") (:baz = :latest)).
There should be an option to define exports there too, but sometimes libraries might want to do explicit per-file or per-function exports which should be OK too.
3) This should be optional so we can still distribute software that doesn't depend on this or currently plays horrible games with packages (like LISA, Maxima, etc)
4) We might also want explicitly declared *features* dependencies on a per-lisp platform basis. For example :sb-unicode or :threads.
This might already exist ;)
Hi everybody,
what you propose (as far as I understand) might already exist, however, is not widely known and not well documented ;)
The basis is (as you proposed) a linux distribution, gentoo linux. The particular advantages of gentoo are (1) it's a source based distribution and thus ideally suited to distribute source code packages and (2) it's more than 'yet another linux distribution' but can be used to manage software on top of other operating systems such as freebsd, mac os x, solaris and even windows.
Gentoo is quite lisp friendly and ships ~200 (somewhat dated) lisp packages out of the box. In addition there is the gentoo lisp project with another ~200 (quite up to date) packages.
The packaging recipes are written as simple shell scripts (see examples for hunchentoot-1.0, weblocks-SVN, cffi-0.10.4, sbcl-GIT or fare-matcher-20070824 ;). Writing those ebuilds generally takes just a few minutes. A simple package update is done by copying the file foo-1.2.ebuild to foo-1.3.ebuild. Eventually ebuilds could even be generated (semi-)automatically. Dependency resolution is very powerful. It's possible to install several versions of one package in parallel (as slots). There is even a way to define optional features for sets of packages (known as use flags).
While one could argue that such a system is not particularly lispy the advantages are (1) it just works and (2) packages can depend on everything that can be installed on a computer (e.g. mod-lisp, databases or image libs) -- with full dependency resolution included - all the way up to the c compiler.
What do you think? Couldn't that make for an 'All-Batteries-Included-Lisp-Meta-Distribution'?
Using Gentoo
That's very interesting. I had no idea that the well-respected Gentoo distribution system could apply to other operating systems than Linux.
However, I rather doubt that they could do all the things on my wish-list for LAIR (see the topic about that), such as the fancy metadata that I'd like to see.
It's worth keeping this in mind as a fallback in case the (rather ambitious) proposal I put forth turns out to not get done. It's a lot better to have something that doesn't do everything one might like than to have nothing at all. Thanks very much for the idea and information!
Gentoo is not the answer to my question.
I was a big fan of FreeBSD "back in the day"; the only reason I left was a lack of hardware support (sata and video card). Major bummer. IIRC, Gentoo started out as a "BSD package system on linux".
Anyway, Gentoo's not the thing for me on linux, and its not quite what I want for CL.
But please, if you think Gentoo is the answer, tell people how to use it everywhere. CL's "library problem" is largely a lack of install ease and marketing. Who knows, if the Gentoo CL system works good enough, I might be able to abandon my own design. :)
A nice analogy
I think that borrowing the distribution analogy from the GNU Linux world is excellent. LispBox alone was a big improvement for introducing new users to the language. I think that community standardization on a system build and distribution model is key. This would allow different distributions to share a common infrastructure (like RPMs) to avoid extra burden on library developers.
I see packaging up lisp libraries into Debian packages, for example, as another Lisp distribution model that would be complementary to Herring's approach but could share the common lisp distribution model.
Currently all the major libraries support asdf, so we see evidence that the community can at least agree on standards occasionally. ASDF may not be the right solution of course. Both mudballs and XCVB appears to be a nice improvements over asdf. So long as the model selected for builds is close enough to ASDF, I suspect that a few volunteer developers could provide patches to all the major libraries to support the new model in a matter of weeks. I'd volunteer to take on the 5 or 6 libraries I know well, for instance.
The big thing we would need out of library maintainers is producing better documentation as well as making all documentation compatible with the distribution model so it is possible to build documentation portals. See my rant under the 'Lisp is so Bad' topic.
Another facility that perhaps the ALU can host is a gateway service, such as that described by Herring in his lightening talk, which allows library developers to do releases via VCS tags as well as tarballs and provides a standard baseline for package versions.