I thought about suggesting special variables when pg asked for suggestions, but I plan on sticking with CL so I thought it inappropriate to... open my big yap? :)
I have heard that even universities are held hostage by students who view college as merely job training, demanding courses in the languages business cannot see past.
Seems to me the unis should fight that fight, but there are a lot of unis in the US, they have to think about marketing, too.
Perhaps as the IT job market continues to shrink the masses will move on to something else and let the computer science departments go back to teaching algorithms.
I haven't met many students that are set on learning Java and only Java (or insert other popular language of choice), mostly its just that they don't know anything better is out there. I have, however, met professors who hold very strong, and wildly incorrect views on Lisp (many of which probably haven't been accurate for 20 years or more). But how is a college freshman supposed to tell a guy with a PhD he is completely incorrect?
I've heard claims that they'll let me program in whatever language I want in upper division, but I have to wonder if that is true or not.
IIRC, Brown has two introductory computer science tracks -- one that starts with Scheme and the other with Java, with the Scheme course recommended for majors. They've got one of the PLT Scheme implementors on faculty, too, so some of the advanced courses are Scheme-based, as well.
I should add that dsb and thus defun support optional and keyword args at the same time only because it is possible; I cannot imagine it ever being sensible. :)
Very nice work on supporting both opt and key args.
Although if you had something like this:
(defun fn (a &o (b 'b) (c 'c) &k (d 'd))
with a usage like this:
(fn 1 'd 'e)
... how would you know whether:
1) 'd is the value of the first opt arg and 'e is the value of the second (the key arg unsupplied)
or
2) 'd is the key for the key arg, and 'e is its supplied value (the 2 opt args unsupplied)
?
Maybe I'm missing something here, but it seems to me that unless you have special syntax for keywords, you will get into trouble.
And if you have to introduce special syntax for keys anyway, it is just as well to make every single argument keyable on its symbol (even vanilla ones), and just worry about combining &o and &rest (which should then be doable).
"with a usage like this: (fn 1 'd 'e) how would you know..."
The interpretation is that d and e are the two optional args, so any caller wanting to supply a keyword arg has to supply the optionals. Recall that I said it was possible, not sensible. :) But in tool design I think we should let users hang themselves rather than guess (perhaps wrongly) that no one would ever come up with a good use for such a thing.
A second, lesser motivation is that CL works that way.
c.l.lisp just offered a much better observation: the optional args above are standard for the various "read" functions, and the start and end keywords are standard for string functions. Read-from-string then is inheriting consistently from both families.
> Having all parameters be keyword arguments as well might be interesting, but it wouldn't avoid optional/keyword confusion.
Why not?
Let's say you have a function with 3 standard args followed by 2 opt args. So you have 5 args, all keyable on the symbol you give them in the def.
Let's further say that in a call to this function, I key the middle standard arg (#2 in the def) plus the first opt arg (#4 in the def) and also, I omit the second opt arg (#5 in the def). So, I'm supplying two standard args apart from the two args I'm keying. Then the function call parser would know, after identifying the 2 keyed args and matching them to positions #2 and #4 in the def, that the first non-key arg supplied corresponds to position #1 in the def, the second non-key arg supplied corresponds to position #3 in the def, and that an arg for position #5 in the def is missing, leading to the usage of the default value for this opt arg.
This would even work when you want to raise an error for a non-supplied, non-opt arg.
Wouldn't this work quite intuitively (keying an arg when calling a function "lifts it out" of the normal "vanillas then optionals" argument sequence, shortening that sequence, put keeping a well-defined order for it)? (You would need special syntax for keys in this proposal. My suggestion is a colon appended to the arg symbol, rather than prepended, like in CL.)
Can someone give a counterexample if they think this somehow wouldn't work?
&rest args are left as an exercise for the reader :-)
It could be, if you had an old function that was using optional arguments, and then eventually had to add even more arguments, which you finally decide to make keyworded; without breaking existing code, you can support both optional and keyword args.
What I would like to see is optional, keyword, and rest arguments. Imagine something like this:
The syntax to add rest args, if it followed the CL example, would be:
(dsb (r1 &o o1 &r rest &k k1) data ....)
If data was (1 2 'k1 3) then most params would be bound as expected and then rest would be bound to (k1 3).
But now the data (1 2 'k1 3 4) causes an error "Key list is not even", ie, once you say &k you undertake certain obligations as the caller. Even if you even up the list:
(1 2 'k1 2 3 5)
...you get an error "3 is an invalid keyword", because 3 appears in a keyword position. This can be avoided by announcing your intention to have undeclared keywords:
(a b &rest rest &key k1 &allow-other-keys)
That of course is CL, and it kinda makes my day that if I were crazy enough to extend dsb in Arc I would end up with:
Ok, and then going forward only the new code has the burden of supplying optionals... hmmm, refactoring at 7am with the demo to the CEO scheduled for 9am?...
Neh, what I meant was, you can have a thing that quacks like a list and never see the need for a dotted pair.
That can be hidden below the abstraction. See, case in point, lists/arrays in Ruby, which are more or less lists without the car and cdr part. Although, you do get a 'nil' if you access beyond the end of the array. ;-)
Ruby lists can quack? Bah, let me know when they can mate, that's when you know you have the same species: ie, can two Ruby more-or-less lists have shared structure, formed say by ruby-consing a different element onto the same list?
And ducks can nest. How about Ruby lists? I have always thought the language should have been called Trep, for Tree Processing language. Can I build a Ruby tree and then quickly flatten by juggling some Ruby ducktails?
As for "That can be hidden below the abstraction", OK, now you have a duck: moving smoothly on the surface and paddling like crazy beneath.
The whole comparison just seems (sorry on more levels than one) daffy.
Indeed, you are correct. Ruby[1] doesn't have "native" lists, it has native arrays. Observe:
irb(main):001:0> [].class
=> Array
There is no Ruby cons, nor Ruby car, nor Ruby cdr. There is y.unshift x or [x] + y for cons (though these cannot constructed "dotted Arrays"), arr[0] or arr.first for car, and arr[1..-1] for cdr, but they denote no structure-sharing. However, flattening would merely be arr.flatten—of course it doesn't work by cdr-juggling, but that's because there are no cdrs.
I'm not really sure what the grandparent post means: "lists without the car and cdr part"? That sounds to me like a very good definition of an... array![2]
Also, that set of duck puns/references was fantastic. Well done.
[1]: Which is a very nice language.
[2]: Well, without the O(1) access time, but I digress.
I can take no credit -- they just popped out of my keyboard. There must be some poorly understood connection between algorithms, data structures, and waterfowl.
Actually, they (the infamous "they") did a study, and apparently, ducks are the funniest animal. Take that with as large a grain of salt as you think is appropriate :)
(a) No apologies needed, I kinda anticipated this question. :)
(b) Perhaps the best use case was just added, the application of DSB to DEFUN. I should extend the examples, tho.
(c) In simple cases where one bit of application generates a result in the form of a list of values, native Arc destructuring such as:
(let (x y . z) data-genned-elsewhere ...)
...works fine, and indeed is the most common case. But as the application becomes more elaborate we might start having one bit of code emitting data of a more interesting nature, and then at the point of consumption we want to be able to parse that data tersely and in a self-documenting way.
So I might have an application where RGB triples are being passed around and eventually handed to OpenGL and app code just emits (list r g b) and the code that talks to OpenGL looks like:
(dsb (r g b) color-data
...tell ogl about r g and b ...)
Fine, but now I decide in one place to emit an alpha value as well. No problem, all the code just emitting RGB stays unchanged, but in one place I can emit RGBA and then modify the consuming location thus:
(dsb (r g b &o a) color-data
...etc...)
If I now want to get into shininess or material or other ogl options, the optional thing gets nuts because I have to remember in which position shininess goes and then make sure I supply enough nils for other optionals to get the right alignment, and that breaks if I have a fun default for one of the optionals I am bogusly providing just to get alignment. keyword args get us out of that trap, and we do not even have to list them in the right order, they get worked out at call time (so, yes, do not abuse this feature).
(dsb (r g b &k (a 255) shiny mat) color-data
... etc)
Now in CL we have DEFSTRUCT and can at some point punt on lists and get into emitting:
...and then the consumer reads the structure attributes so it can talk to OpenGL. The Arc DEFTEM is a solution here if we are restricting ourselves to keyword args. And as I said at the outset, a CL-style DEFUN might be the posterboy for DSB, because we can now decide a DEF needs a new parameter and add it as an optional or keyword arg without changing every call. And where an especially hairy DEF such as open-file will end up with a kazillion parameters, keyword args make calling the function a lot slicker.
heh-heh, don't feel bad, i was not kidding when I said it is one of the craziest macros I ever wrote, and I have written over five hundred. Might be the craziest, actually. Macros are normally hard because we are writing code to write code, bouncing back and forth between the mindset of the expanding code and that of the expanded code we are after as we go from backquote to unquote and back.
But in this case with the keyword args the expanded code then had to include code to continue the binding process, because the expander code cannot guess which keywords had been supplied by the caller. Hence the rather sick stop mid-expansion to produce the kvs binding with remaining runtime args paired into an assoc before continuing on to produce the other keyword bindings, each necessitating a runtime lookup of the keyword before deciding if any default should be used, which default might itself still be code!
Man, I now I am confused. Good thing I wrote the macro before I wrote this comment. :)
Needless to say, the macro was evolved: first just getting required params to work. Trivial, but still. Then optionals, with defaults, including computed defaults. I had a bug here, recall, forgetting nil supplied in an optional position means nil, not the default. Then keyword args.
A bit of a giveaway, btw, is my apology above for leaving behind some print statements in the expanding code. We debug macros just like we debug any other code.
Another tip is that I might have broken out that little state machine in the beginning as a separate function to parse the params like:
(a b c &o (d 42) (e (+ c 1)) &k x)
...into an assoc:
'((reqs a b c)(opts (d 42)(e (+ c 1))(keys x))
ie, Divide and conquer: get that working separately, then tackle the rest. I did not do that because of insufficient lookahead (did not realize what I got myself into). :)
(def prt args
(apply prs args)
(prn))
(mac tst (id form expected)
(let res (uniq)
`(let ,res ,form
(if (iso ,res ,expected)
(prt 'test ,id 'OK)
(do
(prt 'error 'attempting ,id)
(prt 'expected ,expected)
(prt 'got ,res)
(prt 'code ',form))))))
And we can test:
(defun tabc (a b c)
(list a b c))
(tst "vanilla" (tabc 'dog 'cat 3) '(dog cat 3))
(defun tabc-od (a b c &o (d 42))
(list a b c d))
(tst "one optional, nil supplied"
(tabc-od 'dog 'cat 3 nil)
'(dog cat 3 nil))
(tst "one optional, 4 supplied"
(tabc-od 'dog 'cat 3 4)
'(dog cat 3 4))
(tst "one optional, unsupplied"
(tabc-od 'dog 'cat 3)
'(dog cat 3 42))
But do not try this at home without the new improved version of dsb below.
I was mishandling nil when supplied as an optional or keyword argument by defaulting the value. When a parameter is missing we use the default if any, but when it is supplied (even if nil, if you can follow that) then that is the value used.
(mac dsb (params data . body)
(w/uniq (tree kvs)
`(withs (,tree ,data
,@(with (reqs nil key? nil opt? nil keys nil opts nil)
(each p params
(if
(is p '&o) (do (assert (no opt?) "Duplicate &o:" ',params)
(assert (no key?) "&k cannot precede &o:" ',params)
(= opt? t))
(is p '&k) (do (assert (no key?) "Duplicate &k:" ',params)
(= key? t))
key? (push-end p keys)
opt? (push-end p opts)
(do (assert (~acons p) "Reqd parameters need not be defaulted:" p)
(push-end p reqs))))
(with (n -1)
(+ (mappend [list _ `(nth ,(++ n) ,tree)] reqs)
(mappend [list (carif _) `(if (< ,(++ n) (len ,tree))
(nth ,n ,tree)
,(cadrif _))] opts)
`(,kvs (pair (nthcdr ,(++ n) ,tree)))
(mappend [list (carif _)
`(aif (assoc ',(carif _) ,kvs)
(cadr it)
,(cadrif _))] keys)))))
,@body)))
I've used Visual Studio, JBuilder, Eclipse, etc., and for the languages I used them for (C++, C#, Java), I felt the IDEs were essential. When I switched to Ruby, the same quality of IDEs weren't available, but I found I was much more productive in Ruby with a simple editor than I had been with other languages plus an IDE, and I actually enjoyed the lighter weight environment quite a bit.
Would I be even more productive in Ruby with an IDE? Possibly. But it's possible that more powerful languages lessen the need for an IDE. Your love of ACL for Common Lisp would be a counter data point, but it's only one data point. Well, Avi Bryant would tout the Squeak IDE as a great advantage in using Smalltalk. In my case, it's somewhat moot given the lack of IDEs for Ruby. As I develop more in Lisp, I'll be better able to judge the effectiveness of the available IDEs.
Tens of thousands of lines of Ruby haven't been a problem for me with just vim, and now Emacs, and I don't think hundreds of thousands of lines would be a problem either if designed well.
"I found I was much more productive in Ruby with a simple editor than I had been with other languages plus an IDE"
Bingo, and that is why we do not see fancy IDEs for agile languages. The corollary being horrid languages have great IDEs because lawdy we need something!
One thing going on here is reflection. If my language has that I can toss off my own rough IDE-ish hacks in minutes and then they do exactly what I want, too, so the demand just never develops for off-the-shelf IDEs.
I work with tens of KLOC of Common Lisp at a time. I like sitting in a backtrace and using a keychord to jump to a functions definition, or sitting in the inspector and jumping to a class definition in source or with a diffeent keychord to the class in a graphical class browser. ACL integrates everything (except, strangely "who calls?", tho that is available via an unexported function), so I just sail all over my big code bases as fast as I can think, click, and keychord. Some folks, btw, feel the same about Lispworks and Slime, we need an IDE smackdown some day to comapre. :)
"I'm curious what things hold back Lisp from being more popular."
Inertia. Things like Java and Python were close enough to C (syntax and imperative paradigm) that moving to them was just a minor course change. Lisp feels like a U-turn, though I have argued elsewhere it is not really. Meanwhile Java had Sun behind it, giving PHBs a vital warm fuzzy. Lisp looks like the north wall of the Eiger in winter to PHBs.
The IDE problem just confirms anyone's fears if they even get that far, though Franz and Lispworks tried to address that by making unlimited-length trials of their fine IDEs available. Agreed on Emacs+Slime not being a good answer for noobs.
"I focused on IDE's, but I'm also interested in the general question of what things can be done and not done to get more people into Lisp."
Not to worry, it is happening, just step back and look at where languages are going, not the people. Steele famously describes Java as dragging people halfway from C++ to Lisp. Not sure about the fraction. :) Python moved a little more, and Ruby moved a little more. If Arc catches on it will constitute completion of the land bridge from static typing imperative languages to dynamically typed functional languages, but it does not matter because Common Lisp is starting to catch on anyway. And if you look at the ideas behind Lisp and why languages like Python and Ruby are doing so well, Lisp has already prevailed. My 2.
In the future, everyone will use Lisp, but they won't know it's Lisp, because it will be called Microsoft Windows Dynamic Service-Oriented Parenthesis Construction Language for .NET.
Mr Dussud (a respected Lisper in a former life) said MS would be happy to host a Lisp if we would just add strong static typing and lose our silly little OO package, CLOS. A couple of other things, too.
Wow, yeah. I tuned him out when I saw he wanted us to declare our variable types. I did not listen to all of the audio, but in there somewhere you will hear some nutjob asking if, given that Lisp has finally started taking over the landscape and he now wants us to run up the white flag and adopt static typing, he also thinks it is time for mainland China surrender to Taiwan. That was me.
I went up later and shook his hand for being brave enough to come into the lion's den. Strong guy. <ouch>
Profusion of dialects - lack of libraries. There is an inverse relationship between the success of languages and their power. C is cumbersome to write; therefore, everyone treasures the libraries that exist and takes pains to remain compatible. Java, even assuming that it made it 'halfway to Lisp', is clearly still cumbersome enough. Python is already too powerful, but it benefits from its slowness. In many problem domains (GUI, databases, numeric computing), it is way too slow, forcing you to use C-coded extensions, which again help to prevent fracturing of the user base. Where it's fast enough, you get a bazillion of half-assed libraries and frameworks. E.g. Python web programming is a mess and, compared to PHP and Java, not popular at all. (There are also limitations to the VM that matter here but I'm not going into those.)
Lisp is so powerful that it lets everyone modify it, and so everyone goes and creates their own dialect of it. It compiles to fast, native code, so no need to adopt and maintain libraries shared by a large user base.
There is a nice language called D. Unfortunately, it comes with two incompatible standard libraries. How do you like that as a new user? Well, in Lisp it's the same, only times 257. Throw in the oatmeal cum nail-clippings visual appeal (L.Wall) and you might well wonder why it is not less popular than it currently is.
I am afraid I do not know SQL syntax at all and it looks rather horrifying, but here is a more functional approach:
(= *sql-template*
; first, express semantics with the template structure
'(Study (StudyId (int) studyseq) ;; this will always be the key field.
(StudyCode (string 25)) ;; and always encapsulate the type info in their own list
(StartDate (datetime 8))
(Description (string 25))))
The trick above is to be a little creative on the front-end and express some things with structure that will make the code do less work. I eliminated the pk designator and simply specify the first thing after the table name to be a key designator. If there can be multipel keys, the template would need a list in that position and then the destructuring would be (table keys . otherfields).
It does not come to play, but notice that I wrapped the type info in its own list in case we ever do need to get at it algorithmically.
In the code I can now use destructuring to parse the template (and if I had to dig into the field info I would dstructure those, but here I just use car)
(let (tbl key . fields) *sql-template*
(prn (string "insert into " tbl
" ( " (let c nil
(each (f . rest) fields
(when c (= c (string c ", ")))
(= c (string c f)))
c)
", " (car key)
" ) values ( "
(let c nil
(each (f . rest) fields
(when c (= c (string c ", ")))
(= c (string c ":" f)))
c)
", " (last key) ".nextVal"
" ) returning " (car key) " into " (string ":" (car key)))))
This version uses list splicing:
(let (tbl key . fields) *sql-template*
(prn (apply string `("insert into " ,tbl
" ( " ,@(let c nil
(each (f . rest) fields
(when c (push ", " c))
(push f c))
(rev c))
", " ,(car key)
" ) values ( "
,@(let c nil
(each (f . rest) fields
(when c (push ", " c))
(push ":" c)
(push f c))
(rev c))
", " ,(last key) ".nextVal"
" ) returning " ,(car key)
" into " ,(string ":" (car key))))))
By the way, this is almost a solution in Common Lisp using the format string mini-language:
(destructuring-bind (tbl key . fields)
'(Study (StudyId (int) studyseq) ;; this will always be the key field.
(StudyCode (string 25)) ;; and always encapsulate the type info in their own list
(StartDate (datetime 8))
(Description (string 25)))
(format t "insert into ~a (~{ ~a~^,~}) values (~{ :~a~^,~}) returning ~a into ~a"
tbl
(append (mapcar 'car fields)(list (car key)))
(append (mapcar 'car fields)(list (car (last key))))
(car key) (car key)))
It does not do the nextVal bit, and I would have to put bars around things like |StudyCode| to preserve case. But the neat thing is the ~{~} list expander (like @ in backquoted lists) and the escape thing ~^ that arranges for the comma not to print after the last item.
Isn't destructuring cool? We take an anonymous list and quickly associate useful names with what otherwise would be car, cadr, and caddr, etc.
In common lisp when we use destructuring-bind we can have optional /and/ keyword arguments, so you can plan ahead and create an adhoc list like (1 2 :height 3 :width 4) and let :weight default to 42:
(destructuring-bind (x y &key (height )(width 0)(weight 42))
..etc with x, y, height, width, and weight...)
Hmmm, I think I did that for my Arc implementation of defun...shuffle, shuffle, dig..ah:
(mac defun (name params . body)
(w/uniq (rtargs)
`(def ,name ,rtargs
(withs ,(with (reqs nil key? nil opt? nil keys nil opts nil without)
(each p params
(if (is p '&o) (do (assert (no opt?) "Duplicate &o:" ',params)
(assert (no key?) "&k cannot precede &o:" ',params)
(= opt? t))
(is p '&k) (do (assert (no key?) "Duplicate &k:" ',params)
(= key? t))
key? (push-end p keys)
opt? (push-end p opts)
(do (assert (~acons p) "Reqd parameters need not be defaulted:" p)
(push-end p reqs))))
(with (n -1 kvs (uniq))
(+ (mappend [list _ `(nth ,(++ n) ,rtargs)] reqs)
(mappend [list (carif _) `(or (nth ,(++ n) ,rtargs)
,(cadrif _))] opts)
(list kvs `(pair (nthcdr ,(++ n) ,rtargs)))
(mappend [list (carif _)
`(or (alref ,kvs ',(carif _))
,(cadrif _))] keys)
)))
,@body))))
