I'm starting to think that if a language is compilable by default (i.e. without compiler hints from the programmer), it could be an indicator that the language isn't dynamic enough.
For example, arc's macros are fairly powerful as is, but people sometimes wish their scoping was more robust or that you could pass them around as function arguments. When it's learned that these features come at the cost of compilability, the usual reaction is to conclude that the features aren't worth it. I think the proper reaction (when expressivity is more important to you than performance) is to conclude that compilation isn't worth it.
My opinions on this stuff are fairly new and untested, so maybe one of the more experienced lispers here just needs to set me straight. ;)
I like compiled languages like Scheme because they let me be expressive. I can use whatever oddball syntaxes I want, safe in the knowledge that the parsing inefficiencies will only give my application horrible load time. :) Performance isn't nearly as important to me as expressiveness, but I do sometimes write programs for purposes other than thought experiments, and those generally need to finish what they're doing faster than the time it takes for me to give up and rewrite them. ^^ In those cases, I'm happier if I've chosen a language that gives me a fair amount of practicality and a good amount of expressiveness at the same time, rather than one that promises lots of expressiveness but forces me to give it all up in order to get things done.
Fexpr calls--I assume you're talking about those--are pretty much never what I actually want to express in syntax anyway. When I'm writing and reading code, I'm viewing it from a static standpoint, and so I choose syntaxes that make sense in static ways. If the very description of my algorithm means different things at different points in the execution of the program, that's darn confusing. XD (I suppose that would be Necker Cube code.) Not all fexpr-powered programs need to use them that way, but if they don't, what's the point?
Were I to be viewing the program in motion, it would be a different story. Then I'd want a very dynamic syntax, even one that shows me the values of variables and other dynamic information, while still presenting it all in a well-factored enough way that it's easy to digest. I'd do searches on things like "log file" and "data visualization" to find inspiration. But then I'd probably never write in that syntax. Since run time deals heavily in concretes at high speeds, I'd probably look for a way to interact using some kind of shell scripting language, or even a joystick input. :-p
I hope I haven't been too abrasive here. ^^; I actually have only a very weak conscious idea of why I like that Arc's compiled, and it took me several hours to collect my thoughts enough to post. Then this post kinda found itself all at once. ^_^
Not abrasive at all! :) Hope you don't mind if I respond in small pieces (i.e. some now, some later).
> Not all fexpr-powered programs need to use them that way, but if they don't, what's the point?
Am I incorrect in thinking that arc macros are a subset of fexprs in terms of expressiveness? You should be able to define 'mac in terms of an fexpr and eval, and then do all the macro stuff you're used to doing. A disadvantage would be the increased run-time penalty. Advantages could include:
1. Lexical scoping now applies to macros, so you no longer have to use that do.foo pattern to avoid accidental macro-expansion (http://arclanguage.org/item?id=11688)
2. Local and anonymous macros
3. Module system possibilities open up because you can now store callable macros in lists and tables
4. The language implementation can become smaller and more hackable with the elimination of the macro-expander
Do you find any of these to be particularly desirable, or do you have objections about any of them?
I'm reminded again of a post of mine I brought up a while ago, http://arclanguage.org/item?id=11684. [1] The only point I raise against fexprs there--and interpretation of quoted code in general--is that it gives up static optimization and verification. ^_^ This is part of why it took me a while to form a response initially; my main objection to your argument was that the assumption "when expressivity is more important to you than performance" wasn't black and white for me, so I wasn't sure it would be a relevant response.
So I'll probably be a bit wishy-washy in the following responses. Our opinions aren't actually in conflict; they're just motivated by different premises.
Am I incorrect in thinking that arc macros are a subset of fexprs in terms of expressiveness?
I think so. I think the one semantic difference is that you can redefine an fexpr and count on all the existing uses of it to work using the new definition. But that's only a positive. :-p
---
1. Lexical scoping now applies to macros, so you no longer have to use that do.foo pattern to avoid accidental macro-expansion (http://arclanguage.org/item?id=11688)*
The most natural (IMO) way to add this to Arc would be to have the compiler pass around a complete local static environment, rather than just a list of the local variables. It could be as simple as replacing that list with an alist. Of course, then there'd need to be a 'w/mac form too.
I admit this solution adds complexity to the language core. That's issue #4, though. ^_^
---
3. Module system possibilities open up because you can now store callable macros in lists and tables
Yep, that's totally true. You can import such structures globally in either kind of language, especially if they support replacing the global environment so you don't clobber existing things (Penknife's approach), but local scope imports make the meanings of variables in their scope ambiguous until the import is performed at run time, and at that point, compile-time macros will have been missed.
Personally, I like knowing which variables I'm importing at the time I write the code, so even if I do want to locally import a bunch of variables all at once, perhaps for some macro, I find it tolerable to build a complete (with (a _!a b _!b c _!c) ...) form for that purpose. Alternatively I'd limit local imports to statically determinable sets of names (like forcing (w/import-from var-list foo ...) to use a predefined global variable 'var-list). Of course, these aren't as general. ^_^
--
4. The language implementation can become smaller and more hackable with the elimination of the macro-expander
That's also true. :) My opinion is that a smaller language implementation doesn't imply a more convenient language to use, even if it may be a more convenient language to maintain. If for some reason we want to compile some of our code (probably for performance, but also potentially for code generation), and the language already gives us a suitable framework to do that in, we don't have to build that framework ourselves. In fact, it's more convenient to simulate a simple framework (e.g. Arc) within a complicated one (e.g. Racket) than vice versa.
