I added in Arubic-style namespace support, and also implemented dynamic and parameter variables better. If you're interested in reading more, check out these two links:
I finally added in Arubic to Nu. Now all you have to do is `./arc -i arubic` and you'll get a REPL with Arubic functionality. You can also use `(import arubic)` to write a file using Arubic.
Why would you want to do that? Well, here are the current changes between Arc and Arubic, though I plan to add more over time:
[1 2 3] -> (list 1 2 3)
{a 1 b 2} -> (obj a 1 b 2)
Because of the changes above, you no longer need the [] syntax for functions most of the time, so I'm repurposing them to create lists, which I've found to be far more common.
`isa` accepts multiple arguments:
(isa x 'foo 'bar 'qux) ;; Arubic
(in type.x 'foo 'bar 'qux) ;; Arc 3.1
`prn` prints spaces between each argument:
(prn "foo" "bar" "qux") -> "foo bar qux\n"
You can also use `str` and `str?` as aliases for `string` and `string?`.
Small changes, but they do make writing code more pleasant.
2) This helps out a ton with arc2js. In arc2js, now all I need to do is provide an `=` macro and I'll get zap, or=, push, pull, swap, etc. all for free. Without this change, I'd have to define my own custom versions of zap, or=, etc...
"It truly does not make any sense to me why zap is defined like this"
'zap is the only use I typically have for the 'setforms "binds" list (which ensures the subexpressions of 'place are only evaluated once).
Still, 'zap doesn't need to work that way: as long as I'm using a language where I know 'zap evaluates its place twice, I'm pretty much okay with it. It's a wart, but it's not an impediment.
To explore Arc-3.1-like options for a bit, here's a cleanup of Arc 3.1's definition of 'zap:
(mac zap (op place . args)
(with (gop (uniq)
gargs (map [uniq] args)
(binds val setter) setforms.place)
`(atwiths (,@binds ,gop ,op ,@(mappend list gargs args))
(,setter (,gop ,val ,@gargs)))))
If we allow 'setter and 'val to compile and evaluate before 'op and 'args, it gets shorter:
(mac zap (op place . args)
(let (binds val setter) setforms.place
`(atwiths ,binds
(,setter (,op ,val ,@args)))))
I prefer to arrange the compilation and evaluation orders from left to right ('op, 'place, 'args), using a technique like this:
(mac place (place)
(let (binds val setter) setforms.place
`(withs ,binds
(list (fn () ,val) ,setter))))
(mac zap (op place . args)
`(atomic:fn-zap ,op (place ,place) (list ,@args)))
(def fn-zap (op (getter setter) args)
(setter:apply op (getter) args))
"'zap is the only use I typically have for the 'setforms "binds" list (which ensures the subexpressions of 'place are only evaluated once)."
Hm... yes, you're right, `(zap + (foo (bar qux)) 1)` evaluates `(bar qux)` twice, and I don't see an easy/obvious way to fix that in `=`. I'll need to think about this.
It would appear that the `=` operator is already thread-safe even without `atomic-invoke`. In any case, if you're terribly worried, you can always wrap it yourself.
I just got rid of `setforms` completely[1]: not even `=` uses it anymore. I then reimplemented `=` in a much shorter and clearer way[2].
Arc 3.1 takes 80 lines to implement expand=, but Nu takes only 31. And Nu is much clearer and easier to understand as well. In addition, Nu's output is much shorter and is faster:
All constant-time procedures and operations provided by Racket are
thread-safe because they are atomic. For example, set! assigns to a variable
as an atomic action with respect to all threads, so that no thread can see a
“half-assigned” variable. Similarly, vector-set! assigns to a vector
atomically. The hash-set! procedure is not atomic, but the table is
protected by a lock; see Hash Tables for more information. Port operations
are generally not atomic, but they are thread-safe in the sense that a byte
consumed by one thread from an input port will not be returned also to
another thread, and procedures like port-commit-peeked and write-bytes-avail
offer specific concurrency guarantees.
It mentions that hash table assignment is not thread-safe, however if you then go to the hash table page[1], it says this:
A mutable hash table can be manipulated with hash-ref, hash-set!, and
hash-remove! concurrently by multiple threads, and the operations are
protected by a table-specific semaphore as needed. Three caveats apply,
however [...]
In other words, Racket already handles everything, according to the docs. If you're ever worried enough, or run into any problems, it's not hard to wrap it in `atomic` yourself. I'd rather not have the cost of `atomic-invoke` for every assignment, especially if you run all your code in one thread (like I do).
By the way, Nu doesn't use `set!` for global assignment, so I'm not sure if global assignment in Nu is thread-safe or not. But I'd assume it is, since I think `namespace-set-variable-value!` is constant-time.
This ensures that car.foo, (bar), and (scar gs1 val) all happen without interference in between. I suspect Racket at most protects those on an individual basis.
"In pg-Arc, '= on a variable is 'assign without 'atomic. Where 'atomic comes in is when there's a setforms thing to worry about."
I am aware. It still seems to me that if you're dealing with threads, you should wrap assignment in atomic yourself if you're worried about such things. Code that doesn't deal with threads shouldn't have to use atomic.
Perhaps there should be an `a=` macro that's just like `=` but it calls `atomic`. Hm... I wonder... would it be possible to detect whether code is running in the default thread and if not, automatically wrap it in atomic...? May be more trouble than it's worth, though.
That's what I think. Anyone who cares can say (atomic:= ...) or (atomic:zap ...), so I only see a couple of reasons why we'd want to have the 'atomic implicit:
- We want to use it all the time anyway. (I doubt it, but it's hard to tell. I haven't used threads, and therefore I've never bothered to find a way to squeeze utility out of it.)
- There are people who do care, and they'd be better off if the people who didn't care still used 'atomic by accident. (Again, it's hard for me to tell if this is true.)
I just implemented `eachfn` and then changed Arc so the `each` macro just calls `eachfn`. This had a significant speedup, with no cost at all in functionality. At this rate, Nu will end up being just as fast as Arc 3.1, with all the shiny extra features and bug fixes.
So, the rule of thumb is: don't write macros that do a lot of work. Instead, write a function that does the work, and then write a macro that just calls the function. This goes double for macros like `each` that do a type-check at runtime.
...because each does not map. Nor does it some. Nor does it mappend. Nor does it all. Nor does it keep. Nor does it rem.
I don't think you understand what I'm talking about. In Arubic, map is just a macro that expands into mapfn. The semantics are exactly the same as Arc 3.1, it's just easier to say...
(map x foo ...)
...rather than:
(map (fn (x) ...) foo)
The only point of it is to remove the (fn ...) bit, resulting in shorter and easier to read code.
"Focus on just map. Are you saying it doesn't return a transformed list?"
No. You are still misunderstanding me. I will be as clear and precise as I possibly can. In Arc 3.1, you call `(map x foo)` where `x` is a function and `foo` is a list.
In Arubic, you would say that as `(mapfn x foo)`. It's exactly the same, except you use the name `mapfn` rather than `map`. Now, Arubic also provides a macro called `map`, which expands into `mapfn`. In other words:
The sole purpose of this macro is so that you don't need to type out the `(fn ...)` part in `(mapfn (fn (x) ...) foo)`. That's it. The meaning of `map` is the same, it's just that now that it's a macro, it's a lot more convenient to use. But you can still pass in a function directly by using `mapfn`.
---
"You'll have to pry (map car xs) from my cold dead hands :)"
If you dislike any parts of Arubic (whether misunderstood or not), simply do not use those parts. The point of the namespace system is that you can change what you want and use what you want, rather than being tied down to one semantic/syntax/meaning/language. Thus, it is possible (and relatively easy) to use some of the things from Arubic that you like, while not using the parts you dislike.
In this particular case, however, you are misunderstanding me. Your example would be `(mapfn car foo)` in Arubic.
In this particular case, however, you are misunderstanding me. Your example would be `(mapfn car foo)` in Arubic.
No, I'm now pretty sure I'm not misunderstanding you.
My entire objection is to the name switch. Of course I know I can change the default. It's not a sensible default, it's like defining 1 to be 0.99. You're fragmenting the semantics of a name that has thus far had a pretty clear meaning. You're welcome to do this, but it's going to hinder users with any programming background.
I don't understand why the thing you want must be called map. Why not mapmc, for example? Or some other 3-letter name?
"My entire objection is to the name switch. Of course I know I can change the default. It's not a sensible default, it's like defining 1 to be 0.99."
It's not the default. The default is Arc 3.1. You have to opt-into Arubic. In any case, how is it not sensible? It provides the shortest, easiest to read, and fastest code.
---
"You're fragmenting the semantics of a name that has thus far had a pretty clear meaning."
And this is why I think you're still misunderstanding me. The meaning of map is the same. It's just that now you can write (map x foo ...) rather than (map (fn (x) ...) foo), which is shorter and easier to read.
If you're worried that it'll mess up Arc programmers who are used to the naming of map, then as already said, Arc 3.1 is the default. You only get the new syntax if you import Arubic.
I do not see how changing the meaning of names in a new language is a bad thing. That's like me saying that wart should use exactly the same names as Arc, which is absurd: they're different languages. If you want to use a different name for something in wart, then go for it.
Perhaps you still do not grasp that Arubic is a different language from Arc, implemented in a separate namespace so as not to mess up Arc's namespace.
---
"I don't understand why the thing you want must be called map. Why not mapmc, for example? Or some other 3-letter name?"
Suit yourself. I don't understand what I have said to merit hostility.
I know we can all do what we want. I also am aware of lots of things I can try if I don't like something you do. I don't understand these defensive reactions. Was I rude somewhere?
All this while you keep insisting you know what I am misunderstanding. Is that not rude?
Arubic's map is a macro, not a function. Until it grows fexprs the meaning is by definition not the same.
"This is how I like it" is a perfectly valid reason (and would have ended this conversation eons ago). Just don't insist it's all still the same when it isn't.
No, the default is racket. No, it's machine code. Or is it cosmic background radiation? WTF, are you seriously saying that creating a new language isn't an act of choosing defaults?
In Nu, the default namespace is the one corresponding closely with Arc 3.1. And in this namespace, the default definition of 'map is pretty much Arc 3.1's. Therefore, "it's not the default" is true in Nu.
Another namespace in Nu is Arubic. Using this namespace is like using a language other than Arc 3.1, where 'map is a macro. If your problem is with Arubic itself, you're free to ignore the parts of Arubic you don't like (even if you want to use an library written in full Arubic). If the point is you think people shouldn't dilute the overall meaning of "map" with the meaning Pauan gives it in Arubic, that's a moral position, with Pauan as an unintentional villain!
---
What I think is that the meaning of "map" isn't tainted here, at least not any more than usual. In fact, I wanted 'map to be an 'each -like macro before I knew Pauan had the same thing in mind.
If I don't know what language we're talking about and I hear "map," I assume it'll be an abstraction that applies a given transformation to elements of a given data structure (whatever "abstraction," "transformation," "data structure," and "given" mean), along with some accidental complexity suitable to the language, such as the timing of computation, side effects permitted in the transformation, or late-bound dependencies (like 'map calling out to 'cons, such that rebinding 'cons changes what happens).
In a language where macros like 'after and 'each are more convenient to use than higher-order functions like 'protect and Anarki's 'trav, a macro for map is to be expected.
In Arc, almkglor[1] calls this macro "mapeach" in Anarki, and I call it "maplet" in Lathe. Pauan and I call it "map" as long as we don't have naming conflicts to worry about. As long as this macro is the primary way we use the map concept, using that name keeps our programs brief and frank.
Notice that the namespace is now length 2, because it inherits from Arc's namespace. But you don't want to use Arubic, you want to use Arc! But because of the malleability and inspectability of Nu namespaces, it's easy to fix that. Just do this:
The above just reversed the order of the namespace, so now the Arc namespace inherits from the Arubic namespace, rather than vice versa.
This effectively "undoes" the changes that Arubic made, which is exactly what you want: now you can use the functions and macros defined in the Arubic library from within Arc, and everything works perfectly.
I just did some timing tests. As I suspected, the namespace system is quite costly in terms of performance:
1 2 3 4 6 6 7
0% 1732% 31% 22% 20% 17% 12%
What the above means is that when going from 1 namespace to 2, you lose 1732% of speed, meaning your code will now run ~17 times slower. But going from 2 namespaces to 3 has only a 31% drop in speed, and 3 to 4 only has 22%, etc.
What this means is that the initial hurdle of namespaces is quite large, but adding more namespaces after the 2nd is quite decent in terms of performance.
The reason for this is (ironically) speed optimizations. When executing code in the Arc namespace, I actually completely bypass the namespace system entirely, so it's relying on Racket to do global lookups, and Racket is far faster at looking up global variables than Nu is.
That's why going from 1 to 2 namespaces is so slow: Racket is so fast that when it switches to the Nu code, Nu looks horribly slow by comparison. But when going from 2 to 3 namespaces, we're already using the Nu code, so the difference in speed is tiny.
In other words, Racket is so fast it makes Nu look bad.
---
By the way, this only applies to code that is evaluated in a Nu namespace: if you evaluate everything in Arc's namespace, there's no cost in speed at all. So this drop in speed only applies while using things like Arubic, and not Arc itself.
I just added in an interesting feature: automatic namespace creation.
When using the `(import ...)` form, if the file tries to overwrite an already-existing variable, it will automatically create a new namespace. Otherwise, it won't.
This is mostly a performance optimization, since it's entirely possible to load every single file into a different namespace, but using this technique I avoid creating namespaces except when absolutely necessary, while still providing a reasonable amount of library isolation.
I'm still anti-namespaces -- I'd rather work on codebases that don't need them, and I'd rather just rely on tests to catch collisions. But your article makes an eloquent case for them that has had more effect than anything else I've read.
I would like to note something. If all you want to do is write programs and get stuff done then namespaces are indeed a luxury. It is not particularly difficult to use a language (including Arc) to write useful programs, despite the lack of namespaces.
Where namespaces start to become necessary is when you want to change the language itself beyond what can be easily done by adding macros. Then you are in the predicament that you want to write your code in the new (and presumably better) language that you created, but now you have to give up libraries written in the old language.
With namespaces, it's no longer an either/or thing: you can have your cake and eat it too (at the cost of some performance). So, for writing code, namespaces aren't important. But for language experimentation, namespaces are so incredibly useful that I consider them necessary.
The only reason I put namespace support into Nu's core is because I really want to use Arubic, but doing so practically requires decent namespace support. I invented this system out of necessity, which should (hopefully) demonstrate its usefulness.
"Would it be hard for Nu to permit lifting arc primitives?"
Not at all. Arc lifting happens automatically and transparently with namespace inheritance. It works right now. In fact, that's the entire point of the namespace system.
And if you ever want to directly evaluate something in Arc (bypassing the inheritance system), you can call `(w/arc3 ...)`
Okay, I'm still not sure what you mean by "lifting Arc primitives" but I'll do my best to explain how Nu works.
The compiler itself is written in Arc's namespace. There is no separation at all between Arc, the Arc compiler, and Racket: everything is cobbled into one giant namespace. So changing the compiler is as simple as writing an Arc function/macro. This is how ar works, and I liked it enough that I copied the idea for Nu.
That also means no need for the $ primitive because all Racket stuff is prefixed with `racket-`. Thus you would say `racket-regexp-match` rather than `$.regexp-match`.
Note: it's possible to import Racket modules with any prefix you like (including no prefix), but it's there mostly to avoid namespace collisions with Arc stuff.
---
As for doing the same thing with Arc primitives, I assume you mean that if you create a new namespace, you might sometimes want to use the definitions in the new namespace, and sometimes the definitions in Arc's namespace.
You can do that too, by calling eval-w/ with arc3-namespace:
(eval-w/ arc3-namespace
foo)
Because it's common enough to want to load things into Arc's namespace, the above can also be expressed as w/arc3:
(w/arc3 foo)
That will give you the definition of "foo" from Arc's namespace. This shouldn't be necessary most of the time because it's possible to have a namespace inherit from another. In that case, if the variable isn't found, it'll search in the parent.
Thus, if you make a new namespace that inherits from Arc's namespace, you'll automatically get all the Arc stuff for free. That's how Arubic works: it makes a namespace that inherits from Arc, and then changes things. Anything that Arubic doesn't change is automatically taken from Arc.
Ok perhaps your namespaces are the simplest way to 'lift' arc into Nu.
If you implement language A in language B, A is above B in the stack. It's meaningless to call A's code from B, but useful to call B from A. The latter is what I call lifting. But I realize this terminology doesn't work for anything but simple static compilation.
If I wrote an awesome library in Arubic, especially a library that would be difficult to port to Arc, and you wanted to use the library, but didn't want to use Arubic, it would be very useful to write Arc code that used said Arubic library.
As another example, you may want to write a module in Racket that imports a library written in Arc. Even though Arc is implemented in Racket, I don't see how that counts as "meaningless". This example isn't even theoretical, there was somebody on this forum earlier who wanted to do exactly that:
