You know, every once in a while I consider setting up slime, or at least looking at it, but I've never used slime before and I don't really know what it offers.
What are your favorite features of slime, and why would I want to use it over just emacs + arc.el + repl-in-shell?
I think you've edited your post a few times, and given how long it is I'm not quite sure which parts have changed. Perhaps in the future you could break up your thoughts among multiple comments? One advantage would be that I could reply to each point separately.
I don't agree with all of his points, especially those concerning syntax, but it does raise some interesting questions. Is a cons based list system actually bad? Are there better things that we could be doing? And how hard would it be to emulate some of the features that he seems to like so much about Mathematica?
"Is a cons based list system actually bad? Are there better things that we could be doing?"
They are interesting questions. I think that to be a lisp, a language must homoiconic and applicative. A cons-based system is non-essential, and I am coming to agree with the OP that it's not the best way.
Update: Hmm... my mind is playing tricks on me now. What would you use besides conses to represent s-expressions? Maybe that is a good role for cons.
I wonder if there's a way to merge quote and lambda into a single operator. This is done in concatenative languages like Joy and Factor, where a "quotation" acts both as quoted code and as an anonymous function. But I'm struggling to see how you could translate that into applicative terms.
"What would you use besides conses to represent s-expressions?"
What are s-expressions? I thought their main aspect was the way they used lists to represent user-extensible syntax forms (macro forms, function applications, fexpr applications). I'm not a fan, but even if we assume s-expressions, they could always use some other implementation of lists.
--
"I wonder if there's a way to merge quote and lambda into a single operator."
Do you mean like in PicoLisp, where all user-defined functions are lists containing their code, and applying a list causes it to be interpreted as a function?
I don't like it, 'cause you don't get any lexical scoping this way. It probably makes more sense in Factor and Joy thanks to the lack of local variables for run-time-created functions to capture.
"What would you use besides conses to represent s-expressions? Maybe that is a good role for cons."
Arrays, objects, pretty much anything that can represent nested sequences. In fact, in PyArc, conses are a class, because Python has an infatuation with classes.
I don't see conses as an "ultimate abstraction". They're a low-level primitive that can be used to create lists, among other things. To put it bluntly, they're a simple way to implement linked lists. The only difference between an array and a linked list is performance, but they have the same semantics.
Most popular languages choose arrays as their default sequence type, but Lisp chose linked lists. I could represent conses as Python lists (which are like arrays), and then define car so it returns list[0] and cdr so it returns list[1:].
As far as Arc programs would be concerned, everything would look exactly the same. The only difference is that certain operations (like prepending) would be slower with arrays than with a linked list. Specifically, (cons 'a some-list) would be O(n) rather than O(1).
So... to say that conses are bad is like saying that arrays are bad. They're both sequence types, that have differing performance characteristics. Use whichever one happens to suit your language/program the best.
Arc already supports docstrings, and has a built in function 'help that displays them, along with the signature of the function and the filename in which the function was defined. The help information can also be accessed from an op when the arc server is running, but I'm not sure how much of that functionality is Anarki specific. There is also 'sig that displays the signature of a given function, 'fns that searches for a given function name in the list of presently defined functions, and on Anarki there's also 'src that prints out the source of a given function or macro. Unfortunately, Anarki's ppr is currently broken because len no longer works for improper lists.
I've been interested for a long time in runtime accessible automatic documentation, and 'src and 'help go a long way towards providing that. One of the few things I think we still need in that regard for runtime documentation is a means of searching for fns by category, and that will probably involve either adding tags to the docstring, or searching the code for a given pattern. Having the source of a function stored in a table at runtime is incredibly useful, and could allow you to search for all functions that call a given function or macro, or that use a particular idiom.
Either of these are neater than using `and`, which for clarity is better only used as a predicate, in my opinion.
Thanks fallintothis, zck, and shader for your improvements. I'm trying to take everyone's suggestions into account to make a new version, which I'll put in Anarki when I'm satisfied with it.
Why? I'm not sure I understand the extreme dislike of overloaded operators. Not that this comment is necessarily directed at yours, but the concept in general.
If it's because of efficiency, the racket operator is probably fairly efficient anyway, and might even do some compile time type checking to see whether the result is likely to be one or the other.
At some point, instead of having nice operators to encapsulate the alternatives and paying the cost of minimal dynamic checking (if any), you'll end up building your own much more expensive explicit dynamic checking instead.
Also, I'm not sure how this urge for separation of functions can contribute much to arc's main goals of brevity and simplicity. Having one function with a short name to handle many very closely related operations makes plenty of sense most of the time, and lets you use plain + instead of +-float and +-int.
Oh, for myself I routinely use + for joining lists, so I'm firmly in the general purpose + camp.
My point was merely that whatever argument someone may have for preferring an arithmetic-only +, there is an isomorphic argument for more specifically preferring an exact-+ or a floating-+, and so if the former is applicable to your situation you may also want to consider if the latter also applies.
Hi, finally replying to this thread. So, I, a long time ago, found that the + function was allocating memory every time it was called; this a) made things slower than they had to be (according to my old thread, incrementing a variable was half as fast as it should have been), b) triggered garbage collections, which were annoying pauses at the REPL, and c) triggered garbage collections when I was doing, like, (time:repeat 100000 <test procedure>) (note that "repeat" is implemented with "for", which uses +).
The problem with (c) is that it made the time to completion rather variable (depending on whether a GC happened, or sometimes on how many GCs happened), so the results were difficult to interpret. It interfered with my measuring and comparing the performance of my procedures.
So it annoyed me. And I found that just using the Scheme + fixed it, stopped the malloc'ing. To know that this crap was caused by a feature I didn't use and that Paul Graham had seemed to declare was a bad idea, a feature that seemed it shouldn't even be there anymore... I took it upon myself to eradicate it from the source code. And I felt better afterwards.
But now this case-lambda version of + (which is polymorphic) seems to not malloc at all, and a bit of testing by me indicated it performed approximately as well as the raw Racket +; I couldn't see any difference. (Btw, I think I perceived that Racket doesn't malloc for argument lists when you call "apply" or something, but it does malloc--probably copies the arglist--if you access it with car or cdr.)
So polymorphic + is no longer so obnoxious to me, and I no longer feel the need to eradicate it. I'd probably still prefer to use "string" to join/coerce to strings and "join" to join lists, myself, and I'd probably prefer "join" as a generic join operator. But I won't kill concatenate-+ on sight. Since overloading itself seems not to be the cause of the performance problems, I might warm to the idea... maybe overload + for my polynomials--or for matrices, I've been playing with them recently--though I've actually only multiplied them, never added them--perhaps I should overload *...
You may find the following definitions extremely useful. (I do.) Evolved from my previous "mem-use" macro.
;note that "memory" = $.current-memory-use
; and "current-gc-milliseconds", "current-process-milliseconds"
; were supplied in ac.scm
(= gc-msec current-gc-milliseconds
process-msec current-process-milliseconds)
(mac utime body
(w/uniq (gtime ggc gmem)
`(with (,gtime (msec) ,ggc (gc-msec) ,gmem (memory))
(do1 (do ,@body)
(prn "time: " (- (msec) ,gtime)
" gc: " (- (gc-msec) ,ggc)
" mem: " (- (memory) ,gmem))))))
(= time utime)
You might expect some memory or time overhead just from using this, but in fact there doesn't seem to be any:
arc> (time:- 1 2)
time: 0 gc: 0 mem: 0 ;zero overhead
-1
;now I copy the old definition of + from ac.scm to clipboard
arc> (= + (eval:list '$ (read:pbpaste)))
#<procedure>
arc> (time:+ 1 2)
time: 0 gc: 0 mem: 352 ;probably from JIT-compiling +
3
arc> (time:+ 1 2)
time: 0 gc: 0 mem: 64
3
arc> (time:+ 1 2)
time: 0 gc: 0 mem: 64 ;by now this is clearly the usual mem-use
3
arc> (time:+ 1 2)
time: 0 gc: 0 mem: 64
3
arc> (= + $.+) ;Racket +
#<procedure:+>
arc> (time:+ 1 2)
time: 0 gc: 0 mem: 0
3
arc> (time:+ 1 2)
time: 0 gc: 0 mem: 0
3
;now I copy this definition to clipboard:
(define (ar-+2 x y)
(cond ((char-or-string? x)
(string-append (ar-coerce x 'string) (ar-coerce y 'string)))
((and (arc-list? x) (arc-list? y))
(ac-niltree (append (ar-nil-terminate x) (ar-nil-terminate y))))
(#t (+ x y))))
arc> (eval:list '$ (read:pbpaste))
#<void>
arc> (time ($.ar-+2 1 2))
time: 0 gc: 0 mem: 416 ;JIT
3
arc> (time ($.ar-+2 1 2))
time: 0 gc: 0 mem: 0 ;so it allocates zero memory
3
arc> (time ($.ar-+2 1 2))
time: 0 gc: 0 mem: 0
3
Now for more playing around. A Racket loop that counts to 1 million.
arc> (time:$:let loop ((n 0)) (if (> n 1000000) 't (loop (+ n 1))))
time: 3 gc: 0 mem: 760
t
arc> (time:$:let loop ((n 0)) (if (> n 1000000) 't (loop (+ n 1))))
time: 5 gc: 0 mem: 920 ;kinda weird, it alternates 760 and 920
t
;now up to 10 million
arc> (time:$:let loop ((n 0)) (if (> n 10000000) 't (loop (+ n 1))))
time: 36 gc: 0 mem: 760
t
;now we test ar-+2
arc> (time:$:let loop ((n 0)) (if (> n 10000000) 't (loop (ar-+2 n 1))))
time: 1020 gc: 0 mem: 1096
t
arc> (time:$:let loop ((n 0)) (if (> n 10000000) 't (loop (ar-+2 n 1))))
time: 1019 gc: 0 mem: 776
Apparently it makes a significant difference (30x) inside a tight Racket loop. For fun, let's try the unsafe ops too.
arc> ($:require racket/unsafe/ops)
#<void>
arc> (time:$:let loop ((n 0)) (if (unsafe-fx> n 10000000) 't (loop (unsafe-fx+ n 1))))
time: 28 gc: 0 mem: 760
t
Hmm, I had an idea. Put the number check first. New definition:
(define (ar-+2 x y)
(cond ((number? x) (+ x y))
((char-or-string? x)
(string-append (ar-coerce x 'string) (ar-coerce y 'string)))
((and (arc-list? x) (arc-list? y))
(ac-niltree (append (ar-nil-terminate x) (ar-nil-terminate y))))
(#t (+ x y))))
arc> (eval:list '$ (read:pbpaste))
#<void>
arc> (= + $.ar-+2)
#<procedure:ar-+2>
arc> (time:repeat 1000000 nil)
time: 122 gc: 0 mem: 3256
nil
arc> (time:repeat 1000000 nil)
time: 121 gc: 0 mem: 1880
nil
arc> (time:$:let loop ((n 0)) (if (> n 10000000) 't (loop (ar-+2 n 1))))
time: 310 gc: 0 mem: 776
t
arc> (time:$:let loop ((n 0)) (if (> n 10000000) 't (loop (ar-+2 n 1))))
time: 323 gc: 0 mem: 936
t
What, now the Arc loop goes faster than it did with the Racket +. Probably because this function assumes two arguments, while Racket + assumes any number. And now the Racket loop is only 10x slower than with Racket +. (I expect Racket knows how to optimize its own 2-argument +.) By the way, in case you think the Arc loop goes faster than Racket, note that the Arc loop goes to 1 million and the Racket loop to 10 million; Racket is here going 3x as fast as Arc (and if Racket used Racket +, it'd be going 30x as fast).
Oh, um, I should test the case-lambda thing. I copy to clipboard my definition of + from the Optimizations page:
Virtually no difference from the raw ar-+2. Just for reference, so I can be sure ar-+2 only goes faster than Racket + in the Arc loop because ar-+2 effectively declares that it takes only two arguments:
Good, I'm not going crazy. Anyway, that final version of ar-+2 is my official recommendation.
Oh, incidentally, this "time" thing (though I think it was just "mem-use" back then) also helped me figure out that "<", ">", and "is" were allocating memory (provoking my work behind the Optimizations page). For example, I found that (no x) malloc'd; I eventually realized that this is because "is" mallocs, and (no x) is defined as (is x nil).
Also, here's some further relevant usage of "time":
arc> (time:no:= xs (n-of 1000000 0))
time: 2351 gc: 331 mem: 32513488
nil
arc> (time:no:zap nrev xs)
time: 152 gc: 0 mem: 632 ;hell yeah, nrev works well
nil
arc> (time:no:zap rev xs)
time: 254 gc: 0 mem: 32000584
nil
;now here's when it GC's
arc> (time:no:zap rev xs)
time: 371 gc: 117 mem: 31824248
nil
arc> time:len.xs ;this is my fixed "len"
time: 8 gc: 0 mem: 0
1000000
;now, for comparison, let's see the old definition of "len" from ac.scm
arc> (= len (eval:list '$ (read:pbpaste)))
#<procedure>
arc> time:len.xs
time: 461 gc: 261 mem: 68936656 ;OMG this is ridiculous
1000000
I'm fine with overloaded functions (assuming it makes sense to overload them). The problem isn't that + is overloaded to accept multiple types (that's fine). The problem is that it's trying to fulfill two different purposes at the same time (addition and concatenation), so there's inconsistencies with regard to numbers.
Thus, separating those two behaviors into two separate functions can make sense. In this case, it would be `join` being used for the current concatenate behavior, and `+` being used for numeric addition only. This would also allow us to sanely fix certain behaviors:
(+ 5 "foo") -> error
(join 5 "foo") -> "5foo"
One downside with that approach is that `join` is 3 characters longer than `+`. Oh, and it would require changing `join` so it accepts non-conses, but I would consider that a good change, to be honest. It would allow for stuff like this:
(join '(1 2 3) 4) -> (1 2 3 4)
By the way, I don't think it's a super big deal whether + handles non-numeric types or not. But at the same time, I can see some of the reasoning behind splitting addition/concatenation into two different functions.
P.S. With regard to the performance of a highly abstract Lisp written in itself, running on top of the Racket interpreter: given how slow Arc is in general, I doubt + is going to be the bottleneck.
P.P.S. I believe aw's post was mostly about preserving accuracy in those situations where you can't have any sort of rounding errors. Obviously the default + should behave as it currently does, coercing to the more abstract numeric type. However, in the situations where you need that precision, it might be nice to have a specialized + for that, like +-precise or whatever.
"The problem is that it's trying to fulfill two different purposes at the same time (addition and concatenation), so there's inconsistencies with regard to numbers."
If your numbers were Church encoded, then I guess there wouldn't be a difference between addition and concatenation. XD (This is given that concatenating functions is the same as composing them.)
Yeah, sure, and let's just use lambdas for everything:
(def cons (x y) (fn (f) (f x y)))
(def car (x) (x (fn (a b) a)))
(def cdr (x) (x (fn (a b) b)))
(car (cons 'a nil)) -> a
:P
On a semi-related note, I had an interest a while back for implementing a Lisp interpreter using only lambda-calculus. I'm still not sure if it's possible or not, but I wanted to give it a try.
Maybe we need an official jargon file for arc, given that we're having a lot of confusion over things like "compile time", what to call pg's original arc, etc.
I realize that there was an attempt at starting a wiki, but it requires permission to edit and is somewhat different from most wikis I'm familiar with. Maybe a more open data platform would be more accessible, such as wikia? I would quickly set one up there if anyone else is interested (and aw isn't too offended ;)
Well, http://sites.google.com/site/arclanguagewiki/ is up, running, works, is available for editing for anyone who want to edit it, and I've been planning to continue to contribute to it myself. So... certainly anyone can post material anywhere they want, including a different wiki platform if they like it better, and if it's something useful to Arc we'll include links to it just as we include links to other useful material, whether published in a wiki or a blog post or whatever... but I personally would be looking for a more definite reason to post somewhere else myself.
Is there something specific you don't like about Google Sites?
Some of the things I was looking for:
- no markdown. I despise markdown syntax (when writing about code) with the myriad special syntax that I can never figure out how to escape.
- a way to plug in code examples from my upcoming "pastebin for examples" site, which I hope I'll be able to create a Google gadget to do.
- a clean layout without a lot of junk gunking up the page
Sounds like you're looking for some kind of hygienic macros, and may have rediscovered the logic behind scheme adopting them in the first place.
It is possible that properly handling modules at the core language level requires that either macros are only ever expanded and evaluated in their original context, or hygiene must be used to encapsulate that context in the macro-expansion. Or leave it without restrictions, and hope people follow the guidelines.
Not all hygienic macro systems are cumbersome or complicated, and it's possible that we could create one that permits selective breaking of hygiene as desired. One candidate would be an implementation of SRFI 72: http://srfi.schemers.org/srfi-72/srfi-72.html
Hygiene has been discussed on this forum, and I think a few systems may exist already.
This is all handled by the interpreter, so normal users don't see it. Anyways, now `ret` would expand to this:
(foo@message*)
Which would refer to the correct binding in the correct namespace. Hm... but then what if the macro actually does want to grab a value defined in bar.arc? Tricky.
I think your question is really "Wouldn't name-munging accomplish hygiene?" :)
What you're talking about is pretty similar to how Racket accomplishes hygiene by way of 'read-syntax. Everything's wrapped up so you know what its source file is, and that's all I know for now. :-p Seems Racket's system is pretty cumbersome, but then that's probably because it's very static, with modules revealing what they export before any of the exported values are actually calculated.
What you're talking about also sounds extremely similar to Common Lisp's approach to namespaces. I don't know that approach very well, but it could act as some sort of example. ^^
Traditionally, scopes are formed as trees, and if a name isn't found in the local scope, then the parent is checked and so on.
I see namespaces, environments and scopes as different names for the same thing. Thus when arc is loaded, it would create a default namespace, and load all built in functions into that namespace. It's up to you whether user functions default to a "user" namespace (some languages do this) or default to the root namespace. Any newly created namespaces reference that as their parent, and so on down the tree.
If you do implement a true environment system for PyArc, I recommend you do it this way.
I'd also recommend considering making environments a first class part of the language, which is where you seem to be headed. Reifying environments creates many interesting possibilities, including user-level implementations of module systems, selective dynamic scoping, parameterization, and more control over evaluation in general.
It's already done. As said, modules are implemented, but there's still some bugs to work out.
There's a Python variable global_env that contains the base functions (defined in PyArc) and (once I get it working) arc.arc as well. When it loads a file, it creates a shallow copy of global_env and uses that as the namespace for the file.
Then, within the file, if it uses import, that then calls (new-namespace) which once again creates a shallow copy of global_env.
Actually, PyArc does have an environment class, but it's not exposed to Arc. On the other hand, eval and load both accept tables and alists for an environment. Is that what you meant?
So, I already have all the scaffolding in place, I'm just trying to decide on what name to use. I don't really like the name built-ins* but it does seem to be pretty accurate.
If you're creating a shallow copy and you're just treating a namespace as something that maps names to values (rather than mapping names to bindings containing values), then it won't be as customizable in some ways: When you redefine 'some, you don't get 'all for free, and you don't get to maintain different 'setter tables in different namespaces.
I'm going to great pains reimplementing Penknife so that I create an all new core environment from scratch each time, just so that it can be hackable like that. XD But I'm doing this by passing all the bindings around manually so that core-call's implementation can depend on binding-get's binding and vice versa. The core is ridiculously circuitous, with some 11-argument functions and such. Fortunately, the circuitous part is mostly confined to a single, reasonably sized file, which defines a DSL for the rest of the files to use.
Gotta finish up this reimplementation and post it at some point. XD;;;
Okay, so, my plan right now is that if a function is bound in global_env, it has dynamic scope, and if it's bound anywhere else, it's lexical.
This should allow for my shallow-copying strategy to work, but also allow for shadowing core functions. This may break arc.arc though, but I'll tackle that when I get to it.
I have almost no clue what you're doing, but I hope it works out. Treating core variables differently than others sounds nothing like what I would do, so we're separate people. :-p
Yes, it's ludicrous, crazy, and probably insane, but I'm trying it anyways. I suspect it'll break later, though, so I'll have to go back to lexical-scope-for-everything.
By the way, I called it dynamic scope, but I'm not actually sure what it is. It's this weird short-circuiting thing that causes the global_env to jump back one environment level, which then causes it to loop back onto itself if the variable isn't shadowed, but it works (for now).
Edit: nevermind, I had to revert it. Darn. It was such a silly hack too.
Hm... yes, I may end up needing to change that, or hack it in some way to allow better redefining of built-in functions, while still keeping modules isolated.
Sometimes I agree that . would look nicer, but I also feel that as used currently it has a stronger tradition, simpler meaning and is more "fundamental" and generally useful. I use it all the time with its current meaning, but I only rarely use !. A better choice might be to redefine : instead, as that is commonly used for namespaces. Or provide multi-char ssyntax, and use :: .
I do wish that ssyntax was handled at the same level as other racket reader macros, such as brackets, strings, quote, etc. Then you could do foo.'a or foo."a", something I've desired many times.
Yes, I plan to handle ssyntax at the reader level in PyArc.
In any case, with customizable ssyntax, you can change it however you like. You could even remove the built-in ssyntax, so ' ` , ,@ . ! & : ~ wouldn't work any more, and you'd have to use pure S-expressions. I don't know why anybody would want to do that, but I don't see a reason to restrict it either.
Technically, he ' ` , ,@ operators are reader macros in Arc, not ssyntax. ^_^ A reader macro is more powerful, in a sense[1]: Once a certain sequence of characters is read in, the whole reader behavior can be replaced, putting you in a different language. On the other hand, those particular operators don't need to be quite that powerful, and I'm all for implementing them and ssyntax in a single consistent way.
In Penknife, instead of saying `(a b ,c), I say qq.[a b \,c] (where "\," is just an escape sequence qq implements; Penknife syntax is founded on nestable strings). As long as infix operators are capable of acting on arbitrary subexpressions, any variable can effectively be a prefix operator, lessening the need for heiroglyphics.
---
[1]: From another perspective, reader macros are rather innately limited to prefix notation; I believe it can be overcome in certain cases (http://arclanguage.org/item?id=13888), but it means manually managing a stack of things which could be pushed under the next infix operator to come along. Can't tell if that's ugly at this point. ^^
Oh, by the way, since we're making ssyntax reader-level, I might be able to get more powerful ssyntax as well. For instance, " works by consuming the stream until it finds a matching " Ditto for [] and () etc. This behavior hopefully wouldn't be too hard to add, though I'm not sure what the Arc interface to it would be like.
I probably wouldn't be able to move the defaults for "" [] () into Arc, though, because they call Python functions/classes that I don't particularly want to expose to Arc.
Yeah, I know. Unless I made ssyntax way more powerful, I couldn't put stuff like "" or [] in there. I'm okay with that, at least for now. But since it's possible to put ` ' , and ,@ in the ssyntax, I plan to do that. Makes it more hackable in Arc, you know?
Also, since I plan to expand ssyntax at read-time in PyArc, what's the distinction between ssyntax and reader macros, besides the fact that Arc can't define reader macros, and reader macros are more powerful?
There's nothing stopping Arc from having reader macros too, except that at this point there isn't a good standard; it takes Racket calls, and the more I learn about Racket and reader macros, the more I think it has an incomplete standard too. :-p I want to make a reader macro that stops when it reaches a symbol-terminating character--but wait, there are ways to specify symbol-terminating characters, but I see no way to check for them. Time to hack the language core... if only I could. ^^
"what's the distinction between ssyntax and reader macros, besides..."
I think the distinction is how much you're parsing the stream one character at a time (in which case you can dispatch on reader macros) and how much you're parsing it in chunks. Infix syntax always looks like a chunk to me, but as I was saying, infix operators could be implemented as reader macros if we kept/passed enough state in the reader. There could be no distinction at all.
Then you could do foo.'a or foo."a", something I've desired many times.
Can someone post a summary of how they would like ssyntax to work? Along with getting the syntax to work in more cases, I'm vaguely aware for example that people have preferences as to whether a.b.c should be (a (b c)) or ((a b) c), but I don't know what they are.
