I will take ages until we have a decent, reliable implementation of arc.
Um, in what way is Arc not "reliable" today?
But anyway, so what? Extraordinary results usually take years. If you decide not to do something because it will take "ages" to do, you are probably dooming yourself to a mediocre outcome.
why not get involved in the evolution of scheme
I did. If you you look at the introduction of R6RS where it thanks people "for their help in creating this report", you will see my name listed. But I discovered that while I wanted a personal programming language, the members of the committee had different goals, so there wasn't much I was able to offer.
By reliable implementation I mean something that you would bet your project on. When you choose to use it in production software you have certain expectations from it (stability, performance, etc.)
If the reason to start arc was "a different goal of the scheme community", well, then I guess it's all about politics...
Arc is used in production software. Using some other language will be a better choice if it has some feature or library you need that Arc doesn't have. However, for the features that Arc does have, its stability and performance is fine.
If the reason to start arc
To clarify, I was speaking for myself. If you'd like to know why Arc was started, I recommend you read the essays: http://paulgraham.com/arc.html
I guess it's all about politics
No. If someone else wants to travel to the other side of the world and I want to visit my local coffee shop, it's not politics for them to work on buying plane tickets and for me to work on buying a bicycle.
"No. If someone else wants to travel to the other side of the world and I want to visit my local coffee shop, it's not politics for them to work on buying plane tickets and for me to work on buying a bicycle."
Different people have different goals, desires, and needs. Let's suppose that I was the one who wanted to travel to the other side of the world... it wouldn't make sense to say to me, "hey, plane tickets are expensive! use a bike instead!"
And vice versa... if I were the one who wanted to travel to a local coffee shop, it wouldn't make sense to say to me, "hey, bikes are slow! use a plane instead!"
Two different people have two different needs, and the way that they can best achieve their needs are different. Arc isn't designed to be a production language. You can use it for production, but that's not Arc's goal. There are already other languages out there that are suited for production.
It doesn't make sense to demand that everybody use a "production language" just as it doesn't make sense to demand that everybody use a language designed for exploratory programming.
Thus, it is legitimate to say, "I want to do production work and Arc doesn't seem suited to that task", in which case we can try to recommend other languages that are better suited to production work.
But it's pretty silly to say, "Arc doesn't do production work well... you guys should use Scheme/Common Lisp/Java/etc. instead!" because that's assuming that we want to do production work.
It's not a matter of politics that different people have different desires, and that different languages focus on different desires.
Building a language for production software is always about politics.
My sense of arc is that it isn't concerned with becoming a production language that cares about backwards compatibility. So this is my answer to your question. Why arc? Because it's more permissive than lisp or scheme, and therefore a more suitable testbed for trying out programming language design. Because it is free from the political shackles of production software.
I'm not sure anything will come of this testbed, but I find it enormously educational to ask myself 'what if' questions and play with arc until I figure out why it would be a bad idea to design lisp this way or that.
Maintaining backwards compatibility isn't one of the things I need or am looking for in a production language. If it's something you need for your production system, that's a valid requirement that would cause you not to choose Arc. But don't conflate your personal requirements with what's needed for a production system in general.
To the question "do we have a decent and reliable implementation of Arc for production systems" the answer is yes. People use Arc today for production systems.
Is Arc something that you would want to use for your project? I have no idea. I don't know what your skills are or what your requirements are.
I can offer some general guidelines. Arc is a language for hackers. If you're not not a hacker, then Arc is probably not going to be a language for you.
Arc is a language for exploratory programming. If your project is a straightforward engineering effort with a known solution, then while Arc might not be worse than some other platform it probably isn't going to be better.
Arc is a language designed to maximize the individual productivity of a programmer. It is not designed for programming teams working in a bureaucracy. If your project is large enough that you need a team of programmers to work on it, then Arc is probably not going to be a good choice for you.
And then there are particular features that you may need for a particular project. An easy question to answer is "I need X, Y, and Z, does Arc have that?". Then we could answer:
- "why yes, Arc has X, Y, and Z", or
- "why no, Arc doesn't have Y", or
- "no, Arc doesn't have Y but I think it probably will sometime in the future", or
- "no, Arc doesn't have Y and my best guess is that it probably won't".
"To the question "do we have a decent and reliable implementation of Arc for production systems" the answer is yes."
But arc doesn't get regular bugfixes. The queue bug causes it to segfault, and it was fixed over months without any official support, and it still hasn't been integrated into arc 3.1.
When an outsider asks, "is arc reliable?" he's partly asking, "will discovered bugs get fixed in a timely manner?" I don't want to over-sell arc to outsiders, so my answer is "No."
One of us can maintain a more regularly-fixed variant, but there's several of those and the community hasn't converged on one of them.
At this point we're arguing semantics, which is a futile exercise. So yes, we should just say x, y and z rather than fluffy words like 'production'.
In my experience people who use the words 'production use' are often talking in the context of a system that runs all the time, and that has a team of programmers working on it. In this context there can be lots of x's, y's, and z's that go into 'production', but it always includes "should I be afraid of upgrades?" (roti can confirm or deny in this case.) There's a reason perl and python and common lisp and racket have gone through contortions to indefinitely support bad ideas.
I'm not saying this is a good thing, btw. I think this whole model is wrong. But I'm just one guy shouting at the cosmos, and I certainly haven't seen more than the tiniest sliver of what people do with software.
Arc already is a possible production language. Look at C. It is one of (if not the) most popular language... ever. A massive amount of code has been written in C. C is used to write the OS that you are using right now. C++ (which is similar to C) is used in most web browsers.
Yet, I would hardly call C or C++ very good "production languages", at least according to my definition. A smart hacker with a bad language is probably far more productive than a bad programmer with a good language[1]. Languages can help or hinder, but ultimately it comes down to the programmer.
So, yes, you can use Arc for production work. It may not be designed for it, but it is still possible. It also helps that you can use Racket code alongside Arc code, letting you tap into Racket's power when you need it. If you want a language specifically designed for production work, then Arc is probably not what you're looking for.
But if you want a nice lightweight language that is designed to be very hackable, malleable, and concise, while still giving you the underlying power of Racket when needed, then Arc could very well be what you want.
---
* [1]: Of course, a good hacker will probably despise programming in a bad language, but I still think they would be more productive than a bad programmer using a good language.
Having an open implementation has many advantages:
- if you wanted to serialize closures, you could easily implement a serialization strategy simply by inspecting the closure's environment yourself.
- writing a debugging or tracing facility becomes so simple that you could just throw together an ad-hoc solution to a particular need whenever you wanted, instead of having to rely on the debugging facilities offered (or not) by the runtime.
- for a programmer who is new to closures, having everything be completely transparent makes it easy to see what's going on.
One thought I've had is that it could be useful to write an Arc interpreter in Arc. It would be slow of course... but the highest level functions in a program (like the ones that we typically want to serialize to persist closures) often don't need to be fast... as long as the operations that they call (such as in particular anything being done in a loop) are themselves fast.
This would give us four levels of implementation:
- Arc interpreter
- Arc compiler
- Racket
- C
Each higher level is more expressive but slower than the level below it. Thus you can write something at a higher level first because that will be the fastest for you (it will take you less time to write the program), but then if it turns out to be too slow, you can rewrite it at a lower level at the cost of doing more work.
Your note [3] deserves an entire comment by itself, but I'm out of time now so I'll reply later :)
Excellent notes. Those are definitely great reasons to make functions open, rather than opaque blobs. In fact, I see only two downsides to it:
1) Slowness. Then again, Arc's goal isn't speed either. As you mentioned, it should be possible to rewrite the slow bits in a lower layer if needed.
2) Lack of information hiding. This is more serious than point 1, but, I can provide a different mechanism for information hiding. The downside of having a different mechanism is that it requires learning a new mechanism... and I love that closures can be used for information hiding. It's just so simple and axiomatic, you know?
---
"One thought I've had is that it could be useful to write an Arc interpreter in Arc."
I've been mulling that over too, ever since your earlier comment[1], but... I don't really like the Racket runtime (the implementation, not the language). For instance, trying to make Unix shell scripting work with Racket is a big pain, but it's incredibly easy in Python.
Of course, I don't really like Racket the language either. :P Too big and bloated for my tastes. Why do you think I like Arc to begin with? On the other hand, programming in Racket would probably be preferable to programming in Python, if I had to choose between the two.
Oh yeah, and Python seems to be a lot more popular than Racket... so chances are, your users will have Python installed, but not Racket. Proooobably not a big deal, since we're a pretty small group, but still an advantage for Python.
---
"Each higher level is more expressive but slower than the level below it. Thus you can write something at a higher level first because that will be the fastest for you (it will take you less time to write the program), but then if it turns out to be too slow, you can rewrite it at a lower level at the cost of doing more work."
Yes! And don't forget: profile, profile, profile first. I think that's the best way to make good clean code that is also fast. Honestly, though, I rarely need to actually optimize my JavaScript code, because it's usually plenty fast as-is. Chrome's V8 engine is amazingly fast, especially given how dynamic JavaScript is.
trying to make Unix shell scripting work with Racket is a big pain
In what way? Running a Racket program as a shell script, or calling shell scripts from Racket? (I haven't found either particularly difficult... though I can easily imagine that the process might be smoother and better documented in Python).
with respect to Racket vs. Python, an advantage of Arc's axiomatic approach is that it makes it really easy to implement Arc on top of different platforms. So I can use a Python runtime for things that it is it good at, and a Racket runtime for things that it is good for.
Writing an Arc program that works as a shell script. In other words, I want to slap a #!/path/to/arc shebang into a text file, and then call it with ./foo.arc and have it work like an executable.
This works in Anarki (at least, I think so, I haven't tried it), because they made an arc.sh file, and used some kludgy stuff to make it work[1], but in Python it Just Works(tm), no fiddling needed. That's actually true for Arubic[2] as well, because it's written in Python and I designed it that way. You can just use #!/path/to/arubic and it'll Just Work(tm).
I'm probably overly-criticizing Racket for that because I ended up partially implementing arc.sh by myself, then found a forum post that gave a better implementation, swiped that, then modified it. I doubt I'd have cared as much if arc.sh had been included with Arc 3.1 (and worked good).
It's not as big of a deal now that Anarki has arc.sh, but it's just one of those rough-around-the-edges areas. Python also makes it easy to parse command line switches, with the optparse module, though that could be implemented as a library in Arc, so it's not really fair to criticize Racket for that. :P
I'm curious, though, how would you write a Racket program that worked as a shell script?
---
"with respect to Racket vs. Python, an advantage of Arc's axiomatic approach is that it makes it really easy to implement Arc on top of different platforms."
Not in my experience. Arc actually has a lot of primitives[3], not to mention the whole thread/continuations/TCO thing. Implementing a simple toy Arc interpreter? Piece of cake. Making it actually good? A lot harder. That's another reason I'm trying to shove as much into Arc as I can... the more stuff in Arc, the less stuff you need to write in the interpreter layer.
Actually, a lot of my time has been spent debugging small bugs... I've had to wade through a lot of tiny bugs to get Arubic working properly in all sorts of weird edge-cases. Unit tests help a lot with that.
---
* [1]: Actually, I just took a look at Anarki's arc.sh and it looks pretty clean. I'm not sure if it behaves correctly, but I don't feel like testing it to see.
* [2]: I'm renaming py-arc to Arubic, because I like that name a lot. This also gives me some more flexibility... I'm no longer tied to Python. I could implement Arubic in Arc, or Ruby, or another language.
* [3]: Compared to most other popular languages, Arc has very few primitives, but it still has a lot more than I'd like. One thing I did like, though, is that it was incredibly easy to write a tokenizer/parser for Arc, because it has such a regular syntax, due to being a Lisp.
When I implemented a top-down operator precedence parser (in JavaScript) to parse a custom language, it ended up being a lot harder, because the custom language uses syntax, and not particularly regular syntax either.
Of course, to say that Arc doesn't make this easy enough is certainly a valid criticism. (But I don't think we can blame Racket for that).
not to mention the whole thread/continuations/TCO thing
Your point is well taken. The advantage of an axiomatic approach is once you have them implemented, the rest of the system will run on top of them... but the powerful axioms may be hard to implement.
Fair enough. I had somewhat assumed that the problem was Racket, because Arubic handles it just fine. My mistake.
That actually looks like a pretty good command line parser. Thanks for the links!
P.S. Once ar gets into a somewhat more complete state, you may want to consider making shell scripts work good with it. For instance, if I'm in the directory /foo and I run a script that's in /usr/bin and the script uses (load "lib/bar.arc"), I would expect it to load /usr/bin/lib/bar.arc, not /foo/lib/bar.arc.
I made this work in my copy of pgArc, but it'd be nice to have it be more standardized. Obviously it should be possible to load relative to the current directory as well, if your script wants to.
Mind if I go in and start hacking on ar? :P If I decide to make an Arc compiler in Arc, I'd probably base it on ar.
---
"The advantage of an axiomatic approach is once you have them implemented, the rest of the system will run on top of them... but the powerful axioms may be hard to implement."
That is true. I do like the axiomatic approach, but I think Arc can improve and become even more axiomatic, which is one of my goals with Arubic.
Oh, and just so you know the above example produced by the hackinator incorrectly uses aload to load Arc files after arc.arc has been loaded instead of using Arc's load; this a known issue I've listed in the to-do list in the README.
another idea: you could go on the Racket dev email list (http://racket-lang.org/community.html) and say "as a learning experience I've implemented a gc for a small Lisp and was thinking about what to do next such as perhaps trying a tri-color algorithm. I'm curious what gc algorithm you're using for Racket and why you chose it over other alternatives?"
Thanks for advices, I've read some papers on this topic and will take a deep searching into different scheme/lisp groups in following days.
For now I decide to refactor carc to an byte-code one and use the new gc mechanism from the scratch :P
If you'd like, something that would be helpful to us for you to keep an eye out for while you're looking is that we'd like to find a runtime (or learn how make a runtime) which:
- runs on multiple cores, AND
- supports full call-with-current-continuation, tail call optimization, and mutable lists
Our current options include:
- Racket, which can run on multiple cores, but only supports mutating lists within a single core, and
- the JVM, which does support mutating data structures by multiple cores, but doesn't natively support full continuations or tail call optimization.
ok, I think in order to give advice I'll need to understand your goals better. (After all, I could say "you should do X", but if X doesn't lead you to accomplishing something you want, than my advice wouldn't be very useful).
When you say you want to make an Arc compiler, do you mean that you that you'd like to create a compiler that implements all of Arc, and so it could be used for example to run a news forum like Hacker News on?
Or, are you for example primarily interested in learning, and if so, what would you like to learn?
Any goal you have is valid, but it's relevant for your question about gc. For example, one of the choices you have is whether to not support threads, or to support threads but only on a single core, or to support threads on multiple cores. Which one you choose has a major impact on how to approach gc: it determines both what algorithms you can use and also what algorithms you want to use.
This project starts with the purpose of making an S-expression language which can be easily embedded into another project.
Since I really like arc's sugar-syntax, I would like to implement all language specification which arc has, and make it extensible and easy to embed.for above reasons, I would rather do not take any kernel-level threading techniques, but with some user mode thread and in-background dispatcher to support async socket io. Currently, it have tail recursion optimization, continuation, macro and a mark&sweep garbage collector.
Overall, you're right, it's primarily a project of learning how to make a language.
Just so I understand your terminology, by "user mode thread" do you mean that you will have threads at the Arc language level, and that they will be implemented by your runtime by having your C code itself switch between threads? (And thus the Arc language level threads will all run inside of one operating system thread?)
One thought that occurs to me is that the next thing that might be most useful is to have a way to easily see the performance of the garbage collector in a program you're running. Then when you notice "oh look, Arc is allocating a large number of short lived small objects" or whatever you can tune your garbage collector (or look for a gc algorithm) that works well with the pattern you're seeing.
I don't know enough myself about the different gc algorithms to make a suggestion. All I could offer is a wish-list of what we'd like Arc gc to be able to do, which perhaps you might find useful if you went to an algorithms forum or mailing list and found out what gc algorithms might be able to do that.
I'm not able to look at your source code because your copyright notice (it looks like maybe it came from a template?) says "all right reserved" without a notice of the code being released under an open source license. If your code is derived from Arc then you may need to release it under the Perl Artistic Licence (the same as Arc); or, if it's not, but you'd like to release it under an open source license, then the MIT License (http://opensource.org/licenses/mit-license) is a good choice to consider.
It's right since the header files are all created in Xcode 4 with the default template. I will make it under an open source license after reading both licenses. Thank you for the advices!
(for future reference, that is, without speculating on why or why not someone may have made decisions in the past, but if someone runs into a similar problem in the future)
If:
a) you're running a web application on top of Arc, and
b) you have some very long pages that are taking a long time to render, and
c) it's a page that you actually want to be able display to your users
...then there are less drastic alternatives to kicking the page off the front page:
- you can increase caching (at the cost of the displayed page being a bit more out of date)
- break the page up into parts and add "next" and "prev" links
Knowing the code was written with a news site in mind, you could also theorize (or speculate, which ever makes you happier) that the author (aka pg) would value both the quality and currency of the content over pouring coding efforts into handling of overly active, and by that time, older posts.
All speculation of course. I'm just suggesting that, often, there are simple reasons for the choices made behind these kinds of decisions.
All the ideas you list are great ideas. It's funny though, even I spent a bunch of time writing a blogging platform and I still have not bothered to put paging into the listings. I figure at my current rate of posts, it will take me a year before I'll need to worry about it. There are things we should do, in our ideal world, and then there's what we need to do relative to all other things. So, again, I suggest that maybe the reasons for these kinds decisions can often be much simpler than we think. There's not always some elusive statistical computation with all kinds attention and detail poured into them.
[edit1] ignore the below point... point format tricked me up reading on my ultra small screen :)
I don't fully understand what this point has to do with anything:
I like the work you did, but how does this fit into the discussion or your comment? It seems out of place.
[edit2] Also I used to like the fact that this forum allowed for editing within the first hour. I could correct my mistakes before, I figured, anyone had time to even read them, lol... but, that's not working anymore and trying to respond to changing posts kinda sucks.
I wish this forum had a "save as draft" feature! Maybe I'll just write it somewhere else first.
I don't fully understand what this point has to do with anything: - provide a runnable example
If you have a web application that is running too slowly and you want help, then you make it possible for other people to help you if you provide a runnable example that demonstrates it running too slowly.
Oh, nice, you have addressed exactly what I was asking for. Hmm, but question... Why not do this? I mean, you mention that it's not the only way, but this seems to work and doesn't require anything like gensymming.
I guess the advantage of your approach is that you can choose to only import the functions you want. If that's particularly important for some reason, then, sure, do it. But I suspect it might actually be useful to import the entire module, especially at the REPL when you're just starting out using it. Particularly if you have a "scheme apropos" function.
I added the "sort <" part just now, by the way. Can't believe I didn't think to do that originally (though I did at least add it to the "aps" thing somewhat earlier).
There was something or other that wasn't working when using $ directly because it was Racket-specific and it didn't work in Arc's mzscheme module. I forget now what it was though. (Clearly parse-xml is a bad example as it does work with a simple require).
I haven't looked at the code, but on Hacker News a story with few upvotes but zillions of comments can be indicative of an unproductive flamewar. We probably have different dynamics since our discussions are more technical.
