Arc Forumnew | comments | leaders | submitlogin
Separation of concerns
4 points by krapp 2320 days ago | 61 comments
Does it bother anyone else that the forum code is in the same directory as the library code?

Should we have a separate location for applications and, potentially (assuming a future where namespaces, modules and dependency management exist) vendored libraries?



3 points by hjek 2320 days ago | link

I'll just quickly give my unrequested view about your commit[0] changing the directory structure, and again I am sorry if I come across very negative, but I'll try to be constructive in my criticism:

- It breaks anything relying on `run-news`. For example if you have Anarki running as a systemd unit, you'll have to fix that.

- It breaks existing News sites because it expects the `www` folder to be in a different location.

- It creates an overcomplicated folder hierarchy for the Anarki package manager, e.g. if you add a custom app through prompt.arc, it will now be in `anarki/apps/news/www/apps/[user name]/[app name]` rather than `anarki/www/apps/[user name]/[app name]`. Is it really necessary to have another `apps` folder inside the `apps` folder? Why?

- Perhaps when new stuff is added, it is fine to break things, but what new features does this commit bring, or what problem does it solve? I am not sure what this ongoing shuffling around paths and files really helps, but it sure does break stuff. Would it not be more interesting to work on new features for News or new programs in Arc than bike shedding existing code?

I think you should really just check out the existing Arc package manager, prompt.arc. It does have a way of separating user apps from the rest of the Anarki repo. It can be used from within News, but perhaps it needs to be documented more in the README and shown more prominently in the News interface?

I am not saying what commits should stay or go, as the idea with Anarki is that it's open for everyone's commits. However, I do find this folder shuffling somewhat disagreeable, so perhaps I should just keep my own Anarki fork?

Again, sorry for being negative here. Am I being unreasonable?

[0]: https://github.com/hjek/anarki/commit/4d5f5dc7c7e0ce1048cb65...

-----

4 points by krapp 2320 days ago | link

>It breaks anything relying on `run-news`. For example if you have Anarki running as a systemd unit, you'll have to fix that.

Fair enough, although I would point out that, given that this is a language designed for experimentation with an open commit policy, breaking changes should be tolerated and expected if the end result improves the codebase, which in this case I believe it does.

Given that is not a mature language or a stable framework yet, I would disagree that breaking changes, per se, should be avoided as a matter of course. We should be breaking things harder and more often.

And... people just have to fix it once. No one is suggesting a new hierarchy every week. Literally, just that applications should live somewhere other than /lib.

> It creates an overcomplicated folder hierarchy for the Anarki package manager, e.g. if you add a custom app through prompt.arc, it will now be in `anarki/apps/news/www/apps/[user name]/[app name]` rather than `anarki/www/apps/[user name]/[app name]`. Is it really necessary to have another `apps` folder inside the `apps` folder? Why?

The package manager should be updated as well, mea culpa. But that demonstrates that the forum should have been moved to begin with, since it didn't conform to that convention. Also /anarki/www/apps/[user name]/[app name] is more complicated than the convention I want to use, which is /anarki/apps/[app name].

> Perhaps when new stuff is added, it is fine to break things, but what new features does this commit bring, or what problem does it solve?

It solves the problem of the forum being in the same path as the library. Every other language with a web framework separates one from the other.

>Again, sorry for being negative here. Am I being unreasonable?

For the criticism? No, I'm the newb, my last commit was kind of garbage, I can take it and I welcome it.

For wanting to give up and switch forks because I moved some code into a new folder? I think that is a bit unreasonable. Try to meet me halfway here, I'm not just breaking stuff for the lulz.

-----

3 points by akkartik 2320 days ago | link

The risk here is that everytime a newcomer tries out our codebase and finds it broken, we lose a potential recruit. And we don't have so much incoming attention that we can afford to be lax with it. Our situation is akin to a store that is mostly empty: we can't afford to just shut down longer, because that way lies death. We have to stay open for business.

Not allowing in obvious breakage is also a way to show respect for your collaborators, other people programming on the same codebase. When you come to Arc you would be demotivated if the news app didn't work, if you had to first get it working, figure out who broke it and how. It's the same with others. When we find a few hours to hack on something fun, we all prefer that it start out in a non-broken state. And we try to leave it in a non-broken state.

So I have a suggestion for you: everytime you create a PR, first try to break it yourself. Ask yourself what changed, and what it may affect. Try to catch the really obvious stuff yourself, like "does news load, does it look the same, can I submit a post?" Run the unit tests. Report in the commit message or PR what all you did by way of testing. These exercises will make you a better programmer. Tell yourself that the goal is to make changes in an efficient manner, solve the problem in a single PR, rather than require more changes to fix breakage in one change, that cause their own breakage, and on and on.

None of this is particularly serious. It's not like any breakage here is a deal-breaker. As long as others see you making an effort to get better I think you'll get a lot of support.

-----

4 points by zck 2317 days ago | link

> Run the unit tests.

Certainly a lot of things could be tested more; this would help.

But also, we might want to set Github to require the tests to pass on a PR before letting it be merged. I could be wrong, but I think a PR can be merged even if the tests fail.

This is a slight step away from "anyone can commit anything", I acknowledge. But it's also a step towards stability. At least the person would have to fix or delete the failing tests before merging.

-----

5 points by rocketnia 2317 days ago | link

Fixing other people's test breakage is one of the only things I can do for Anarki that doesn't make me feel like I'm breaking things myself. :-p

-----

4 points by zck 2315 days ago | link

Well, it's a great help! Writing tests can help too, but just being around and helping is wonderful.

-----

3 points by krapp 2320 days ago | link

Ok. I will try to be more careful in the future. In fact, I won't even merge until someone looks at the commit, fair enough?

It isn't broken, though, at least not if "broken" implies an unusable state. I did test the forum before I pushed, I just didn't catch everything... I didn't even know prompt.arc existed, or what it was for, but apparently neither did you. ;)

But news does still work. It just doesn't work from the same location as previously, which I think is hardly setting fire to the store.

-----

2 points by i4cu 2320 days ago | link

For what it's worth I agree with the app/news changes. So Good Job I say.

And with that said, after looking at the anarki directory it's apparently becoming a dumping ground for people to place files at the top level. I'm not sure why this is happening, but I hope others take a look at what you have done with app/news and continue the trend of organizing things better.

-----

2 points by akkartik 2320 days ago | link

Ah, I see, it was just the `run-news` script that was broken. Yes, I can imagine accidentally breaking that myself :)

Thanks for being agreeable to the requests of others. In the end, that's all any of us can do.

-----

2 points by hjek 2320 days ago | link

> We should be breaking things harder and more often.

That line of thinking is completely incompatible with my requirements as I'm using News for my own web page that sort of needs to work. Fork time, I guess.

-----

4 points by i4cu 2319 days ago | link

Another option is to have stable branches/tagged releases.

As for "breaking things harder and more often". I don't think 'harder' has to happen 'often'. I'd be ok with 'harder' when need be with more 'often' being good.

This language has 3-6 people active users and maybe another 6-10 lurkers. More changes like the app/news change have potential to garner more users and more users means more changes which may lead to a fuller and richer language.

Having the news app so heavily integrated was a poor design choice to begin with. This is a language and the language development should not be constrained by a single application which is currently the case. That app needs to be a downstream consumer of the language.

Ideally we can get to a point where you can just repo your own app and marry it with a tagged release while collectively the app users can report any breaking changes.

If we don't do this the language is dead for all but a hand full of hardcore arc lovers + news appers.

-----

3 points by krapp 2319 days ago | link

I think a stable branch is a good idea. People can experiment elsewhere while not stepping on anyone's toes.

>Ideally we can get to a point where you can just repo your own app and marry it with a tagged release while collectively the app users can report any breaking changes.

Currently, it looks like "apps" are mapped to an admin namespace and will only work for that user, so that would need to be changed to make them portable.

-----

2 points by hjek 2316 days ago | link

> I think a stable branch is a good idea. People can experiment elsewhere while not stepping on anyone's toes.

I think it's possible to experiment with things while still considering stability and backwards compatibility. I recently reworked the password hashing function, but I took care to not break the logins of every user, when doing that.

I must say I'm a bit upset by you being so apparently casual and careless about creating several breakages[0][1][2] in News that I've had to fix, and then suggesting that I should just get off the master branch if I don't like your "experimentation".

I will. I'm done with this.

Experimentation is not the same as carelessness. Seriously, WTF is this?:

    (abs-link site-url* "anarki" "http://github.com/arclanguage/anarki"))
> Also, having form urls be absolute should prevent browsers from complaining about the insecure field.

[0]: https://github.com/arclanguage/anarki/issues/105 [1]: https://github.com/arclanguage/anarki/issues/104 [2]: http://arclanguage.org/item?id=20527

-----

3 points by rocketnia 2316 days ago | link

Please be patient with krapp, hjek.

You also made a bunch of breaking changes the repo a while ago. Because of those, I got more of the tests running under the Travis CI script to help us all monitor them without having to ask each other to remember to run them all the time. I really hope I didn't make you feel ashamed about that. You got to make changes, and I got to make changes, and I felt like the repo was better with both of those together. The same is true here.

-----

3 points by hjek 2315 days ago | link

I did break some things in Anarki, and I appreciate you spending time helping me fix those breakages and adding more automated tests. Thank you.

What bugs me here is not so much that stuff gets broken now and then, but this nonchalant attitude to breaking things, and lack of acknowledgement for others that have to wipe up the mess.

> And... people just have to fix it once.

> Given that is not a mature language or a stable framework yet, I would disagree that breaking changes, per se, should be avoided as a matter of course.

Also, I have a strong aversion to CAPTCHAs, partly because they're non-free Google SaSS, but also just because they are just plain annoying. Why would anyone want to inflict that upon themselves? There's already plenty of ways to deal with spammers in News.

And, quite unrelated, I'm not big on Github as their site requires non-free JS, and I'm particularly wary of them since they were bought by MS, so that is just one more reason to host the code elsewhere. So, I've forked Anarki, and we'll see what happens: https://notabug.org/hjek/knark

-----

3 points by krapp 2315 days ago | link

>What bugs me here is not so much that stuff gets broken now and then, but this nonchalant attitude to breaking things, and lack of acknowledgement for others that have to wipe up the mess.

You don't have to wipe up the mess. I didn't ask you to wipe up the mess, and I didn't expect you to. I also suspect you're conflating my beliefs about whether or not Anarki should be willing to make breaking changes when necessary with some general lack of regard for whether or not anything works at all, and that's not the case.

I'm new to lisps, to this lisp in particular, to this framework, and to working in a team. I admit (and have admitted) that I make mistakes and I've picked up some bad habits that I need to correct. If I know about an issue, I will gladly deal with it.

But, as to the specific issues you listed... I don't know what happened with the css (it worked for me when I pushed it but your comment that it might have needed testing in other browsers is correct,) but #104 wasn't broken, but misconfigured. And you've explained at length already why that entire commit was misguided, but it wasn't broken.

But sometimes I do miss things, forget things, there are some mistakes I might not even see because I'm working on Windows. But I am trying to make things better here. So thank you for cleaning up after me, and I will try harder in the future.

That said, I stand by moving news, and I stand by my statement that we should be making breaking changes (with the caveat that those changes improve the code.) Now is the time to do that, when there are so few people actually using this codebase, as opposed to later when there might be dozens or hundreds, or never, simply because change is difficult to deal with. Chances are most of the entire universe of people using this fork in production are already in this thread. Moving news to a new folder shouldn't be a reason to create a hostile fork.

And that's why I agreed with i4cu's suggestion earler that a stable branch might be a good idea... which for some reason you interpreted as a personal attack,

    "(...) and then suggesting that I should just 
    get off the master branch if I don't like your "experimentation"."
Rather than more charitably, that you should stay on the master branch and the unstable branch should be elsewhere, as I was actually conceding your point and agreeing with what I thought would be an equitable solution.

-----

2 points by hjek 2313 days ago | link

> You don't have to wipe up the mess. I didn't ask you to wipe up the mess, and I didn't expect you to.

I agree. That was my own expectation, and also I might have been wrong in expecting stability from a repo that's meant to be "extremely permissive in accepting patches."

> And you've explained at length already why that entire commit was misguided, but it wasn't broken.

That I don't agree with. I would consider a commit, that makes all links not work in the default config and prepends the site name to already absolute links, broken as well as misguided.

> But I am trying to make things better here. So thank you for cleaning up after me, and I will try harder in the future.

That's great. It's good to see work done on Arc. Looking forward to see where this goes.

> Moving news to a new folder shouldn't be a reason to create a hostile fork.

Let's just call it a fork, not a hostile fork, shall we? Also there's some other reasons listed in the readme of the fork.

-----

3 points by akkartik 2315 days ago | link

I agree that we should hold ourselves to some standard about not breaking Anarki for each other.

It's our job, yours and mine and that of other long-time participants, to explain the norms that we have here. And to do it in a nice way, realizing that what's obvious to us isn't necessarily obvious to newcomers.

This is a pretty non-standard repo where anybody can make changes, so it's understandable that a newcomer may think stability is not important. We have to articulate precisely what sorts of stability are necessary and what we don't consider important. It's a tough job. So I really hope you stick around to help us out :)

I'm actually curious how you run a production service on Anarki. Everytime I ever tried to do that I would maintain a cutout in between that was wholly in my control, and decide what patches should flow from Anarki to it.

-----

2 points by hjek 2314 days ago | link

> So I really hope you stick around to help us out :)

Definitely!

> I'm actually curious how you run a production service on Anarki. Everytime I ever tried to do that I would maintain a cutout in between that was wholly in my control, and decide what patches should flow from Anarki to it.

Usually I'm trying out the latest commits on a local laptop before doing git pull on the server. Works fine. If it's broken for some reason, then I'll just do a `git reset --hard [known-good-revision]`, because the `www` folder with all the news data is outside the git tree anyway so it stays as it is (and no-one have yet been changing the data format, so that's not an issue either.)

I find it a lot simpler to manage than for example Wordpress (although Wordpress on the other hand has tons more featuers and plugins).

-----

3 points by i4cu 2315 days ago | link

> Also, I have a strong aversion to CAPTCHAs, partly because they're non-free Google SaSS, but also just because they are just plain annoying. Why would anyone want to inflict that upon themselves? There's already plenty of ways to deal with spammers in News.

Languages allow people to make their own choices on what is worthy to build. Debates like this are why it's a shitty idea to have apps baked into the language.

-----

3 points by hjek 2314 days ago | link

True. The tight coupling between Arc and News can in some cases be a problem.

It goes a bit further than the folder layout though, and I find the border between app.arc and news.arc a bit murky.

-----

4 points by i4cu 2314 days ago | link

> I find the border between app.arc and news.arc a bit murky.

It could just be the name that's offsetting you, but app.arc is an application library just like srv.arc is, only manages application ops/state instead of server ops/state. The best way to look at it is that app.arc, like srv.arc, can optionally be used by different applications (like news.arc, blog.arc or whatever), which is why it's under lib. Someone who wants to build an app, needs to know what libs can be used and those can be. They also need to know that news.arc is not a library to to use for a new shiny app they want to build and when they want to share their app where to put it. Right now the only place is under app/, and ideally (IMHO) we can move them out further someday.

-----

4 points by hjek 2313 days ago | link

Agreed. Yet it is also possible to imagine apps that could benefit from tighter coupling with News, e.g. to access user 'karma' and to integrate into the layout and navigation of News.

-----

3 points by i4cu 2313 days ago | link

> to access user 'karma'

That, to me, is a good argument for a karma.arc library.

> to integrate into the layout and navigation of News.

which is an app. :)

-----

3 points by hjek 2313 days ago | link

Interesting idea.

Also, the password reset email function in app.arc requires `this-site` to be set, but `this-site` is set in news.arc. I'm not sure how I'd go about fixing that.

-----

4 points by i4cu 2312 days ago | link

I hadn't realized that arc could even point to an unbound variable and still have the file load without errors. Oh my.

In Clojure there are namespaces and I can rebind vars like such:

  (alter-var-root #'app.arc/site-url* (constantly "http://news.example.com/"))
So I could just put values (like site-url*) into app.arc with default settings and reset them from news.arc. Provided that it's utilized dynamically then all is good.

However lacking this functionality in arc I would just move the required config settings to app.arc and be done with it. You're still loading app.arc at the top of news.arc so the only real problem is app users need to know to change the settings therein (a comment would do).

-----

2 points by krapp 2313 days ago | link

>which is an app. :)

News already has a repl and "apps" in prompt.arc, and I'm already working on getting them to work together. That could provide the basis for a plugin (app, what have you) system.

-----

2 points by i4cu 2319 days ago | link

> I think a stable branch is a good idea. People can experiment elsewhere while not stepping on anyone's toes.

I think people are already experimenting elsewhere even if its only their local machines.

The real problem is who is going to manage changes. As I understand it, the reason it's open is that no one is willing.

-----

3 points by akkartik 2320 days ago | link

I hadn't tried running it yet. You also make some good points that I hadn't immediately considered just from scanning the change. I wasn't aware of prompt.arc, for example. Thanks.

-----

3 points by hjek 2320 days ago | link

It is indeed a problem that Arc doesn't have private module scope. Currently you get in trouble when two libraries have a variable with the same name.

Rocketnia has done some work on namespaces in anarki/lib/ns.arc but I haven't had too much of a look at it yet. I would personally appreciate if there was a little bit added to the README about how those work.

I'm not sure what you'd consider problematic about news.arc being in the lib folder, but if you'd like to put it in another folder, then why not? (But also: why?)

-----

3 points by rocketnia 2318 days ago | link

Have you seen lib/ns.arc.t? If you want more examples than that... could you give examples of what examples you'd like to see?

Keep in mind that if you use ns.arc to load an Arc library in a namespace, you'll usually have to work around problems with unhygienic macros on a case-by-case basis. I don't think this would typically be any easier than working around whatever problems were preventing you from loading it in the main namespace.

When I made ns.arc, I did make a few changes to Anarki so it would cooperate a little better with Racket namespace manipulation. When I made Anarki fit into the Racket ecosystem with `raco install anarki` and `(require anarki)` and `#lang anarki` support, I made several more changes that made it possible to do things like load more than one instance of the Anarki built-ins in separate Racket namespaces. However, ns.arc is still primarily a library for manipulating Racket namespace objects. Any use it has as a library isolation tool is still aspirational.

-----

2 points by krapp 2318 days ago | link

One persistent problem is that the HTML macros have to work around everything else, and as a result, the implementation is haphazard.

You can't build an HTML table with (table) for instance, it has to be (tab)... but just ignore that (tab) doesn't print \t. I accidentally broke things adding a (body) macro because other macros were using that for expansion. The best possible world would be one in which every HTML macro matched its corresponding HTML tag, or if Anarki supported XML syntax natively. Without some kind of namespacing or scoping the former would be impossible, and I suspect the latter would annoy Lisp purists.

-----

2 points by rocketnia 2318 days ago | link

I'm really glad you have an opinion about what the HTML library should do. Without bold, breaking commits like yours and hjek's, Anarki doesn't go anywhere. They're pretty much the only way Anarki gets maintained at all.

What you're saying would make sense if html.arc were a self-contained library for public use. Why does it have shortcuts for some HTML tags but not others? Someone in the public might want to use those others!

I think I'd approach this differently: Every client of html.arc can already say (tag foo ...) or (tag (foo ...) ...) to write an HTML tag. Verbosity is sometimes valuable; it can let us grep the codebase for something like "tag" to find all the places HTML tags are generated. So if the shortcuts of html.arc are causing distress over their inconsistency, I would remove all of them... from html.arc, at least.

Sometimes brevity is valuable too. It helps us fit a bunch of code on one page, for instance. So those things I would remove from html.arc could still belong in another file. Or maybe a few other files; sometimes utilities are self-consistent on their own but are inconsistent when viewed as part of a group.

I think your vision of how these shortcuts should work is self-consistent, and it would even work well as a Racket library. In Racket, we can use `(require (only-in ...))` or `(require (except-in ...))` to require some identifiers from a library without getting identifiers that clobber the ones we're using for other purposes. And we can use `(require (prefix-in foo: ...))` to require a library's entire set of exports under a prefix.

Anarki doesn't have Racket's `only-in`, `except-in`, or `prefix-in` abilities; it only has the ability to load everything in a file. But Anarki users can achieve something similar: The library itself can use more verbose names like `body>` or something, and users can write `(= body body>)` if they want to get rid of the verbosity.

---

I think I've actually seen that `body>` naming convention somewhere, but I can't find it. While I was looking for it, I noticed a similar level of brevity in almkglor's whtml.arc on the old Arc 2 Anarki branch:

https://github.com/arclanguage/anarki/blob/arc2.master/whtml...

  (w/html
    ('head ('title (prn "Page")))
    ('(body style "font-size: 200%")
      (prn "HELLO!")))
Scoped DSL techniques like this one can be a nice way to get some greppability (searching for "w/html") while also getting some brevity (shaving three characters off of "tag " to arrive at "'").

-----

1 point by krapp 2317 days ago | link

>What you're saying would make sense if html.arc were a self-contained library for public use.

The thing is, it is, whether it's meant to be or not. Arc's documentation describes it under the heading "html generation" (in general.) It's in what appears to be Arc's "standard library" /lib, so any newcomer is going to see it there and think that's what they should use when they want to generate html. And it has the rather canonical-seeming name of "html.arc."

If html.arc isn't supposed to be Arc's general purpose library for HTML generation then what do we call it when we make it?

I think discussions about APIs and interfaces are missing the confusion (which, admittedly, may just be mine) over what exactly /lib is supposed to be for?

>I noticed a similar level of brevity in almkglor's whtml.arc on the old Arc 2 Anarki branch:

I really like it... but wow am I not capable of even reading that code ._.

-----

3 points by rocketnia 2317 days ago | link

"Arc's documentation describes it under the heading "html generation" (in general.)"

That documentation is for Anarki Stable, the branch meant for users who aren't thrilled with Anarki's changes and just want to use official Arc, but who would rather not have to fix official Arc's known bugs on their own. On that branch, html.arc probably is the only HTML-generation library we can or should document.

Despite not being written as documentation for the Anarki master branch, it's nevertheless some of the most accessible documentation for that purpose. I see that's pretty frustrating. It's good that you're mentioning it or no one would work on it.

Yesterday, since this was on my mind thanks to these threads, I wrote some code for generating a simple HTML page based on Anarki's (help foo) documentation. Once I get this to automatically deploy with each new Anarki commit, it should give us a much more up-to-date documentation resource to link to.

---

"any newcomer is going to see it there and think that's what they should use when they want to generate html"

It pretty much is! Even if it's not as good as you can imagine it being, it's still the go-to HTML generation library for Anarki.

When I implied it wasn't "a self-contained library for public use," what I meant was that it was primarily made to get the webapps to work. It's quite a non-minimalistic approach to HTML generation if you ask me -- I'd rather treat every attribute uniformly rather than having a big table of special cases -- but in other places it does only what it needs to do for the purposes of those applications.

Paul Graham pursued the web server code in order to put pressure on the "core language" in arc.arc, to make sure he was focusing on operators that were actually good for something. He released the language to the public so he could put a wider variety of pressure on arc.arc. The operators in arc.arc were never finished, so html.arc was probably a long way from being finished itself.

---

"If html.arc isn't supposed to be Arc's general purpose library for HTML generation then what do we call it when we make it?"

If someone makes a new library for HTML generation, we can potentially embrace that as the new html.arc. Anarki is unstable enough that this would be pretty normal (although I think people have had a polite tendency not to delete something in case someone else was still attached to it).

---

"[paraphrased] What is /lib for?"

Personally, I'm reluctant to call it a "standard library" without a standards document, but it does "come standard" with Anarki, so...

Of the 14 files in the release of arc0.tar, the libs.arc file contained this:

  (map load '("strings.arc"
              "pprint.arc"
              "code.arc"
              "html.arc"
              "srv.arc"
              "app.arc"
              "prompt.arc"))
So essentially half of the files in the first Arc release were essentially lib/ files that just hadn't necessitated their own folder yet.

Looks like there was always a lib/ directory in Anarki, even as far back as the first Git commit. At this point it's the obvious place to put files that some people might be interested to (load ...) and some people wouldn't care about. If someone has a single .arc file to share, dumping it there is easy regardless of what state it's in, and it usually fits right in.

---

"I really like it... but wow am I not capable of even reading that code ._."

Aw, I bet you could! :) Do you want to? Maybe if you've got a specific piece of code that perplexes you, we can help out.

Anyhow, one of the only reasons so many things are still stuck in the "arc2.master" branch is that porting them would be too much work. It could be that whtml.arc's time for porting has arrived. :-p

-----

2 points by krapp 2317 days ago | link

>Aw, I bet you could! :) Do you want to? Maybe if you've got a specific piece of code that perplexes you, we can help out.

I'm used to more verbose languages and the redundancy of heavier syntax. Sometimes Arc reads to me like a diagram sketched on a napkin.

>It could be that whtml.arc's time for porting has arrived. :-p

To me it still feels like a kludge for the actual problem, which is a lack of namespaces. Maybe it's time to try out ns.arc?

-----

4 points by rocketnia 2316 days ago | link

Right now, the only things ns.arc can import into local scopes are non-macro values. (Or rather, it can import macro values, but it determines what value they have well after the code it's in has been macroexpanded, so you'll just see them as (annotate 'mac ...) values, and calling them will be a function call that fails with an error.)

What you want are local macros (Common Lisp's `macrolet`, Racket's `let-syntax`). Suppose we name Arc's equivalent `w/mac`.

Then (w/html ...) could expand to this:

  (w/mac html tag-html
         head tag-head
         body tag-body
         ...
    ...)
To implement `w/mac` requires changes to the macroexpander in ac.rkt. I don't know how much this will make sense to you, but this is the approach for anyone who wants to take it: It requires a change to the way `ac-call` looks up the macro's value (`ac-macro?`) so that it can look it up from the local scope (the `env` parameter). This in turn means the local scope value `env` needs to be refactored so it's a hash table rather than a list, and then `w/mac` can look it up from that table. The `env` parameter isn't passed into macros, so either `w/mac` would need to be implemented as a special form or we'd need to come up with a way that macros can optionally declare that they want to receive an `env` parameter.

This would be something of a breaking change to Anarki because it means Anarki will now allow a local variable to shadow a global macro. That was the problem you encountered with (body) in the first place; some calls to local variables were expanding as macro calls now because the macro lookup ignored local variables altogether.

There might be some code out there of the form

  (let foo 1
    (bar foo))
where (bar ...) is a macro call that expands to (foo ...) for some global macro `foo`. This code would now fail.

For some time I've thought it would be worth it for the local macro support, but I'm too afraid to make the breaking changes.

Nevertheless, a while back, there was a consensus here that the way things worked already was a bug. See "Global macro names take precedence over local lexical variables" at [1]. I don't remember if this bug was ever fixed in Anarki; it doesn't seem to be fixed now. (Maybe there was another consensus to reverse it? It could happen.) It's the very same bug you encountered with (body) in the first place.

[1] https://sites.google.com/site/arclanguagewiki/arc-3_1/known-...

-----

2 points by akkartik 2316 days ago | link

lib/tem.arc was extracted out of the original codebase. There may be other examples like that.

-----

3 points by rocketnia 2316 days ago | link

Hmm, it doesn't look like tem.arc was in any of Anarki's versions of the official releases. Didn't you factor out that file?

-----

2 points by akkartik 2315 days ago | link

Yeah, that's what I meant :) but I was lax in my phrasing.

https://github.com/arclanguage/anarki/commit/2820bbf7ee

-----

2 points by rocketnia 2315 days ago | link

I'm not sure what you're saying it's an example of...

-----

2 points by i4cu 2314 days ago | link

Your comment detailed what the libs.arc file contained.

the code within tem.arc was originally part-in-parcel within those files, but was extracted out into its' own file. So he was just commenting that there is other code that exists in anarki that was originally included and that there may be more examples of such.

-----

3 points by akkartik 2314 days ago | link

Right. It's not the case that everything in libs/ that isn't an original filename is new code.

It was just a minor nit :)

-----

3 points by rocketnia 2314 days ago | link

Ah, I didn't even know I was implying that. Thanks for clarifying. :)

-----

4 points by i4cu 2314 days ago | link

Anytime you need us to let you know what you're thinking just let us know. We got you covered.

-----

3 points by akkartik 2314 days ago | link

😂

I was responding to this:

"Of the 14 files in the release of arc0.tar, the libs.arc file contained this:

  (map load '("strings.arc"
              "pprint.arc"
              "code.arc"
              "html.arc"
              "srv.arc"
              "app.arc"
              "prompt.arc"))
So essentially half of the files in the first Arc release were essentially lib/ files that just hadn't necessitated their own folder yet."

But rereading it now I think I was responding to the contrapositive of what rocketnia said :)

-----

3 points by krapp 2313 days ago | link

Is that an emoji?!

-----

2 points by krapp 2320 days ago | link

>Currently you get in trouble when two libraries have a variable with the same name.

Ha. Yeah. (cough (body) cough).

>I'm not sure what you'd consider problematic about news.arc being in the lib folder

It's not news.arc per se, it's everything belonging to the forum (folders, static files, tests, etc) and the general practice of keeping applications in the same place as the standard library (for want of a better term.)

>(But also: why?)

As long as Arc only has a single application (ignoring the blog, which no one seems to care about) then it doesn't matter... but once people want alternate versions of the forum, or other applications, or once remote dependency inclusion becomes a thing, then separating the core from application and vendor code makes things far better organized than including everything into a single directory.

-----

2 points by akkartik 2320 days ago | link

Heh, I really hope there never comes a day when Arc has vendor code. I'd really like for it to continue encouraging people to hack on their dependencies. And that requires keeping the code nearby. So a separate directory for apps is fine, and you're right that it allows versions of the same app to coexist. But hopefully we never start shoving libraries into a system path..

-----

2 points by krapp 2320 days ago | link

I was assuming it would work like PHP where dependencies are just cloned into into a local /vendor folder namespaced by owner/repo.

But that assumes there's a package manager, which assumes there's a consensus and standard around package management, and namespaces, and modules, and... we can just stick a pin in that for now.

-----

3 points by rocketnia 2319 days ago | link

When I saw your change, I liked it. I think it's pretty weird for news.arc to be part of lib/.

I get the impression lib/ was created in Anarki to declutter the root directory, and people weren't sure if news.arc objectively wasn't a library or if they just hadn't thought of how to use it as a library yet, so it went into lib/ and stayed in there like the rest of the less essential .arc files. :-p

-----

3 points by krapp 2318 days ago | link

The dependencies for news (srv.arc, app.arc, html.arc) should be libraries, but they still seem to bake in too many assumptions (in particular, around HTML generation and tables) and interdependencies to be useful as standalone libraries. It's hard to tell where the line is sometimes, and I honestly suspect news and the supporting files were written without considering there should be a line between "library" and "application" at all.

It's also difficult to find documentation on what's in /lib or to know how stable the code is, because no one seems to use most of it.

-----

4 points by akkartik 2318 days ago | link

> It's also difficult to find documentation on what's in lib/

Can you give examples? If you give concrete examples we can try to improve matters.

Many libraries have a long header comment up top, like rocketnia's ns.arc.

Anarki comes with online help. Try saying `(help deftem)` at the commandline. Admittedly online help is patchy inside lib/ but make it a habit to check it if you aren't already.

Lastly, it's worth looking inside the CHANGES/ directory for major features.

> ..or to know how stable the code is, because no one seems to use most of it.

If by 'stable' you mean "we shouldn't change it", then nothing is stable, inside or out of the lib/ directory.

If by 'stable' you mean "working as intended", then tests are our proxy for intent. There's tons of tests under lib/ and I run the tests fairly often. If a library doesn't have tests then I have a lot less to say about it.

So "no one seems to use most of it" feels like an exaggeration. Most code under lib/ is constantly being monitored for bitrot.

> The dependencies for news should be libraries, but they still seem to bake in too many assumptions and interdependencies to be useful as standalone libraries.

I think here you're cargo-culting[1] notions of "good practice" without considering their costs. Not everything that other people use is a good idea for Arc. We have a tiny number of users, which makes it better to keep the community in a well-integrated codebase rather than fragment it across several "lines". Interfaces have a tendency to balkanize users on one side or other of them. We don't want that; our tiny community doesn't really benefit from the scaling benefits that interfaces provide.

If and when Arc grows 10x or something, and we have three different sets of html generation libraries, we can think about drawing better lines. Until then I suggest a different "best practice": YAGNI[2]

[1] https://en.wikipedia.org/wiki/Cargo_cult

[2] https://en.wikipedia.org/wiki/You_aren%27t_gonna_need_it

-----

2 points by i4cu 2318 days ago | link

> We don't want that; our tiny community doesn't really benefit from the scaling benefits that interfaces provide.

Are you sure? After all, these issues are being reported as problematic already [1,2].

[1] http://arclanguage.org/item?id=20490 [2] http://arclanguage.org/item?id=20524 "(cough (body) cough)"

> If and when Arc grows 10x or something...

This could also be a chicken and egg problem. Poor interfaces could be a good enough reason to stay away from Arc.

-----

3 points by akkartik 2318 days ago | link

But those are comments from the same person I'm responding to?

Did you perhaps have a similar opinion as well? I certainly recall other people asking for a package manager.

To be clear, what I said is just my opinion, and I should have said "we don't need that" or "it's a bad idea" rather than "we don't want that", which sounds like I'm speaking for others. If a couple of us feel strongly about adding interfaces, you should feel free to do it and see how it goes. If I'm right then you'll eventually realize that it's only serving as bureaucracy. If you do enjoy it then I'll eventually realize that I'm wrong ^_^

-----

3 points by rocketnia 2318 days ago | link

I think API boundaries arise naturally from syntax that doesn't fit on one screen. Edits to one piece of code can only be quick and easy if they manage to preserve the assumptions that make the rest of the code work. These assumptions form an API boundary, and at some point it becomes easier not to break the code if that boundary is documented and verified.

Currently, we break up the API boundaries in Anarki by function and by file. We document the function APIs with docstrings, and we write verified documentation for them with (examples ...). We verify the file APIs with "foo.arc.t" unit test files.

If we put more effort into documenting or verifying those boundaries, it would hardly be a big change in how we use the language. We could perhaps add a bunch of "foo.arc.doc" files or something which could compile into introductory guides to what foo.arc has to offer (whereas the docstrings still act as API documentation). And we could perhaps add an alternative to Anarki's (require ...) that would load a library in a mode that faster faster, abiding by an interface declared in a "foo.arc.i" file.

It seems like the most drastic change in philosophy would happen if we broke Anarki apart into sections where if someone's working on one section and discovers they're breaking something in another section, they don't feel like they can fix it themselves.

I suppose that happens whenever people put Arc libraries in their own separate GitHub repos. There are actually a number of those repos out there, including my own (Lathe). Even aw, who was a strong believer in avoiding unnecessary black-box interface boundaries, would still put things in various repositories, and the hackinator tool would collect them into one file tree.

It also happens naturally with poor knowledge sharing. Seems like despite my attempts at explaining ns.arc, I'm still the only one with a very clear idea of how it works and what it's good for, so other people can't necessarily fix it when it breaks.

(Come to think of it, that makes ns.arc a great test case for pulling out an Anarki library into its own repo. That'd save other Anarki maintainers the trouble of having to fix it to keep Anarki's tests running.)

Hmm, even if some libraries are maintained separately from the Anarki repo, Anarki's unit tests can still involve cloning some of those repos and running their tests. If Anarki maintainers frequently break a certain library and want to help maintain it, they can invite its developer to fold it into the Anarki repo. And if the developer wants to keep maintaining it in their own repo, the Anarki project can potentially use a hackinator-style approach to maintain patches that it applies to those libraries' code. We have a lot of options. :)

-----

2 points by akkartik 2317 days ago | link

"Hmm, even if some libraries are maintained separately from the Anarki repo, Anarki's unit tests can still involve cloning some of those repos and running their tests. If Anarki maintainers frequently break a certain library and want to help maintain it, they can invite its developer to fold it into the Anarki repo. And if the developer wants to keep maintaining it in their own repo, the Anarki project can potentially use a hackinator-style approach to maintain patches that it applies to those libraries' code. We have a lot of options. :)"

This would be fine. Whether we consider something an API has nothing to do with how we divide code into repositories or sections. Perhaps the key is to know who is calling us. I'd be happy to provide a guarantee to anyone who "registers" their repository with Anarki: if it's easy for me to run your codebase, I'll check it everytime I make a change to Anarki, and ensure that it continues to pass all its tests. If I break an interface, I'll fix all its callers and either push my changes or send the author a PR.

I don't know yet how "registering" would work. But we can start with something informal.

-----

2 points by krapp 2317 days ago | link

>I don't know yet how "registering" would work. But we can start with something informal.

Perhaps just the functionality to clone a repo into /lib under the requisite namespaces as a way to include remote dependencies, including pinning to branch and version? No need to "register" anything at first.

-----

2 points by akkartik 2317 days ago | link

"It seems like the most drastic change in philosophy would happen if we broke Anarki apart into sections where if someone's working on one section and discovers they're breaking something in another section, they don't feel like they can fix it themselves."

Exactly! It's a huge problem in the programmer world today that we are indisciplined about this distinction. There's a major difference between making a boundary just to make edits easy and "APIs". APIs are basically signalling to most people that they shouldn't be crossing them, and guaranteeing that anybody calling them will be able to upgrade without modifying their code. That is worlds away from just not being able to see everything on a single screen.

Since we use words like "abstraction" and "interface" for both situations, it is easy to cross this line without any awareness that something big and irreversible has changed.

Currently nothing in Anarki is an API. I recommend we be restrained in using that term, and in introducing any APIs.

-----

2 points by rocketnia 2316 days ago | link

"There's a major difference between making a boundary just to make edits easy and "APIs"."

Maybe there's a difference somewhere in what we mean by "API." There's a distinction to make, but I wouldn't make that one.

I'm saying APIs arise naturally at boundaries people already feel aren't worth the effort to cross for various plausible reasons. Recognizing an API boundary is a positive thing: Then we can talk about it and help people understand the ways in which it's worth it for them to cross the boundary. But talking about it also makes it something people form as a concept in their mind and form expectations around, so it can become a more imposing boundary that way if we're not careful. We should seek a sweet spot.

The important distinction is that sometimes the reasons people don't cross boundaries are good ones, and sometimes they're baloney.

I think what you and I want to avoid for Anarki is documentation that gives people baloney reasons to think it's not worth it for them to edit the Anarki code.

While we can try to minimize the good reasons that exist, and we can encourage people to try it before they knock it, there's no point in denying that the good reasons sometimes exist.

If you ask me, the reason we have to settle for a "sweet spot" is totally a consequence of the fact that Anarki is made of text. Text loves bureaucracies. Someone sees text, they ask for the Cliff's Notes, and then someone has to maintain the Cliff's Notes.... So you've gotta get rid of the text, or you'll have an API boundary.

Programming without bureaucracy would be like having a conversation without writing it down. I believe in this. I think conversational programming is exactly what we can do to simplify the situation of programming in the world, and Era's my name for that project, but Anarki is a more traditional language, and it takes a more traditional trajectory into APIs.

Our documentation can try to convey "This is all subject to change! Edit it yourself, open an issue, or drop by on Arc Forum for some help! We count on contributions like yours to make Anarki a good language. Since others are welcome to do the same thing, watch out for changes in this space." We could put an "unstable, just like everything else here" tag on the corner of every single documentation entry.

Anarki is far from the only unstable codebase people have ever seen. Being a Node.js user, my code has regularly been broken by upgrades, and Node.js isn't even a version 0.0.x project. I think only a certain number of people would mind if Anarki promised stability and then couldn't deliver on it, let alone mind if Anarki made it clear in the API documentation that there was nothing stable to find there in the first place.

---

"guaranteeing that anybody calling them will be able to upgrade without modifying their code"

You know me well. I probably said that pretty insistently in the past. It's something obsess over and a major requirement I have for code I write (and hence all its dependencies), even though things like Node.js show me it's not a current norm in software.

It's the 100-year language ideal. Spend a lot of time to get the axioms just right, and then the language is stable and nobody ever feels the need to change it since there's nothing to improve on! In 100 years, Arc is supposed to be the language that's too perfect to be anything but the stablest language around.

I still believe in that ideal, but not in quite the same way. People will come up with a reason to promote their alternative language, even if it's just to say they're different than their parents. But I do believe languages can be designed to be more compatible with their forks, even to the point where it's difficult to say where one language ends and another begins. The resulting unified "ecosystem of all languages" could have a much simpler design than the "ecosystem of all languages" we currently have.

That's also part of my approach in Era. I make decisions in those codebases with the goal of simplifying the ecosystem of all languages.

I don't think this is something I would pursue in Anarki. Arc was very far from the the 100-year language upon release, it was unstable, and it seems like Anarki has pretty much been maintained by and for people who could embrace that part of Arc's journey.

-----

3 points by akkartik 2318 days ago | link

https://github.com/arclanguage/anarki/commit/fe2c9eb2d6

I think I just thought of the top-level directory as the language, and lib/ as the standard library.

-----