I think two changes would be better for modularity than changing assignment, however:

First, limit the ability of macros to overwrite existing symbols to the file in which the macro is called. This would allow modular or imported code to use macros without worrying about global namespace collisions, or needing to do anything special with assignments.

Second, and related to the first, would be to implicitly namespace based on file location. Currently, Arc uses a single namespace, and a reference to a root directory that's used to disambiguate relative paths called with (require).

We would ignore the root folder and use the remaining path and file name as the namespace.

This would allow explicitly resolving possible namespace collisions by referring to the namespace where necessary, the way it's done in other languages (at least in C++.) So if you had, say, a macro in news.arc with body as a parameter, first, it wouldn't mutate body in html.arc, and you could refer to (html!body) directly to disambiguate, using the same syntax for namespaces as tables.

An added benefit would be that imported code from a repo (adding a vendor path) would work as is namespaced as vendor/file or vendor/app/file.

It wouldn't be necessary to declare namespaces in most cases then. We could use the existing with statement to declare or extend namespaces instead of something like (declare-sub-namespace), similar to 'using' in C++

    (w/namespace foo
      (= bar (table))

    (foo!bar)

I've been trying to get the tables generated by the personal data link in news to export as JSON for a while now[0]. Part of the problem seems to be related to this - Racket doesn't seem to know what to do with #tagged tem or #hash (much less nils.)

Ironically I've found (others may feel differently) that while pg may have wanted the purpose of Arc to be exploratory programming, he seems to have done so with a number of his own assumptions baked in to the language, implicitly limiting exploration to what he considers to be correct, and to what correlates with his personal style. It's like Arc is Henry Ford's Model T: you can have it in any color you like, as long as you like black.

Do we ever expect Anarki values to be somehow better than Racket values are? If so, then they shouldn't be the same values. (The occasional "interop headaches" are a symptom of Anarki values being more interchangeable with Racket values than they should be, giving people false hope that they'll be interchangeable all the time.)

I think this is why Arc originally tossed out Racket's macro system, its structure type system, and its module system. Arc macros, values, and libraries could potentially be better than Racket's, somehow, someday. If they didn't already have a better module system in mind, then maybe they were just optimistic that experimentation would get them there.

Maybe that's a failed experiment, especially in Anarki where we've had years to form consensus on better systems than Racket's, and aligning the language with Racket is for the best.

But I have a related but different experience with Cene; I have more concrete reasons to break interop there.

If your goal is for arc to have widespread adoption then being able to leverage racket in a meaningful way will help get you there.

Currently the ability to drop into racket is not getting people to use arc, it still seems people would rather just use racket. http://arclanguage.org/item?id=20781

Common Lisp's approach is that the way a symbol is read incorporates information about the current namespace. That way usually all symbols, even quoted ones, can only have collisions if they have collisions within the same file, and this makes hygiene problems easier to debug on a per-file basis.

What Common Lisp and Clojure call namespaces are like prefixes that get prepended to every symbol in a file, changing them from unqualified names into qualified names.

I think namespaces are a fine approach for Arc. If Anarki's going to have both, it's probably best to rename Anarki's interactions with Racket namespaces (like in ns.arc) so they're called "environments" or something, to reduce confusion. I think they will essentially fit the role of what Common Lisp calls environments.

realms ?

    (def map1 (f xs)
    "Returns a list containing the result of function 'f' applied to every element of 'xs'."
   -  (if xs
   +  (if (~empty? xs)
        (cons (f car.xs)
              (map1 f cdr.xs))))

- the assumption baked into this argument is that cdr of an empty list returns an empty list. Switching nil to #f and letting empty list be truthy avoids this problem.

Second option, part a: I actually find `~empty?` clearer than `some` in this case. Also `some` means something different in Arc.

  (expect subject pattern else
    then)
  ->
  (match subject
    [pattern then]
    [_ else])

  (def map1 (f xs)
    (expect xs (cons x xs) ()
      (cons (f x) (map1 f xs))))

Let's put it this way. Racket is an excellent choice for implementing languages. It has every feature you could want. And yet it is very difficult to do even the simplest things. For example, when an error is raised in Anarki, you'll see a stack trace that points to ac.rkt rather than the actual location within the arc file that caused the error. This is because, frankly, ... Well, let's just say I've redacted some expletives here. But those expletives were aimed at the fact that it's really quite difficult to translate the vision in your mind into an implementation using Racket.

Now, that being said, if you sit down with Racket and take the time to learn it, you will find that it's one of the most robust, flexible, and performant language runtimes in existence. The thread-local variable support is a killer feature. The custodian support is rock solid, which makes sandboxing trivial and super reliable. And it's the premiere implementation of Scheme, so it might continue to have a thriving community for decades to come. But it took me years to become very effective with Racket. (This says more about my own shortcomings than about Racket, though!)

I would suggest checking out Scott Bell's Lumen lisp: https://docs.ycombinator.lol/

It's a Lisp that runs in both JS and Lua. It's self-hosted, meaning all .js and .lua code is generated by Lumen itself. Diff this technique with the fact that ac.scm is written in Scheme rather than Arc. Very mysterious! I remember how electrifying it felt that first day I stumbled across it and realized the significance of what I was looking at.

It's one of the most incredible projects I've ever seen. It's so small and clear, yet does so much – a perfect example of why simple systems run circles around competitors.

No matter what your experience level is with Lisp, the code will seem straightforward: https://github.com/sctb/lumen/blob/55b14ca8aafeaf6b0ca1b636d...

There is a branch that adds Python support at https://github.com/shawwn/lumen/tree/features/python which speaks to Lumen's flexibility and power. (I wonder if there is another Lisp that you can use natively from three different languages without any FFI. And if there is, I doubt it's <3000 lines.)

At this point I use Lumen to explore new languages. E.g. I learned R by implementing a Lumen to R compilation target: https://github.com/sctb/lumen/pull/193

Is that really "the simplest things"? :-p It seems to me Arc goes out of its way to avoid interleaving source location information on s-expressions the way Racket does. Putting it back, without substantially changing the way Arc macros are written, seems to me like it would be pretty complicated, and that complication would exist whether Arc was implemented in Racket or not. (I think aw's approach is promising here.)

---

It's fun to see Lumen is designed for compiling to other languages even when it's self-hosted. That's always how I figured Arc or pretty much any of my languages would have been written once they were self-hosted.

Surprisingly it's possible to get pretty close. The trick is to read-syntax rather than read, and then have ac map over syntax objects properly. At that point it's a matter of using eval-syntax rather than eval.

The takeaway is that the error messages are much, much nicer. I'm talking "the error is at line 213 of app.arc in function init-userinfo" nicer.

Whoops, wrong link! It's at https://news.ycombinator.com/item?id=17963471.

ycombinator.lol is just because fixed-point combinators are funny:

  (define Y (λ(b)((λ(f)(b(λ(x)((f f) x))))
                  (λ(f)(b(λ(x)((f f) x)))))))

Lumen looks awesome. I was looking for the docs to determine how much of arc actually exists within lumen, but I couldn't find anything. So if you do this, please let me know.

That makes it a lot easier to answer a question like "Is X a good choice for Y?" It depends on Y! :-)

They can if you can nest them, or include references to other keys (like foreign keys in SQL or something) It's inefficient but it's possible. You can even represent trees in a flat array if you work hard enough.

Apparently pg did it in part to be "unPC" in regards to HTML. (see the linked quote in the 'inline javascript' thread[0].) It is weird that, in the name of exploratory programming, he chose the most complicated and finicky layout imaginable just to stick it to the W3c and their rules, man.

Fortunately, I think Anarki is already moving in the right direction. Table layout should be almost entirely removed (I think polls still use it) and I've been adding css classes and IDs to make the HTML more amenable to scripting. Unfortunately, this means we can't really use anything from HN without rewriting it. There's still a bit more work to do in removing old html like fontcolor, etc.

But the interesting question would be where to store state? HN appears to have endpoints that handle it (?collapse=id) so I assume there is just a list of closed ids for each user somewhere on the server. It could probably also be done with local storage but that wouldn't persist across browsers.

Which is the same for voting and page generation.

ie. Right now there's a big cost on the servers because all the work is done on the servers. If you start looking at it from the perspective of not adopting that cost then you might as well say the same for voting + all the html creation and just write the whole thing in js where you only fetch data.

Yea, I don't think something like checking whether a comment is member of the list of hidden items by a user would really add workload of any significance.

> [...] you might as well say the same for voting + all the html creation and just write the whole thing in js where you only fetch data.

I don't think so either. My comment was that "all the work is done on the servers". For the whole app. That is looking at all of the cost in aggregate (every interaction requires a http request, and requires throttling, session handling, authentication, html page generation, and so on....).

> Does that not lead to an awful lot of traffic sometimes? (I'm imagining a version of News that would transmit all submitted content to let the client do the sorting and searching instead of doing it on the server.)

- Arc is too slow to handle file uploads. Arc is not suitable for web apps that handle image and video upload.

- This is not a problem with Arc but with News: It relies a lot on state. I'm not a purist, but it is to an extend making it a bit difficult to hack on sometimes, e.g. the `unmarkdown` function because original input is "forgotten", and how the score of an item is just a numerical value rather than something that can be derived from voting data which makes `unvote` difficult to implement correctly.

I like Clojure as a language, but I'm a bit overwhelmed with choice when it comes to web frameworks and the package managers you need to use those web frameworks (somewhat similar to Common Lisp), whereas with Racket and Arc there's just one place to start. Do you have any experience with Clojure web frameworks?

FYI cljs = clojurescript.

When Clojure first came out there was a core set of libraries that everyone flocked to:

1. Compojure (a routing library)

  - similar to arc's defop

example:

  (defroutes app
    (GET "/" [] "<h1>Hello World</h1>")
    (route/not-found "<h1>Page not found</h1>"))

  - parses the web request and converts it into a hash-map of meaningful values.
  - similar to srv.arc

2. Hiccup (HTML Soup)

  -  similar to hmtl.arc, but uses a data structure to provide flexibility.

example:

  [:head
    [:meta {:http-equiv "Content-type"
            :content "text/html; charset=utf-8"}]
    [:title "adder"]
    [:link {:href "/adder.css" :rel "stylesheet" :type "text/css"}]]

https://mmcgrana.github.io/2010/07/develop-deploy-clojure-we...

note: it's a 2010 article so some of it's outdated, but the idea would be the same.

At that time everyone was racing to make more robust web frameworks. Many of them were from people doing the above stuff only adding features. However, shortly afterwards cljs was released and another slew of web frameworks came out as people embraced writing web apps client side. Then again, shortly afterwards, Facebooks React became the new thing and advanced the idea of further separating out the data content from the UI composing for rendering. At this point data models (i.e. big hash-maps) and syncing that data to the UI became the new norm. And even since then more advanced frameworks came out, such as Fulcro, that further extend the data modelling & syncing features (https://github.com/fulcrologic/fulcro).

Through out all this many web frameworks became abandon-ware and now it's really hard for a newbie to make sense of which one to use. In my opinion:

1. If you want to do what Arc does (server side page generation) then use Compojure + Hiccup.

2. If you want to write basic client side cljs code there are dom libraries like:

- Dommy https://github.com/plumatic/dommy

- Domina https://github.com/levand/domina

These are fairly simple to use.

2a. If you want to write client side cljs code that takes advantage of React then use Reagent. https://github.com/reagent-project/reagent (Much better more interesting that Domina/Dommy)

If you have a desire to enter the more advanced data-model-UI-syncing arena where I would probably use Fulcro (but haven't). Note that these advanced frameworks like Fulcro expect you to know much more about state management / data modelling and it could be a steep learning curve for some.

I've been developing cljs web apps for over 4 years. Over these years I've tried some of the frameworks, but I ended up writing my own as none of them could do what I needed.

Is that helpful?

All of that may seem like too much, but remember you really only need Compojure + Hiccup to be where Arc is at.

Yes! Thanks a lot for the write-up.

That Compojure example does look quite familiar and Arc-like, and looks like it can handle multipart post requests too[0].

I'd never heard of Fulcro before. Given what is often emphasised about Clojure, at first glance at the Fulcro docs I'm a bit surprised how often they mention state and mutations:

> The other very common case is this: You’ve loaded something from the server, and you’d like to use it as the basis for form fields. In this case the data is already normalized in your state database, and you’ll need to work on it via a mutation.[1]

Also, I'm too much into graceful degredation to ever go all out Cljs, unless it was for a phone app. But I find that it's often interesting to see how people do things in Clojure, even when not using that language, so I'll be giving those Fulcro videos a look.

[0]: https://github.com/whostolebenfrog/compojure-multipart

Well things on the client side can be sometimes be mutable. No one gets around the fact the DOM is a mutable only object. But besides that, the Fulcro library has labelled one of their feature's a 'Mutation'. Which was probably a bad choice, but it has nothing to do with the immutability of the underlying cljs object that it uses for that "Mutation". You'll notice the example is using 'swap!'. That means it's modifying an atom; Where an atom is an interface to make changes to the immutable object it holds. So really 'swap!' takes the change request, constructs a new version the original thing held in the atom, with changes, then 'swap's it with the original item inside the atom. The original thing was never changed (no changes to existing slots in memory). Hence clojure's things are immutable, and they are in Fulcro too, accept when changing the DOM tree.

Basically the way I interpreted it[1] is that "major version" is a meaningless concept. If you're making incompatible changes, rename the package. If that's a disincentive to making incompatible changes -- great!

> Rich Hickey pointed out last year that the convention of bumping the major version of a library to indicate incompatibility conveys no more actionable information than just changing the name of the library.

Yes, I guess Hickey is applying this idea of immutability not only to data structures but also to APIs and even databases[0] with the proprietary database service Datomic. Interestingly Datomic uses Datalog as query language, so it's straight forward to apply at least some of those ideas with the Racket Datalog package[1].

As of now I have a basic web forum working with all data storage done in Datalog (except for file uploads!). I'll post some code once the design is a bit more settled. It's still in the breaking-things-all-the-time phase.

What I find a bit tricky about Datalog is that relations are stored together with other facts, which in my mind feels a bit like storing code in a database, but maybe I just haven't wrapped my head around it yet.

Does anyone here have favourite articles or talks about logic programming?

[0]: https://www.youtube.com/watch?v=EKdV1IgAaFc

I can't speak for Datalog, but I've used Datomic.

If it's the same, then a 'fact' is comprised of an entity (the id), + an attribute, + a value.

Cool!

> The relationships are made by storing an entity id into the value slot of another fact. Thus the model is both flat (being a list of facts) and hierarchical (they can point to each other). It pretty much becomes a graph database. Is that what you mean?

Yes, exactly! What I worry about (because I'm fairly new to logic programming) is whether it could potentially be difficult to update a program where storage of business logic and storage of data aren't separated?

If we assume we have a Hacker News web app where we have a fact: One day Alice submits a story with the title "How to peel onions"; and we are thinking "Why on earth did she post that here?!?" So we add this relation to our code: A story is `irrelevant` if it has the word "onion" in the title. Then, the next day we get another fact: Now Bob has submitted a story called "How onion routing works". This new story by Bob then makes us reconsider our definition of `irrelevant`.

In a typical imperative program we'd just edit the code and redefine the `irrelevant` predicate, and it would take effect next time we run the program (or instantly if we enter it at a repl). But here in our logic program we store this `irrelevant` relation in our graph database, so even though we have removed it from our code, it is still sitting there in the database along with all the facts, outside the reach and responsibility of git, or whichever VCS we're using.

  -----------------------------------------------------------
  Entity              | Attribute | Value
  -----------------------------------------------------------
  person-id-001       | name      | Alice
  person-id-001       | stories   | [story-id-001, story-id-002...]   

  story-id-001        | headline  | "How to peel onions"
  irrelevant-word-001 | stories   | [story-id-001, ...]
  irrelevant-word-001 | word      | onion
  -----------------------------------------------------------

So all of this assumes a few things:

- Your DB supports a cardinality of 'many' items in the value slot.

- Your query language can perform joins.

Of course none of this helps when someone changes a headline, but only full-text search DB's will help you do that.

  -----------------------------------------------------------
  Entity        | Attribute          | Value
  -----------------------------------------------------------
  globals       | irrelevant-words   | [irrelevant-word-001, ...]   
  -----------------------------------------------------------

What's interesting (at least to me) is that Datomic has a function called 'retractEntity' [1] which auto-magically removes all references of an entity in any value slot when you retract the entity. Man I love Datomic :)

I can make sense of that when there's just one instance of this app running. Yet imagine the scenario where the web app has been published, and suddenly other people are running this web app. If the business logic is then changed, somehow I'd have to tell those people: "Oh btw, when you're running `git pull` next time, then you just also gotta run this query to retract some of the old relations from the database."

Definitely not a problem for me yet, but I can just smell it coming. I could add those retractions to the code, but they would have to stay there indefinitely, because it's not possible to tell if those retractions have taken place on everyone's databases yet.

Maybe I'm over-thinking this.

> What's interesting (at least to me) is that Datomic has a function called 'retractEntity'

Looks like the one called `~` in Racket's Datalog[0].

Is this irrelevant-word example a real feature you're building into the app or a contrived example to understand Racket DataLog DB use?

If it's a real feature, and I'm assuming it is. Then I'm also assuming that when a story is submitted you're parsing the title and adding the relationship to the current set of irrelevant-words that are stored.

So the question's are:

1. How are you going to remove the past relationships between stories and an irrelevant word that's getting removed? (looks like we've answered this).

2. How are you going to make sure past stories gain the relationship to newly added irrelevant words?

3. How are you going to handle title changes.

It's a simplified and slightly contrived illustration of an issue in this pre-alpha code I haven't published yet, perhaps just because I haven't thought of a name for the project yet. But yes, let's assume it's a real feature for now.

> 3. How are you going to handle title changes.

I like Hickey's idea of accretion of data - with a timestamp! - and not forgetting previous facts. I think he's talking about it in The Database as a Value[0]. So, a story could have a few different titles in the database, and the newest one is the one you get to see.

The thing is, it's easy to add a timestamp to facts as a way of not considering old facts without forgetting them, but not to relations. For example in Racket Datalog[1]:

    (! (voted "i4cu" 'up 134))

    (! (voted "i4cu" 'up 134 1542151773))

    (! (root A B) (ancestor A B) (parent null A))

> 2. How are you going to make sure past stories gain the relationship to newly added irrelevant words?

Datalog queries reflect the current set of facts and relations, so the possible irrelevance of a story would not be stored anywhere, so it wouldn't need to be updated.

> 1. How are you going to remove the past relationships between stories and an irrelevant word that's getting removed?

Some as above. For example, in a place oriented program a story object could have a boolean attribute `irrelevant?`, whereas I'm sending a query every time this value is needed, so no stale `irrelevance` attributes are stored anywhere.

> After you get handle on these then what you really need to do provide an interface, from within the apps admin tools, to trigger the noted functionality. This way the business logic is in the app and it's modifying the data.

Yes, that is kind of there already, as in having functionality for changing titles. The `irrelevant` functionality is not there now.

Ok, I think I just need to work on getting this code publishable, because it might be easier to discuss tangible examples.

[0]: https://www.infoq.com/presentations/Datomic-Database-Value

So where will you're data be? In a local data structure? I read your racket Datalog link, but it doesn't show any details for the database side (i.e. durability etc.) even though it's labelled a database.

Yes, I think. The database just stored in memory but it can be serialized and saved to the disk using `write-theory`[0] and loaded `read-theory`. That is what I'm doing for now, and it's a very naive and inefficient to do a full database dump rather than just appending new data, and I presume it's particularly in this area where Datomic is way more optimised and well thought out.

> Also, I'm curious what made you choose a graph db. It seems like you're inheriting a lot of complexity and I'm wondering what the benefit is over a more traditional sql or nosql db.

Well, I did the initial work on the web app: creating user accounts, adding posts and replies, and then I got to data storage. Initially I did a News-style flat-file database, just saving data as lists in files, that are then loaded into memory when the program starts. It mostly worked but also felt a bit complicated, and I thought that perhaps I should just use a proper database?

What I like about news.arc is that you can just launch it without any configuration, so MySQL and PostGreSQL were out of the question, and I started reading a bit about SQLite. But I've also had this fascination with logic programming, from what people are posting here[1][2], and from reading a bit of The Reasoned Schemer, and I watched some of those Rich Hickey talks again, where he talks about Datalog, which happens to be available for Racket.

There are just some things that are incredibly simple in declarative/logic programming. For example, if you have facts about stories being `parent` of their replies, then it's simple to just define the `ancestor` relation, and when you have the `ancestor` relation, you automagically get `descendants` without having to write any code, because it's just the inverse of `ancestor`:

    (! (:- (ancestor A B)
           (parent A B)))
    (! (:- (ancestor A B)
           (parent A C)
           (ancestor C B)))

So, SQLite is still on the table. I'm not too familiar with NoSQL, but my impression is that they are all about speed and scalability of data storage. I haven't used MongoDB but isn't it essentially just like storing JSON in a file, except faster? It would be interesting if any of those could be used in conjunction with Datalog though, if don't add too complexity for the sake of increased speed.

[0]: https://docs.racket-lang.org/datalog/interop.html#%28def._%2...

[1]: http://arclanguage.org/item?id=20650

Datomic vs. DataLog

Datomic uses DataLog as part of its query language, but that's pretty much where the comparison should end. Things like "treating the database as a value", and features such as data accretion that Rich talks about have nothing to do with DataLog. They're features of Datomic. So for example when you mention never retracting data, well your data size is going to continuously grow unless you write your own data management layer on top. Datomic, on the other hand, does this for you. When you want to query the database over time, then you're going to need to store time intervals for all of your data and incorporate that into each query. Where as in Datomic (which has a time log) you can pass in the DB itself as a value (with an associated time interval) and Datomic will make sure your queries are working against the dataset that accounts for the time interval.

I'm pointing this out because it seems to me that you're doing (or are going to be doing) a lot of work that may not be worth it for what you're trying to accomplish.

Nosql

> I'm not too familiar with NoSQL, but my impression is that they are all about speed and scalability of data storage.

Yes and No. Often speed can be a feature Nosql dbs advertise, but really, for me anyway, it's about flexibility and ease of use. Traditional RDBMS, for example, require creating schemas. Many Nosql databases don't require a schema at all which makes it easier to use and more flexible to change. Nosql's are often a key-value store so it can be really easy to take a hash-map or table of data from your code and just dump it into an nosql datastore and be able to query it.

My personal favourite is Redis and it might be worth considering for your app.

You can:

- store a value under a key [1]

- store table data [2]

- store values in a set [3] (which allows intersection/difference queries)

- store values in a sorted-set [4] (which allows you query by some numerical value like timestamp)

- use it to manage relationships [5]

The reasons I mention Redis is that the HN app is very well suited to it. HN only keeps 'x' amount of data in memory. And in Redis the data lives in memory. Also Redis allows you to set expiry times on data for auto eviction [6]. And Redis also supports ordered lists [7] which can make it useful for lisp based languages.

However it's not embedded. And if that's a requirement I'd almost suggest you move away from Racket and adopt a language that has more options for embeddable databases. I guess if you're willing to roll your own (and it looks like you may be) then that's awesome too.

But in case you decide otherwise... The library I use is Redis Carmine [8], but there are Racket clients [9].

1. https://redis.io/commands/set

2. https://redis.io/commands/mset

2a. https://redis.io/commands/mget

3. https://redis.io/commands/sadd

4. https://redis.io/commands/zadd

5. search: "Representing and querying graphs using an hexastore" https://redis.io/topics/indexes

6. https://redis.io/commands/expire

7. https://redis.io/commands/lset

8. https://github.com/ptaoussanis/carmine

I'm not sure I totally agree with this. I think that apart from talking about the design of Datomic, he also has a more general point against what he calls PLOP (PLace Oriented Programming), which Datalog does address.

For example in plain Racket a value is lost if something else is put in its place:

    > (define foo 'bar)
    > (define foo 'baz)
    > foo
    'baz

    > (! (is foo bar))
    > (! (is foo baz))
    > (? (is foo X))
    is(foo, bar).
    is(foo, baz).

> The reasons I mention Redis is that the HN app is very well suited to it. HN only keeps 'x' amount of data in memory. And in Redis the data lives in memory. Also Redis allows you to set expiry times on data for auto eviction [6].

Interesting. Just checked news.arc, and yes `initload*` is set to 15000. Interesting idea from Redis with expiry times. I'll check it out. I hadn't considered the scenario of storing text enough to max out on memory, because it would probably be premature optimisation, but good to keep in mind. I'd like to give Redis/Rackdis a try; thanks for the suggestion. I've been hosting an Etherpad Lite instance, and Redis was painless to setup.

> I'm pointing this out because it seems to me that you're doing (or are going to be doing) a lot of work that may not be worth it for what you're trying to accomplish.

Yeah, I was thinking more along the lines that Datomic has built-in functionality to address the caching, cache eviction, and indexing that goes along with all that data accumulation. But you're correct, DataLog does accumulate facts.

> Interesting. Just checked news.arc, and yes `initload*` is set to 15000.

I did the same thing, about 6 or 7 years ago, that you're doing now. I ported HN to Clojure (which is actually how I learned Clojure). If memory serves me correctly when I was doing the work I realized I needed a real DB if I wanted to support load balancing. i.e. I needed to centralize the data for the authentication and fnid session info. I think Arc calls them fnids... You probably know better than I do now, but Arc has all this code to expire these session fnids and so, for me, Redis was just a good fit for that task.

Anyways, I'll be sure to take a look at the final result of your work.

Cool!

> I needed to centralize the data for the authentication and fnid session info. I think Arc calls them fnids... You probably know better than I do now, but Arc has all this code to expire these session fnids and so, for me, Redis was just a good fit for that task.

The Racket web server is quite "batteries included" and comes with these different managers[0] for dealing with expiration of sessions/continuations, such as the LRU manager:

> The memory limit is set to `memory-threshold` bytes. Continuations start with 24 life points. Life points are deducted at the rate of one every 10 minutes, or one every 5 seconds when the memory limit is exceeded. Hence the maximum life time for a continuation is 4 hours, and the minimum is 2 minutes.

> If the load on the server spikes—as indicated by memory usage—the server will quickly expire continuations, until the memory is back under control. If the load stays low, it will still efficiently expire old continuations.

When I was referring to load balancing and centralizing the data I was referring to many web servers sharing a centralized/external source for auth/session data.

I'm unfamiliar with racket's web server 'servlets'. The docs are little unclear (at least to me). Can these servlets live on a separate server so that the data can be shared between web servers? I'm guessing that was/is not a requirement for you, but I'm just interested in knowing if that's how it can work.

Uh oh, you're getting me interested in Racket now. I can't have that... I have too many projects :)

Probably. I assume that serializable continuations[0] from stateless servlets can just be stored wherever, like in Redis or something, instead of in the memory of one server.

> I ported HN to Clojure

If that is something you have published, it'd be fun to see, whether it's finished or not.

> Uh oh, you're getting me interested in Racket now.

My impression is that Clojure is faster, less verbose partly due to clever syntax and provides more immutable data structures than Racket. But when it comes to documentation and error messages, I find Racket more coherent and comprehensible.

Say, if I wanted to connect to a SQL databse, with Racket I'd use the DB module[1], end of discussion. But with Clojure there's Korma, ClojureQL, Persist, HoneySQL, Yesql, a JDBC wrapper from Clojure contrib, SQLingvo, oj, Suricatta, aggregate, Hyperion, HugSQL, and probably a few more[2][3]. That multitude of libraries with similar purpose may be useful in some cases, sure, but also potentially a bit overwhelming for beginners, so I guess that's why I found it easier to get started with Racket.

[0]: https://docs.racket-lang.org/web-server/stateless.html#%28pa...

[1]: https://docs.racket-lang.org/db/

[2]: https://stackoverflow.com/questions/294802/use-a-database-wi...

I actually tried to look it out the other day during this conv, but it's buried somewhere unavailable right now. If I find/get to it I'll post.

> That multitude of libraries with similar purpose may be useful in some cases, sure, but also potentially a bit overwhelming for beginners, so I guess that's why I found it easier to get started with Racket.

Agreed. Navigating the volume libraries and the options available is a real pain in the beginning, but once you get past that, then it's not bad at all. At the same time, take a look at the quality of Clojure's Redis Carmine Library vs. Racket's Redis Libraries. Miles apart.

Good point.

Anyways, I'm going with SQLite. It's really fast[0] and I just found out about recursive selects[1].

[0]: https://www.sqlite.org/fasterthanfs.html

1. The ability to read them. Especially on output. It may seem small, but when you have a larger deeply nested map it really matters.

  {:user "akkartik"}

  (map :user list-of-maps)

Here's a good function to separate a collection of maps by any fn/ifn.

  (def separate (juxt filter remove))
 
  (separate :user data)

5. Destructuring bind. Eg:

  (defn myfn [{:keys [user] :as record}]
    (str user " rocks"))

Clojure:

  > (def players
       {:joe {:class "Ranger" :score 100}
        :jane {:class "Knight" :weapon "Greatsword" :score 140}
        :ryan {:class "Wizard" :weapon "Mystic Staff" :score 150}})

{:joe {:class "Ranger" :score 100} :jane {:class "Knight" :weapon "Greatsword" :score 140} :ryan {:class "Wizard" :weapon "Mystic Staff" :score 150}})

Arc:

  (= players
    (obj 'joe (obj 'class "Ranger" 'score 100)
         'jane  (obj 'class "Knight" 'weapon "Greatsword" 'score 140)
         'ryan  (obj 'class "Wizard" 'weapon "Mystic Staff" 'score 150))) 

#hash(((quote jane) . #hash(((quote weapon) . "Greatsword") ((quote class) . "Knight") ((quote score) . 140))) ((quote ryan) . #hash(((quote weapon) . "Mystic Staff") ((quote class) . "Wizard") ((quote score) . 150))) ((quote joe) . #hash(((quote class) . "Ranger") ((quote score) . 100))))

-- Readability --

Reading that output is beyond a headache. It should be against the law! But more importantly, when writing code/troubleshooting you can't copy the evaluated output make a modification and then call the function again. I know you can bind it and apply functions to modify it, but often that's a real hassle compared to copy/paste.

-- Destructuring bind --

See links [a,b]

I think simple cases don't show the difference.

  > (def book {:name "SICP" :details {:pages 657 :isbn-10 "0262011530"}})
  
  > (let [{name :name {pages :pages isbn-10 :isbn-10} :details} book]
      (println "name:" name "pages:" pages "isbn-10:" isbn-10))
	  

    name: SICP pages: 657 isbn-10: 0262011530
	

Clojure has DOZEN's of helper functions (and important traversal functions) that don't exist in Arc. Yeah I could write them, but I still have to. I was highlighting the ease with 'separate', but here's even 'deep-merge'....

  (defn deep-merge
    "Recursively merges maps. If keys are not maps, the last value wins."
    [& vals]
     (if (every? map? vals)
         (apply merge-with deep-merge vals)
         (last vals)))
		 
  > (deep-merge players {:joe {:weapon "LongBow"} :jane {:weapon "Lance"}})
  

-- Nil values --

I'd even be fine with any falsey values like boolean false. But Arc will not even store an #f value. To suggest I can't pass in REAL data with falsey values is really limiting. I can't pass in an options table argument. For every operation I would have to add more data or nuanced data or nuanced operations to support "this value was set, but it's set to 'NO' I don't want that option.".

[a] https://clojure.org/guides/destructuring#_associative_destru...