It seems to me, the original problem was this: One defines a template with a non-nil value as the default for a fields argument, then upon instantiating the template one wants to pass in a nil value to eliminate that fields default non-nil value from being part of the resulting table.
By making templates first class objects you have permitted these over-rides to occur, but at the same time you have introduced an additional layer of complexity for every user of templates (and in some cases even tables), when they don't care about that specific circumstance.
So at first glance I can't help wonder why not just re-write templates in a manner that eliminates poor handling non-nil values. Then we could have maintained the inst function while also allowing for other methods to be created, such as an 'inst-with' that allows nil values to over-ride defaults (or just have inst allow for over-rides).
I'm not sure how templates are written, but I imagine they make heavy use of tables, which is where the nil pruning is probably occurring. So templates would instead need to use association lists up until the output stage where data then gets pushed into a table.
Or, if instead, you want tables to support nil values, then that's also a change worth considering, but I see the introduction of a whole new type as a means to solve a certain circumstance as overkill with downside effects.
You're absolutely right in the historical particulars. Templates started out as light macros atop tables, and their serialization was indistinguishable from that of tables. But they always seemed like a separate type given all the names we've devoted to manipulating them separately from tables. I thought the layer of complexity already existed, and was trying to simplify things so people didn't need to know that templates are built out of tables.
The problem with not supporting nil values in templates -- and therefore in the data models of any webapp -- is that you can't save booleans as booleans. That seems ugly.
Adding new names to paper over problems with existing names seems ugly as well.
Your idea of questioning the design of tables themselves is interesting. I hadn't considered that. A persistent data structure seemed like a separate beast, distinct from a table. But perhaps all tables should be easy to persist. Hmm.
---
My attitude towards anarki has been that it's trying to be the core for a hundred-year language, so it should get the core right without regard to how things were before. So we should avoid unseemly hacks for things that seem simple -- like storing a user pref that's a boolean -- without regard to how commonly they happen, or how easy workarounds are to find.
But I can absolutely imagine that others might have a different perspective on anarki. Let me ask you this: how would you design templates if you were doing so from scratch? I'm trying to get at how much of your argument hinges on compatibility. Perhaps there's a more general question here that we should be discussing. Perhaps we should pay more attention to lark's use case: not wanting to think about arc's internals.
Hmm. Well I see that as a separate issue. I think nil and boolean false are very different things. nil means no value or false while false means a false value. In fact I was recently going to suggest that arc should support proper booleans and that arc tables should also store both boolean values, while maintaining it's nil pruning feature. Which really stems from wanting to easily transform to/from json or edn like formats. Currently one has to fudge arc's write-json by passing around symbols for 'true / 'false.
> how would you design templates if you were doing so from scratch?
Well that's interesting that you ask, because I did just that when I implemented templates in Clojure. Of course Clojure supports true boolean values and maps can hold nil values, so it was trivial and very useful.
My Clojure templates are on steroids though. Not only do they match most of the features in arc, but they are cumulative, and accept anonymous functions as field values. The values can also refer to other fields inputs or results.
1. In bad shape (wrote it early on)
2. Includes partial features not fully implemented.
3. Has references to functions I can not split out without
creating a bunch of work.
4. Includes features you would not care for (datomicish).
5. Has oddities that make you wonder "why like that",
until you realize you can pass in say a map of args
instead.
With the above reasons, I was going to say I'll pass on releasing the code, but at long as you're ok just getting the scaffolding that will not run for you then here you go:
(def index* (ref (hash-map)))
(def templates* (ref (hash-map)))
(def mutes* (ref (hash-map)))
(def selfs* (ref (hash-map)))
(defmacro deftem [name & fields]
`(let [tem# (quote ~name)
order# (evens (list ~@fields))
fmaps# (apply hash-map (list ~@fields))]
(dosync (alter templates* assoc tem# fmaps#)
(alter index* assoc tem# order#)
fmaps#)))
(defmacro defmute [name & fields]
`(let [tem# (quote ~name)
items# (list ~@fields)]
(dosync (alter mutes* assoc tem# items#)
items#)))
(defmacro defself [name & fields]
`(let [tem# (quote ~name)
items# (list ~@fields)]
(dosync (alter selfs* assoc tem# items#)
items#)))
(defn invoke-fields
([tem base fields allowables]
(invoke-fields tem base fields allowables nil))
([tem base fields allowables testfn]
(let [fks (keys fields)
selfs (@selfs* tem)]
(reduce
(fn [m k]
(assoc m k
(if (detect? k fks); must use 'detect' opposed to 'find' for nil vals must be inserted.
(aifn (fields k)
(try (it m)
(catch Exception e (it)))
(let [bfn (base k)]
(if (and (detect? k selfs)(fn? bfn))
(try (bfn (merge m {k it}))
(catch Exception e (bfn)))
it)))
(aifn (base k)
(try (it m)
(catch Exception e (it)))
it))))
(hash-map) allowables))))
(defn invoke [tem & fields]
(let [temx (split-name tem)
tem1 (first-identity temx)
atem? (is (last temx) "+")
xfn (type-fn tem)
temk (xfn tem1)
base (@templates* temk)
prox (@mutes* temk)
fval (first fields)
fmap (cond (map? fval) fval ; for file loading map of saved records
(coll? fval) (apply hash-map fval)
:else (apply hash-map fields))
imap (invoke-fields temk base fmap (@index* temk))]
(reduce
(fn [m [k v]]
(if (or (missing? k base)(nil? v)(detect? k prox))
(dissoc m k)
(assoc m (if atem? (nsify temk k) k) v)))
(hash-map) imap)))
