Arc Forumnew | comments | leaders | submitlogin
extend vs message passing, redux
2 points by akkartik 5188 days ago | 22 comments
Looking at Pauan's implementation of import (http://arclanguage.org/item?id=14707) I was reminded of his comparison of extend[1] and message-passing a couple of months ago (http://arclanguage.org/item?id=14261) and how I considered it bogus. Now that there's an implementation of message-passing, let's redo the comparison[2]:

  (def clone(x)                          ;  (def clone(p)
      (annotate 'heir (obj val (table)   ;    (object parent p
                           parent x)))   ;

    (defcall heir h                      ;      call   (fn (k default)
      [or rep.h!val._                    ;               (or self<-k
          rep.h!parent._])               ;                   (self<-parent k default)))
                                            
    (defset heir(h key value)            ;      set    (fn (k (o v))
      (= rep.h!val.key value))           ;               (set-attribute child k v))
                                            
    (extend keys(x) (isa x 'heir)        ;      keys   (fn ()
      (keys rep.x!val))                  ;               (keys child))
                                            
    (extend vals(x) (isa x 'heir)        ;      ;; ?? [3]
      (vals rep.x!val))                  ;      ))
As others[4] predicted, message passing does require multiple method signatures, and that's largely why the claimed huge line-count savings have evaporated. Both approaches now seem roughly equivalent. Some scenarios will gain you a few lines on the left; others will save a few on the right. Even the case in the original comparison is gone, making the two sides seem even more similar. It's mostly just moving parens around. (http://arclanguage.org/item?id=14314)

I was skeptical at the time partly because there seemed no attempt to make the comparison fair. Here I've tried to make both versions do the same thing as much as possible. There's no need for iso on the left because a default comparison of recursively comparing reps serves us. There's no fill-table because it just works once you have set. There's no each because coerce takes care of it, and because it'd also have to be defined on the right to iterate over the parent.

I want to acknowledge: message-passing does provide things extend doesn't. You can hook into an existing type, or a single instance, without needing to create a new type. But this superficial comparison distracted me from the true advantages. I'm adding several new hooks in wart because of Pauan's explorations of message-passing: isa lets me hook into new types, write lets me influence how they're printed at the repl. Just bringing slices to arc is worth the journey.

---

If it's down to subjective preferences, would I use message-passing in arc? Since it seems to provide no advantage or hook that I can't reimplement in isolation I'm happier giving each such hook its own top-level def. Collecting overrides for an existing type in one second place (in addition to the original implementation) seems to provide no organizational gains. I think the tie-breaker is coerce. There's no way to nicely override it using message-passing since you could dispatch on either source or destination, and it would be a wart as the only top-level override. I'd rather give up on overriding instances, and embrace extend :)

---

[1] The original comparison used wart, but that proved just a distraction. Wart has redundant keyword args for clarity but otherwise looks the same as extend.

[2] ..with the same grotesque ascii table :)

[3] Pauan, does ar support vals? Or is there no need?

[4] rocketnia? Others as well? I can't find it now. I think it came up at least twice.



1 point by Pauan 5188 days ago | link

"As others[4] predicted, message passing does require multiple method signatures, and that's largely why the claimed huge line-count savings have evaporated."

Actually, I never refuted that point. Tables have always had a keys, call, and set attribute. Even while providing all three, it still ended up being far less verbose than wart (at the time). If you have improved wart so that it's less verbose, that's great.

---

"It's mostly just moving parens around. (http://arclanguage.org/item?id=14314) "

I can agree with you that with proper macros, like defcall or defset, you can indeed make extend much less verbose. That doesn't change the fact that at the time I made those remarks, extend was indeed much more verbose.

In fact, rocketnia was at one point describing rulebooks, which used a macro to do all the fancy extend shenanigans. (Or at least, I think that's what it does)

---

"Pauan, does ar support vals? Or is there no need?"

No need. In Arc/ar `keys` and `vals` use `each`, and `each` uses `maptable`, so my implementation extends maptable and gets everything for free.

---

"If it's down to subjective preferences, would I use message-passing in arc? Since it seems to provide no advantage or hook that I can't reimplement in isolation I'm happier giving each such hook its own top-level def."

I'll note two things:

1) I used message passing to implement objects, but the idea of message passing is possible in other areas, like functions. So I'm going to assume you were talking about my object.arc library.

2) The point of this is not necessarily to enable you to do new things that you couldn't do before... in fact object.arc itself uses extend to accomplish what it does.

The point is to make dealing with data types easier and (in some cases) less verbose. One of the problems with extend is inefficiency... if you have to extend things every time you create a new clone, for instance, that'll add up if you make multiple clones.

With objects, you just extend once and then use the interface. Secondly, you can easily change an object's attributes at runtime, which is a lot clunkier and harder to do with extend. You can also implement delegation a lot easier. For instance, I am planning on adding in a get-attribute attribute that would let you seamlessly and easily delegate to another object:

  (= foo (object ...))

  (= bar (object get-attribute (fn (k)
                                 (get-attribute foo k))))
Bam, with that little code, you have prototypes.

As always, it is possible to do everything with extend, but that does not mean that everything is easy. I'll also note that I do indeed use extend quite a bit, and even decided to give up on my coerce-to-fn idea (ala Anarki) and just use extend.

The point was not to replace extend, the point was to say, "oh boy, extend is pretty clunky in these few areas... hm, let's fix that!" And so I did.

One of my goals was to allow you to create something with type 'table and have it seamlessly work just like existing tables. That goal has been accomplished.

I'll note that namespaces in my import.arc library are seen by Arc as... tables. Which is why w/namespace works on tables. There is no special 'namespace type, they're just tables. That's because I'm viewing types as an interface, and so my object.arc library makes that easy.

---

I'd also like to note that it really doesn't matter whether anybody likes my object.arc library or not. Let's suppose I was wrong and extend is fantastic and perfect and my object.arc library is just redundant... well, then we can get rid of it and just not use it.

On the other hand, let's suppose that extend is fantastic, but there are a few areas where it's clunky and verbose... then you can use extend most of the time, only loading up my library when you actually want the functionality it provides.

Before, it was about getting it into Arubic's core, so I had more of a vested interest in it. But now it's just a normal library, so we can simply use it or not as we wish.

However, regardless of whether I was right or wrong, I'm glad that I have at least caused some contemplation, and possibly changes in wart for the better.

-----

1 point by rocketnia 5188 days ago | link

"rocketnia was at one point describing rulebooks"

My rulebooks aren't what you're thinking, I think. They're only a more bureaucratic kind of 'extend. I tend to store extensible operators as mutable collections of extensions, which mostly comes in handy so I can store a reference to a rulebook somewhere (like in a variable in a namespace) before all the extensions have been introduced, and it'll automatically reflect those later extensions. I can also sort rulebooks to establish precedence or peek into rulebooks to troubleshoot individual rules without leaving the REPL.

-----

1 point by Pauan 5188 days ago | link

Right, I was referring to your def-extension-type: http://arclanguage.org/item?id=14338

Whether you would actually use such a macro in practice, I don't know, but it was relevant to my point that you can use a macro to make extending less verbose.

My apologies for using the term "rulebooks", I couldn't recall the name of the macro you were describing.

-----

2 points by rocketnia 5187 days ago | link

Oh, I implemented 'def-extension-type in lathe.js as lathe.deftype().

  // Define a constructor _.Seq and a rulebook _.iffirstRb.
  lathe.deftype( _, "Seq", "iffirstRb" );
  
  // Define a constructor _.Seq using an existing rulebook.
  lathe.deftype( _, "Seq", _.iffirstRb );
This is not to be confused with the 'deftype I was talking about in that thread. >.>

I use it in exactly two places so far, and the rest of the time I've been cheating by defining types in a more traditional OO JavaScript way. But I'm totally interested in figuring out things like this that could help. I think Traits.js is a pretty interesting option for this, and I'm not against message-passing at this point either. ^_^

-----

1 point by akkartik 5188 days ago | link

"if you have to extend things every time you create a new clone, for instance, that'll add up if you make multiple clones."

Yeah, it's a pain if you program in a prototype-based style.

"..at the time I made those remarks, extend was indeed much more verbose."

No, that example hasn't actually changed much. My earlier example had defcall and defset (http://arclanguage.org/item?id=14244). I believe versions also are in anarki. I deliberately left out fill-table and each because they were unnecessary. The only reason I bring it up now is that I feel able to discuss real code on the right column.

iso I only realized later wasn't necessary (and I still haven't gotten around to adding the default clause).

-----

1 point by Pauan 5188 days ago | link

"Yeah, it's a pain if you program in a prototype-based style."

Actually, it's a pain anytime you have a function that constructs objects, like the built-in `table` function in Arc. To work around that, you might give it a special type, like 'my-table and then use extend only once... but then where do you store the actual functions? The obvious place is to store them in a hash table... but then you're right back to objects. :P

---

"I deliberately left out fill-table and each because they were unnecessary."

You're right, that wasn't a fair comparison of wart. I apologize. I hadn't actually used wart, so I was going off of what you said, and also my experience with Arc/3.1, where you would actually need to extend things like iso, maptable, etc.

So perhaps my point should have been more along the lines of "god Arc/3.1 is too verbose with data types", since it would seem that ar and wart are both far better than Arc/3.1.

---

You said "coerce takes care of it" How? I don't see you using defcoerce anywhere in your example.

-----

1 point by akkartik 5188 days ago | link

"You said "coerce takes care of it" How? I don't see you using defcoerce anywhere in your example."

Yeah I ignored each in the comparison because it wasn't on the right column either. You'd need to specialize it to ignore the method fields, and maybe to iterate over the parents as well, right?

-----

1 point by Pauan 5188 days ago | link

Naw, it's not needed on the right column. `each` works out-of-the-box with my object.arc library. Though you could probably do the same thing with wart, if you wanted to.

Basically, if the object has a `keys` attribute, it uses that, otherwise it iterates over the object's attributes. So by extending maptable, my library gets keys, vals, and each, all for free.

This lovely property is because of Arc, though, not my library.

---

However, it does demonstrate that it's more convenient to simply create an object, rather than having to create a new type (like 'heir) and then use defcall, defcoerce, etc.

So the real point isn't that my library is perfect, the point is that our current way of thinking of types is flawed. We should either try to improve it (which my library does), or dump it entirely and use something completely different (rocketnia seems to prefer this approach).

Ultimately, I think the reason dealing with data types is clunky is because our type model is clunky. extend vs. objects is an interesting debate, but as you pointed out, with proper support, extend is more-or-less just as concise as objects in most circumstances.

So the real question is about dealing with types. How do we differentiate different things? My object.arc library provides one possible way. Using extend provides a slightly different way.

Because really, that's what my library is all about. If it's shorter than using extend, that's a nice bonus, but the reason I made it is so I can easily create an object that has a type of 'table and behaves like a table.

-----

3 points by Pauan 5188 days ago | link

So, I would like to take the time to apologize for the massive thread earlier, and all the arguing. You were right, akkartik, I should just go and do it and not worry about what you guys think. I wasted a lot of time and emotions on something that ultimately didn't matter a lot. Yes, using objects is better in some circumstances, and I still think there's some interesting stuff to explore here, but it's not worth arguing about.

I would also like to apologize for making it appear that message passing (with or without objects) was drastically better than wart. I was not trying to deceive, I really did think that things were that verbose, in large part because I was ignorant. Nonetheless, that's not an excuse for the things I said, or the way I said them.

-----

3 points by akkartik 5188 days ago | link

That's very classy of you! This thread helps me understand where you were coming from in that thread. And it helps me to know that there isn't something about message passing that I totally missed :)

Now let's go back to discussing import!

-----

1 point by akkartik 5188 days ago | link

How do we differentiate different things?

Indeed. I think I prefer creating new types because I don't like something that looks like a string but isn't. Rails, for example, has these things called scopes that encapsulate sql queries. If you try to print a scope you see an array. But if you treat it like an array you can end up in sixteen different subtle kinds of trouble.

So I'd rather have new types that coerce transparently to tables rather than have a non-table claim to be a table. But I can absolutely imagine being equally productive with a programming style that plays more fast-and-loose with prototypes so that different tables have subtly different behaviors.

-----

1 point by Pauan 5188 days ago | link

Yeah, but my idea is that types are interfaces. So if a scope is pretending to be an array, then by golly it better act like an array too. If it doesn't, then it's breaking the interface contract, so shame on it.

Then again, the current interfaces in Arc are fairly lax... conses mostly just need to support car, cdr, call, and set... tables need to support call, set, and keys.

So for making things that behave like tables, it just makes sense to give them a type of table, since the interface is so easy to adhere to. Like namespaces in import.arc, for instance. It makes sense to give them a type of table.

---

However, you're right that sometimes you might want something that's similar to, but slightly different than an existing type... in which case you'll probably want something else. On the OOP side you have subclasses and prototypes. On the Arc side you have... uh... extend, I guess, which is more general.

So let's imagine that object.arc is basically a thin skin over extend... in which case we might as well treat objects and extend as equivalent, since anything objects can do, extend can too.

With my types-are-interfaces idea, you could either give your new data a new type, or you could give it multiple types, or you could go rocketnia's approach and use `table?` predicates, or similar. I'd be interested in hearing about other approaches.

-----

2 points by Pauan 5188 days ago | link

By the way, this reminds me of Traits.js: http://traitsjs.org/

I remember being sorta-vaguely interested in it a long while ago, but never tried it. Now that I've learned Arc, all I can see is that the examples would be half as short (or shorter!) with macros. :P

If I were to strictly translate the example on their home page, it might look like this:

  (def make-color-trait (c)
    (trait color (fn () c)))

  (def make-point (x y)
    (trait-compose (make-color-trait 'red)
                   (trait getx  (fn () x)
                          gety  (fn () y)
                          print (fn () (string x '@ y)))))
                          
  (= p (make-point 0 1))
  (p 'color) -> 'red
Okay, not quite half as short, but even so. From what I can tell, traits seems to be an interesting way to combine objects together, according to a contract. Not saying this is "The Way", but it might be interesting to write a traits.arc library to play around with it.

---

Also, apparently traits were invented in Self... which also invented prototypes (!):

http://en.wikipedia.org/wiki/Trait_%28computer_programming%2...

-----

2 points by akkartik 5188 days ago | link

types are interfaces

Yes I remember noticing that word 'interfaces' in the old thread but not recent ones :) Interfaces seem useful in java and Go because there's some type-checking attached to it. If you're planning to add type-checking - so that objects claiming to be tables have to implement certain methods, or the object call immediately errors (gets _called_ on it ^-^) - that's an interesting direction. I won't immediately follow you but I'll watch to see where you end up :) But without type-checking there can be no notion of an interface. 'Duck-typed interfaces' is an oxymoron, I think.

-----

1 point by Pauan 5188 days ago | link

"If you're planning to add type-checking - so that objects claiming to be tables have to implement certain methods, or the object call immediately errors (gets _called_ on it ^-^) - that's an interesting direction."

I wasn't planning on it, no, and I think we can still gain uses out of types-as-interfaces even without type checking. However, Traits.js might just change my mind. But that would be more complicated than my simple object.arc library.

-----

2 points by aw 5188 days ago | link

I would personally treat a type like "table" to be an interface. So anything could be a table, as long as you could use it like a table.

Your example of something that claims to be an array but gives you trouble if you treat it like an array strikes me as more an issue with that particular bad implementation...

Interesting question though: what would treating types like interfaces look like from an Arc point of view? I'll have to think about that one :)

Update: I wrote this before I saw Pauan's comment :)

-----

1 point by akkartik 5188 days ago | link

:) In that rails example scopes don't claim to be arrays. Hmm, perhaps it's just that printing them on the console is confusing. Perhaps other forms of overloading aren't so bad, and perhaps I can even handle the console with a slightly different mindset - not judging values by how they print.

-----

1 point by aw 5187 days ago | link

Using extend or defrule with a test for type seems like a common pattern... perhaps we should extract that into its own macro.

-----

1 point by rocketnia 5187 days ago | link

Pauan and I were talking about something like that at https://convore.com/arc-runtime-project/getting-critical-cha..., but it was specific to defcall. I wonder if the idea of a type-table case could make sense for lots of operations.

In Lathe/arc, I filter on the first argument's type using "ontype" or multiple first arguments' types using "ontypes":

  (use-fromwds-as rc (+ lathe-dir* "orc/orc.arc"))
  (= my (nspace))
  
  ; These are already defined in orc.arc, but I'm redefining them here
  ; for the sake of example.
  
  (rc:ontypes rc.oiso2 (a b) (cons cons) my.cons
    (and (rc.oiso2 car.a car.b) (rc.oiso2 cdr.a cdr.b)))
  
  (rc:ontype rc.otestify () fn my.fn
    self)
  
  ; General pattern:
  ;
  ; (rc:ontypes <rulebook> <parameters>
  ;     (<type of a leading parameter> ...) <rule label (optional)>
  ;   <implementation>)
  ;
  ; (rc:ontype <rulebook> <parameters other than the first>
  ;     <type of the first parameter> <rule label (optional)>
  ;   <implementation, where "self" is the first parameter>)
https://github.com/rocketnia/lathe/blob/master/arc/orc/orc.a...

In orc.arc, this comes with an inheritance system I don't use much, but I've found I use 'ontype itself a lot when defining new types. Porting the inheritance system would be hard without porting Lathe's rule precedence system along with it.

In lathe.js, I recently tried adding lathe.instanceofRule( ... ), which defines rules that check the type of their first argument. This was largely a failure, as it clutters up the signature line so much that I end up spreading it over three lines of code, when often I could have had a one-line signature and a one-line explicit check. I haven't given up on it completely, though, since it decreases my token count slightly. :-p

-----

1 point by akkartik 5187 days ago | link

Do you mean like defgeneric?

Update: I've been mulling adding 'def :type' to wart for some time, so that I can say (def prn(x) :type 'integer ..) rather than (def prn(x) :case (isa x 'integer) ..).:

  (extend-macro def(name params &body body) :case (iso :type (car body))
    `(def ,name ,params :case (isa ,(car params) ,(cadr body))
       ,@(cddr body)))
But I often want to dispatch on different args. I'm not sure the first or last arg are sufficiently common to merit a shortcut.

-----

1 point by Pauan 5187 days ago | link

In my experience, it's usually the first argument, so I think having a :type would be useful, and in the cases where it isn't, you could fall back to :case (isa ...)

But of course, that's up to you.

-----

2 points by akkartik 5176 days ago | link

Yeah after holding off for a week I decided it would be useful.

https://github.com/akkartik/wart/commit/9559f27eec3994d9779b...

Update 2 hours later: Turns out if isa can handle a list of types, so can :type.

https://github.com/akkartik/wart/commit/1a8400dfebb16bd154da...

-----