Arc Forum | Jarc 15 Released (some benchmark comparisons)

Arc Forum

Jarc 15 Released (some benchmark comparisons)

5 points by jazzdev 5323 days ago | 2 comments

I wanted to do some benchmarking and ran into some stack overflow issues. This release corrects those problems. It was mostly changes to arc.arc, but I also had to add a compiler optimization to look for

  (assign foo (fn (...) ... (foo ...)))

and do tail-call optimization. The function merge in arc.arc uses this pattern. The release is available in the usual place:

  http://sourceforge.net/projects/jarc/files/

So now Jarc will run the benchmark in arc.arc

  Jarc> (repeat 5 (let td (n-of 10000 (rand 100)) (time (sort < td)) 1))
  time: 410 msec.
  time: 200 msec.
  time: 182 msec.
  time: 156 msec.
  time: 156 msec.

It gets faster after a few runs (because of the JIT compiler?) and gets to the speed of Arc 3.1:

  arc> (repeat 5 (let td (n-of 10000 (rand 100)) (time (sort < td)) 1))
  time: 165 msec.
  time: 162 msec.
  time: 161 msec.
  time: 161 msec.
  time: 161 msec.

But rainbow still kicks butt!

  arc> (repeat 5 (let td (n-of 10000 (rand 100)) (time (sort < td)) 1))
  time: 152 msec.
  time: 92 msec.
  time: 77 msec.
  time: 73 msec.
  time: 73 msec.

It starts at the speed of Arc 3.1 and then gets even faster! My hat's off to you, conanite.

2 points by rocketnia 5295 days ago | link

I only just downloaded and tried out Jarc 16 (which you don't have a thread for, but you mentioned at http://arclanguage.org/item?id=11915), and I noticed this:

  Jarc> (do:compose idfn idfn)
  (stdin):1: Error: Type-error:
    compose is not of type LIST

I was using do:compose in some generated code as a way to force the macro meaning of 'compose, as opposed to the special meaning (compose ...) has in functional position.

There's an easy workaround--(do (compose ...))--and there's an easy fix too. In Jarc's utils.arc, 'expand-metafn is slightly different from arc.arc's 'expand-metafn-call: Instead of checking the caar of the remaining arguments with (iscar (car fs) 'compose), it checks the car with (is (safecar fs) 'compose). So it doesn't actually splice together nested 'compose calls; it just sometimes causes an error taking the cdr of the symbol 'compose. :)

As long as I'm looking at the source for 'expand-metafn, the if clause for 'andf seems a bit off:

        (if (car f) 'andf)
        (cons 'and (map (fn (f) (cons f args)) (cdr f)))

I think you mean (is (car f) 'andf) there. ^_^ Also, that implementation of 'andf evaluates the outer arguments a variable number of times. The Arc 3.1 version--I haven't looked at others--evaluates them exactly once by expanding like this instead:

  ((andf a b) c d)
  =>
  ((fn (gs1 gs2) (and (a gs1 gs2) (b gs1 gs2))) c d)

Furthermore, the parameters of 'if (and 'and, by extension) should be macro-expanded when the (if ...) form is compiled, but Jarc doesn't do this.

Whew! Here's a transcript highlighting some of those differences:

Jarc 16:

  Jarc> (mac foo args (prn "expanding foo with " args) `',args)
  #3(tagged mac #<procedure>)
  Jarc> (and (foo) (foo "true"))
  expanding foo with nil
  nil
  Jarc> (if (foo) (foo "true"))
  expanding foo with nil
  nil
  Jarc> (foo&foo)
  expanding foo with nil
  nil
  Jarc> (foo&foo "true")
  expanding foo with ("true")
  expanding foo with ("true")
  ("true")
  Jarc> (foo&foo (do prn!evaluating "true"))
  expanding foo with ((do prn!evaluating "true"))
  expanding foo with ((do prn!evaluating "true"))
  ((do prn!evaluating "true"))
  Jarc> (idfn&idfn (do prn!evaluating "true"))
  evaluating
  evaluating
  "true"

Arc 3.1 (but your gensyms may vary; I load Lathe in my libs.arc):

  Use (quit) to quit, (tl) to return here after an interrupt.
  arc> (mac foo args (prn "expanding foo with " args) `',args)
  #(tagged mac #<procedure: foo>)
  arc> (and (foo) (foo "true"))
  expanding foo with ()
  expanding foo with (true)
  nil
  Jarc> (if (foo) (foo "true"))
  expanding foo with ()
  expanding foo with (true)
  nil
  arc> (foo&foo)
  expanding foo with ()
  expanding foo with ()
  nil
  arc> (foo&foo "true")
  expanding foo with (gs1999)
  expanding foo with (gs1999)
  (gs1999)
  arc> (foo&foo (do prn!evaluating "true"))
  expanding foo with (gs2000)
  expanding foo with (gs2000)
  evaluating
  (gs2000)
  arc> (idfn&idfn (do prn!evaluating "true"))
  evaluating
  "true"

-----

1 point by conanite 5317 days ago | link

Hi jazzdev, thanks for the hats-off :))

There are a bunch of benchmark tests with rainbow under src/arc/lib/bm-tests.arc - I wanted to cover different aspects of the system's performance (error handling and continuations for starters; I'd like to add tests specifically for conditionals, quasiquotes, different kinds of invocations (with/without optionals, destructuring etc)). Rainbow performs better in some cases, and arc3.1 beats rainbow in other cases. I'll have to grab the latest jarc and run a 3-way comparison - more news later!

-----