anywho, i took a glance at the code. i know you're going for performance so maybe that affected the mechanisms you used, but here's some suggestions anyways:

* Lua's string concatenation operator is .., as in

  return "str1" .. "str2"

  local tinsert = table.insert

* string.gsub might be faster than iterated replacement

* a key in a table can be anything, even functions and other tables, for the hex2int you can therefore have something like:

  { ["1"] = 1, ["2"] = 2, ... ["a"] = 10 }

* as far as ipairs is concerned, a nil value signifies the end of a table, so if you have:

  { 1, 2, nil, 3, 4 }

* table.concat might be useful somewheres

* a map-like function where on each iteration you return not just a value, but a key-value pair might be useful. eg pretending you don't care about performance, reverse could be written:

  function reverse( a )
      local len = #a
      return tblmap( a, function( k, v )
          return len-k+1, v
      end )
  end

* one way i've set up bi-directional tables before is with __call. in that form, tbl[key] would access one way, and tbl(key) would access it in reverse. another option is something like tbl[1][key]/tbl[-1][key]. depending on how you're using the table, you can memoize in one of the directions and/or set up some sort of little database hidden behind that single table

* Lua doesn't have macros, but it does have first-class environments. among many other things this means it has anaphoric macro-like capabilities, except for the need to pass in expressions through closures or strings. take for example these lines from luarc.lua:

  local sep     = sep or ' '
  local pattern = string.format ('[^%s]+', sep)
  local f       = string.gmatch (s, pattern)
  local match   = f()

  function seq( ... )
      local that = nil
      local original_env = getfenv( select( 1, ... ) )
  
      for i = 1, select( '#', ... ) do
          local fn = select( i, ... )
  
          local new_env = { ["that"] = that }
  
          -- have new environment inherit from old, otherwise
          -- any global variables used in the fns will be unavailabe
          setmetatable( new_env, { __index = original_env } )
  
          setfenv( fn, new_env )
  
          that = fn()
      end
  
      return that
  end

  function seq( ... )
      local that = nil
  
      for i = 1, select( '#', ... ) do
          local fn = select( i, ... )
  
          that = with( { ["that"] = that }, fn )()
      end
  
      return that
  end

  local match = seq(
      function() return sep or " " end,
      function() return string.format( "[^%s]+", that ) end,
      function() return string.gmatch( s, that ) end,
      function() return that() end )

  do
      local test_env = {}
  
      local i = 0
      setmetatable( test_env, { __index = function( tbl, var )
          if var == "var" then
              i = i + 1
              return i
          else
              return _G[var]
          end
      end } )
  
      setfenv( 0, test_env )
  end


  > print( var )
  1
  > print( var )
  2
  > print( var )
  3

http://www.lua.org/manual/5.1/manual.html

but the user never sees any of that. if you want an array or tuple you type

  local blah = { 1, "two", function return 3 end }

  local blah = { bleh = "yep", bleagh = "whatever" }

  -- both of the following are valid
  print( blah["bleh"] )
  print( blah.bleh )

  local blah = { "my", "name", "is", [12] = "slim", [888] = "shady" }

  local blah = { "element 1", "element 2", name = "son", lastname = "goku" }

  local blah = { { "star", "bucks" }, { "apple", "sauce" } }

  local blah = { [{ 1, 3, 5 }] = "odd", [{ 2, 4 }] = "even" }

and keep in mind that Lua is likely the fastest interpreted language... or at least it was before the Javascript optimization race started. i think one of the Javascript engines has features inspired by the Lua VM

and then there's metatables, which are properties that can be set to control the behavior of values in different situations. for example, one of the properties is __index, which is referred to when a table doesn't have a value for a given key. this enables straightforward memoization, inheritance, hidden data management, infinite data structures, etc

other metatable properties determine the operation of the value under addition, comparison, etc

you could, for example, implement complex numbers as if they were part of the language, simply by creating a seed table called i. when an expression such as 1 + 2 * i is reached, the __mul property would perform the complex multiplication, then return a new Complex table which would in turn perform the addition. along each step it would be a normal table, so you could do, for example:

  print( (1 + 2 * i).polar_form )

  print( 8 * i^5 + 2 * i )

i've been thinking of making an APL-like DSL (would be nice to be able to implicitly map and whatnot) in this sort of manner. i haven't looked deep into Ruby but i believe it has things of this nature

here's a description of metatables:

http://www.lua.org/manual/5.1/manual.html#2.8

the fact that tables are used everywhere is unoverstatable. environments are tables, therefore you can do powerful things with environments. like writing your own import/include function, reading and writing global state in a file, anaphora and implicits... pretty much a bunch of crazy shit

along with this some very well-chosen features such as coroutines, closures, the implementation of most language features as modules (eg file.open, coroutine.wrap) to keep the syntax clean, etc

Well, I'm still playing with Arc, and particularly still love its webapp's model, but I'm really fed up with the lack of libraries especially when I want to do real work, so I actually wanted to see if its web server could be easily implemented in another (good) language. Well, it can, even if it's not as elegant as Arc's code.

What I've learnt :

First, macros are really great, I miss them ; Lua has extensions if you want to play with macros, but I didn't use them here

I already wrote about that, now I'm sure : Arc has too many basic datatypes. It has string, symbols and characters, Lua only has strings, and that's clearly enough ; a lot of code in Arc is converting strings, symbols and characters into each other.

What do you want to be designed? One of the major problems about "designing" a new lisp is that, since anyone can "do it" in their spare time, they don't see the point. Maybe they're right. ^^

As for strings and symbols, I don't really know how they are actually implemented, but as far as I know, the idea is that 'foo and 'foo are the same memory location, while "foo" and "foo" are not necessarily the same object(thus allowing string mutation). Or maybe I'm wrong ?

Lua's strings are implemented the same way : "foo" is always the same location as any other "foo" and string manipulation really doesn't seem to be a problem at all...

    (= foo '((key1 . val1) (key2 . val2)))
    (= bar (cons '(key2 . different-val2) foo))
    bar
    => ((key2 . different-val2) (key1 . val1) (key2 . val2))

Probably means we want to have some sort of "source" slot too, so that we can display shadowed values. Hmm. The association sublists e.g. '(key1 . val1) can probably be directly shared, but lists also imply an ordered set, so we need to store that info too. Hmm.

>As for strings and symbols, I don't really know how they are actually implemented, but as far as I know, the idea is that 'foo and 'foo are the same memory location, while "foo" and "foo" are not necessarily the same object(thus allowing string mutation)

This is correct. And when you really look at it, changing strings as if they were arrays of characters is hardly ever done in Arc; usually what's done is we just read them off as an array of characters and build a new string.

> In any case, characters just seem useless...

  >  => ((key2 . different-val2) (key1 . val1) (key2 . val2))

Two symbols 'foo can always be considered to mean the same thing, two strings "foo" can't, they may be only the same by coincidence. This matters if automated refactoring tools become available because you can safely rename all occurrences of a symbol, the same cannot be said of strings.