"I don't think there is a better candidate for an IDE target than Eclipse. It is very pretty."
Eclipse is ugly as sin.
Eclipse belongs to that school of UI design that further divides and subdivides the available space into smaller and smaller little tiles and panes until you are squinting at your code through a keyhole and distracted by all the stuff surrounding it. Yes, there are ways around this, but you have to fight the default paradigm. Most other IDEs adopt the same paradigm (to varying degrees).
Also, your whole point about IDEs does not account for the growing popularity of Python and Ruby. I know you point to an online Ruby interpreter, but these languages rose to popularity with most noobies first experiencing them through a text editor + shell/REPL environment. Stop and think about being a noobie downloading Eclipse and firing it up for the first time. What is the first step? How many steps will it take to see a useful result of your labor?
Versus an intro tutorial of:
python
>>> print "Hello world"
Hello World
Congratulations! You're now a Python programmer! (Hope I didn't mess that up.)
Sorry for the rant, but I think many IDEs are largely a cargo cult phenomenon. pg has a quote somewhere about finally figuring out what an IDE is for; something like "it generates all the code that your Lisp macros generate for you!" IDEs are largely band aids to make bad languages not hurt quite as much.
I've used Visual Studio, JBuilder, Eclipse, etc., and for the languages I used them for (C++, C#, Java), I felt the IDEs were essential. When I switched to Ruby, the same quality of IDEs weren't available, but I found I was much more productive in Ruby with a simple editor than I had been with other languages plus an IDE, and I actually enjoyed the lighter weight environment quite a bit.
Would I be even more productive in Ruby with an IDE? Possibly. But it's possible that more powerful languages lessen the need for an IDE. Your love of ACL for Common Lisp would be a counter data point, but it's only one data point. Well, Avi Bryant would tout the Squeak IDE as a great advantage in using Smalltalk. In my case, it's somewhat moot given the lack of IDEs for Ruby. As I develop more in Lisp, I'll be better able to judge the effectiveness of the available IDEs.
Tens of thousands of lines of Ruby haven't been a problem for me with just vim, and now Emacs, and I don't think hundreds of thousands of lines would be a problem either if designed well.
"I found I was much more productive in Ruby with a simple editor than I had been with other languages plus an IDE"
Bingo, and that is why we do not see fancy IDEs for agile languages. The corollary being horrid languages have great IDEs because lawdy we need something!
One thing going on here is reflection. If my language has that I can toss off my own rough IDE-ish hacks in minutes and then they do exactly what I want, too, so the demand just never develops for off-the-shelf IDEs.
I work with tens of KLOC of Common Lisp at a time. I like sitting in a backtrace and using a keychord to jump to a functions definition, or sitting in the inspector and jumping to a class definition in source or with a diffeent keychord to the class in a graphical class browser. ACL integrates everything (except, strangely "who calls?", tho that is available via an unexported function), so I just sail all over my big code bases as fast as I can think, click, and keychord. Some folks, btw, feel the same about Lispworks and Slime, we need an IDE smackdown some day to comapre. :)
