Re: I have a dream: A modern scripting language everywhere

[Warren Young <wyml at etr-usa dot com>]

On Dec 9, 2014, at 3:32 AM, Corinna Vinschen <corinna-cygwin@cygwin.com> wrote:

> On Dec  8 15:59, Warren Young wrote:
>> 
>> Anyone else see a path out of this situation?
> 
> The neat thing with shell scripting is that it's relatively easy and by
> using tools like awk, sed, etc easy to extend in functionality(*).

True, but that’s also the biggest weakness of shell scripting: it takes a fork() to get almost anything of consequence done.

It was seen as a big advance when we got internal arithmetic in the shell instead of having to call out to bc.  Arithmetic is one of the core competencies of electronic computers going back to Konrad Zuse, yet shell scripts couldn’t do it internally until, what, ksh93? [1]

Mind, I do a lot of shell scripting, and I’m happy to have it, relative to the situation in stock Windows.

The POSIX shell environment might actually be the biggest draw of Cygwin for me.  Some might say it’s the package repository instead, but I suspect most Cygwin users are installing a relatively small set of non-Base packages.

But let’s get back on topic:

I don’t believe we would have kSLOC scale shell scripts if we had something more capable to write highly-portable programs in, which can sit in the gap between sh and C.

Imagine how much nicer the Autotools would be if they could count on something more powerful than POSIX.1.  It’s a big enough problem that it’s been independently re-solved several times, in unsatisfactory ways:

- CMake and Jam are written in C, so you have to build a separate tool before you can build the program you actually wanted, rather than just run a script under an interpreter already on the system.

- SCons and Bakefile require Python, and Ant requires Java.

None of these tools behave like the Autotools, just writing out a platform-independent set of scripts that run pretty much everywhere, except for Bakefile, which is built on top of Autotools.

> The complexitiy and inhibition
> level to learn a complex scripting language is higher, I think.

The gap between Bash and Perl/Python/Ruby is indeed large, but not so large as the gap between Bash and C.

> But the worst problem with non-shell scripting languages is that you
> introduce yet another level of "library" functions.

This is part of what I was getting at with my “too big” observations.

This cannot be a fatal flaw, though.  The success of C and C++ prove that it is possible to have a sincerely standardized language with a large-ish library.

A standardization effort focused on increasing the likelihood of widespread reimplementation to near-certainty with an expectation of high compatibility among implementations would have to keep a lid on the scope of the library, just as has been done for C and C++.

If Perl, Python or Ruby went through such a process, I’d expect them to leave quite a lot of the current “standard” library out of the formal standard.  This can be done successfully, as with .NET Micro Framework and J2ME. [2][3]

Just to pick on one random example, it wouldn’t be reasonable to expect Ruby’s gdbm module to survive a standardization effort.  Otherwise, even Ruby’s standard library is too big to expect anyone else to reimplement it.[4]

I carefully qualified the term “standard” above because I want to sweep aside such “standards” as the current .NET one, which is little more than a formal effort to document whatever Visual Studio happened to be able to compile at the time they started work.  We have Mono *despite* the ECMA and ISO standards, not as a result.

It’s looking like .NET is going to follow the Perl/Lua route.  Microsoft recently released enough of .NET under a liberal enough license that we may end up with just one widely-ported open source implementation, rather than a true standard in the C/sh sense.[5]

Python does have a bunch of independent reimplementations of the core language.  (PyPy, Iron Python, Jython…)  But, the “batteries included” philosophy of the library again puts it into the “too big” camp, unless you prune it way back as part of standardization.

Javascript and Scheme are both sincere standards in the C/sh sense, and thus have been widely reimplemented, with high levels of cross-compatibility.  Lua could be in the same camp, but the MIT license it’s released under makes formal standardization and reimplementation pointless.  These three are all *too* small to solve our problem here: they’d have to be considerably extended to compete with shell scripting.

> the shell itself never
> requires "extensions".  No "plugins”.

POSIX.1 is the shell’s standard library.

I don’t have to tell you that it’s pretty big. :)

> And then, years after creating all this stuff, a couple of core
> developers are really unhappy with what they created because it's
> missing some important feature, or it's too slow, or whatever, and they
> start to create "version 2" of their scripting language, deliberately or
> accidentally breaking backward compatibility with the already given
> ecosystem.  It's a mess.

This certainly does happen, but observe that D and Objective C have so far failed to kill off C and C++.

Also observe the resilience of Perl 5 and Python 2.  “New and better,” is correctly seen here as “breaks all my existing code.”



[1] Is that not where $((a+b)) and typeset -i came from?

[2] http://en.wikipedia.org/wiki/.NET_Micro_Framework

[3] http://en.wikipedia.org/wiki/Java_Platform,_Micro_Edition

[4] http://www.ruby-doc.org/stdlib-2.1.5/

[5] http://blogs.msdn.com/b/dotnet/archive/2014/11/12/net-core-is-open-source.aspx