Re: Module Considerations

[Aubrey Jaffer <jaffer@martigny.ai.mit.edu>]

   Date: Tue, 14 Oct 1997 03:17:18 -0400
   From: Jim Blandy <jimb@red-bean.com>
   References: <199710131708.NAA16353@totoro.red-bean.com>
	   <m0xKwhm-000cuUC@aubrey.jaffer>
   X-UIDL: f93c8f79a821b349f1afb2f55e61cabc

   Is there a reason you replied only to me, and not to the list?  If
   not, I'd like to forward your message, and my follow-up, to the list.

I have advocated against search-paths for years; it is not news.  But
the explanation of the cached catalogs is new and might be interesting
to others -- so go ahead.

   I find the sheer volume of guile@cygnus.com upsetting sometimes, but I
   want to discuss matters of Guile's design in public as much as
   possible, even though I don't feel obliged to make everyone happy.
   I'd rather wade through a bunch of mildly flamy messages than be
   secretive.


   Date: Tue, 14 Oct 1997 03:13:58 -0400
   From: Jim Blandy <jimb@red-bean.com>
   References: <199710131708.NAA16353@totoro.red-bean.com>
	   <m0xKwhm-000cuUC@aubrey.jaffer>
   X-UIDL: 2c4497935616beedbe46404858ce07a3

   >This is a bad idea.  Search paths can only screw up (with unintended
   >name clashes) and eat lots of time compared to computing the one true
   >path.  SLIB maintains caches of module-name to files-to-load maps
   >called "slibcat" and "implcat", both located in the
   >implementation-vicinity.  SLIB require will recompute the cached maps
   >whenever "slibcat" is deleted or missing.

   I'm not sure I understand what you mean here.  I'm feeling a bit dumb,
   but I can't quite figure it out from the SCM docs or the SLIB sources.
   May I ask a few questions, to figure out the approach you've got in
   mind?

   - SLIB recomputes slibcat from a static list of modules and filenames
   stored in mklibcat.scm.  To install a new module, you need to
   update mklibcat.scm.  Is that correct?

Yes.  "slibcat" is for SLIB packages.  It is updated with every SLIB
release.  "slibcat" contains the SLIB version string.  "require.scm"
checks this version against the value of the identifier *SLIB-VERSION*
whenever "slibcat" is loaded.  If they don't match, "slibcat" is
rebuilt.

Each Scheme implementation using a shared SLIB builds its own
"slibcat".  This is because filename conventions vary slightly between
implementations.  By the way, the identifier *catalog* should be saved
with value #f when dumping (unexec) an executable; this way SLIB
doesn't go stale.

   - SLIB loads implcat, if it exists.  I don't see any provisions for
   computing it on demand.  Is this correct?

"implcat" is rebuilt whenever "slibcat" is rebuilt.  The way to
trigger rebuilding is to delete "slibcat" (remember that "slibcat" is
per Scheme implementation).

   - What is the suggested procedure by which a system administrator
   should install a package intended for use by all the system's users?

If by system you mean all SLIB users:
	Modify "slib/mklibcat.scm";
	then change *SLIB-VERSION* at the top of "slib/require.scm".
If by system you mean SCM:
	Modify "scm/mkimpcat.scm";
	then delete "scm/slibcat".
If by system you mean GUILE:
	Put package files into a directory on the path-list;
	delete "guile/implcat".
It is quite reasonable to write "guile/mkimpcat.scm" to grovel down
paths looking for packages -- the difference between this and
searching for every LOAD is that the searching will only done when the
system configuration changes.

   Once installed, how should users gain access to the package?

(require '<package-name>)

   - What is the suggested procedure by which an individual on a
   multi-user machine should install a package for her use alone, but
   which will not be visible to other users (perhaps because it is not
   stable, or unpopular)?

The only code dealing with "mkimpcat" is the following from the end of
"slib/mklibcat.scm":

(let ((mkimpcat (in-vicinity (implementation-vicinity) "mkimpcat")))
  (cond ((not (file-exists? mkimpcat))
	 (set! mkimpcat (string-append mkimpcat (scheme-file-suffix)))))
  (cond ((file-exists? mkimpcat)
	 (slib:load-source mkimpcat)
	 )))

How about creating "$HOME/usercat" (by loading "$HOME/mkusrcat.scm")
whenever it is missing or when the creation date of "slibcat" changes?

  Or is this a wrong-headed thing to want to do?

This should be supported.

   - Setting aside performance issues (which I agree are relevant, but
   I'm trying to figure something else out here), is it bad design to
   have an open-ended list of directories which a scheme system will
   check for modules?

Yes.  Large programs like WB (btree database with SCM interface) often
have files for internal use with collision-prone names like "all.scm",
"test.scm", "stats.so", and "sys.so".  Putting a directory like wb
(which actually has these files) into a search path will cause much
grief for users.  Requiring byzantine directory structures for
packages only creates more opportunity for screwups.

WB is a good example of how "implcat" works.  "mkimpcat.scm" has the code:

	(cond ((or
		(add-link 'db
			  (in-vicinity wb:vicinity "db.so"))
		(add-link 'db
			  (in-vicinity wb:vicinity "db" link:able-suffix)
			  (in-vicinity wb:vicinity "handle" link:able-suffix)
			  (in-vicinity wb:vicinity "blink" link:able-suffix)
			  (in-vicinity wb:vicinity "prev" link:able-suffix)
			  (in-vicinity wb:vicinity "ent" link:able-suffix)
			  (in-vicinity wb:vicinity "sys" link:able-suffix)
			  (in-vicinity wb:vicinity "del" link:able-suffix)
			  (in-vicinity wb:vicinity "stats" link:able-suffix)
			  (in-vicinity wb:vicinity "blkio" link:able-suffix)
			  (in-vicinity wb:vicinity "scan" link:able-suffix)
			  (usr:lib "c")))
	       (add-source 'wb-table
			   (in-vicinity wb:vicinity "wbtab"))
	       (add-alias 'wb 'db)))

This COND deals with the cases of an archived dll "db.so" existing,
multiple separate dlls ("handle.o") existing, and just running WB from
the Scheme code (which I use for debugging).  Notice that only the
multiple dlls requires libc.

LINK:ABLE-SUFFIX is a parameter to the ADD-LINKS routine.  During
implcat generation, it is bound to the appropriate suffix for each
dynamic-linking configuration.

Another example shows how mkimpcat.scm can control the order of
system libraries linked in dynamically:

	(add-link 'turtle-graphics
		  (in-vicinity (implementation-vicinity) "turtlegr"
			       link:able-suffix)
		  (x:lib "X11")
		  (usr:lib "m")
		  (usr:lib "c"))

Changing the order breaks it.

   - Is it unacceptably inefficient to search a list of directories to
   find a module?  This is a matter of opinion, but I would like to hear
   yours.

Yes, but you might be able to fix the speed.  My objection to
search-paths is that they DO THE WRONG THING!