DIRECTORY-DOES-TOO-MUCHDIRECTORY returns a list of truenames, "one for each file in the file system that matches the given pathname".
The problem is that sometimes one wants the truenames for just one or a few of the files that match, or one wants to interleave processing of each file as it is found, to minimize the start-up time when processing large directories.
DIRECTORY-GENERATOR which would accept the same argument spectrum as DIRECTORY and return a function which, when successively applied, would yield each of truenames in the list of truenames that DIRECTORY would have returned, and then NIL to indicate no more files are available.
(DEFUN FIND-DEFINING-FILE (MUMBLE) (LET ((FN (DIRECTORY-GENERATOR "/moby-dir/*.lisp"))) (DO ((TRUENAME (FUNCALL FN) (FUNCALL FN))) ((NULL TRUENAME) NIL) (WHEN (FILE-DEFINES-P TRUENAME MUMBLE) (RETURN TRUENAME)))))
This example shows how a system with some distributed processing ability might interleave file accessing and processing:
(DEFUN COMPILE-WORLD () (LET ((FN (DIRECTORY-GENERATOR "/moby-dir/*.lisp"))) (DO ((TRUENAME (FUNCALL FN) (FUNCALL FN))) ((NULL TRUENAME) NIL) (INITIATE-DISTRIBUTED-COMPILATION TRUENAME))))
(DEFUN FOO (X) (OR (NOT (PATHNAMEP X)) (NULL (SET-EXCLUSIVE-OR ; why doesn't CL have SET-EQUAL ? (DIRECTORY X) (LET ((FN (DIRECTORY-GENERATOR X)) (DIR '())) (DO ((TRUENAME (FUNCALL FN) (FUNCALL FN))) ((NULL TRUENAME) (REVERSE DIR)) (PUSH TRUENAME DIR)))))))
CL primitive that maps more directly to underlying OS primitives.DIRECTORY in much this way.DIRECTORY-GENERATOR as:
(DEFUN DIRECTORY-GENERATOR (X) (LET ((DIR (DIRECTORY X))) #'(LAMBDA () (POP DIR))))
Implementing it more directly is probably either a fairly small task or clearly impossible. Either way, not much work.
DIRECTORY continues to be needlessly inefficient in some cases.DIRECTORY now lists them. This is a minor restriction but would fail for systems that explicitly sort their results or file systems that randomly reorder directories (e.g. on every access). Set equivalence is probably just as good a test if anyone cares.