
                LISP to UNIX  v1.0

                  Micheal Hewett
                   11 June 1995


1.  Introduction

  The set of functions in "lisp-to-unix.lisp" implements a low-level
  character-based protocol for communicating between a LISP program
  and a UNIX program.  This has been used to handle the interaction
  between several AI programs (in LISP) and their X/Motif display 
  programs.  It could also be used to communicate between a LISP 
  program and a number-crunching program, or between an expert agent
  and a WEB-searching program.  Note that this is a very low-level 
  protocol - the user typically implements one level of software above 
  this level.


2.  Using the LISP-to-UNIX package

  Say a LISP program wants to draw some diagrams on a screen.  The
  LISP program breaks the diagrams down into circles and squares,
  then calls (DRAW-CIRCLE x y radius), (DRAW-SQUARE x y width height),
  etc.  The DRAW-CIRCLE, etc., functions implement a graphics layer
  which is just above the communication layer.

  The operation of drawing a circle is handled by an X/Motif drawing
  program running on the same machine.  So the command needs to be 
  sent to the drawing program.  The DRAW-CIRCLE command writes the
  command to the external stream, where it is read by the drawing
  program.  



3.  Protocols

  a.  LISP to UNIX
  ----------------
  Most UNIX programs are more comfortable handling character input,
  so it works best if the LISP program sends character commands.
  A protocol definition is created, for example:

      'A'   -   Draw circle
      'B'   -   Draw line
      'C'   -   Draw square
      ...

  The DRAW-CIRCLE function in LISP then writes out the data (using
  EXTERNAL-RAW-OUTPUT):

    A 27 53 135                 - draw a circle at (27,53) of radius 135.

  This can be done using the (EXTERNAL-RAW-OUTPUT ...) function.

  All UNIX programs must recognize the following commands:

      'q'   -   Exit program
      's'   -   Start string - all characters until the next EOS
                               belong to a string.
      '\0'  -   EOS (End of string)


  b.  UNIX to LISP
  ----------------
  Conversely, the LISP program may need to receive input from the
  UNIX program.  LISP works better when receiving lists and symbols
  so output from the UNIX program is typically in that form.  For
  example, to send a notification of a mouse press, the following
  data might be written from the UNIX program:

     (:MOUSE-PRESS   312  45  :CLICKS 2 :TIME 1372736)

  The LISP program can then (READ) the external input stream and 
  retrieve the data.


4.  Streams

  In Lucid LISP and Allegro Common LISP, a low-level function will
  start a UNIX program from within LISP and set up streams (pipes)
  that are used to communicate with it.  The UNIX program reads
  input from STDIN and writes to STDOUT.  The LISP program receives
  either two streams (input and output) or one I/O stream with which
  to communicate with the UNIX program.  This is all handled by the 
  LISP-to-UNIX functions.

  In LISP-to-UNIX you have an EXTERNAL-STREAM object that you treat
  as a LISP stream.  You pass it into the low-level communication
  functions so that they can write to the appropriate external program.
  In this way, a LISP program can communicate with several external
  UNIX programs.


5.  Functions

  See the file "lisp-to-unix.lisp" for a complete list of functions
  (in case this documentation is out of date).  As of this date,
  the following functions are available.

  open-external-program       (path direction)

     Runs a UNIX program (located at "path") and opens streams
     in the appropriate direction(s).  Direction can be
     :INPUT, :OUTPUT, or :IO.  Returns an 'external-stream' structure.


  close-external-program   (external-stream-structure)

     Given an external-stream structure, closes the streams and
     sends the 'exit program command to the external program.


  external-flush              (external-stream-structure)

     Calls (finish-output ...) on the output stream of the
     external stream.

  external-read               (external-stream-structure)

     Reads one LISP form from the external input stream.


  external-raw-output         (external-stream-structure args...)

     Writes one or more LISP forms to the external output stream.


  external-string-output      (external-stream-structure format args...)

     Writes one or more formatted LISP forms to the external
     output stream.


  with-no-external-flush      MACRO: surrounds any of the above fns.

     Unless this function is used around the EXTERNAL-xx-OUTPUT
     functions, they do a (FLUSH ...) after each write.  A flush
     should only be done once after the entire command is sent,
     so when some raw output is followed by some string output,
     you should use this macro to surround all but the last call
     to an output function.


6.  Internals

  The only internal structure is the EXTERNAL-STREAM structure, which
  you do not need to know any details of.  There is one global
  variable, *external-flush-enabled*, which is bound by the
  WITH-NO-EXTERNAL-FLUSH macro, so you don't need to access it
  directly.


7.  Shell interaction

  Under Allegro CL and Solaris, the "run-shell-command" function
  starts up a shell in which to execute the external program.
  This may cause some problems if your shell initialization files
  (e.g. ".kshrc" or ".cshrc") execute some commands that expect
  an interactive shell.

  To bypass this problem, the lisp-to-unix code sets the shell variable
  LISP to T so that you can test for it in your initialization file.
  For example, in KSH, you can write:

     if [ "$LISP" = "" ]            -- if not in LISP
     then
          ...do whatever commands...
     fi


8.  Testing

  The function (TEST-L2U ...) in "lisp-to-unix.lisp" can be used to 
  test the interface.


9.  Packages

  Currently, all of the LISP-to-UNIX functions are in the CL-USER package,
  but this can be changed with no difficulty.


10.  Speed

  The interface is quite fast in practice, although remember that it
  is still doing I/O, so programs can get I/O bound.  For example,
  using the interface to write debugging info to another program does
  slow down the program noticeably.  On the other hand, sending screen
  drawing commands is pretty fast.
