Copyright 1998 Lars T Hansen.

$Id$

	     README file for the Larceny FFI implementation

Status
------

Callbacks are not fully implemented -- the integration with call/cc is
not finished.  Don't use callbacks yet.

(This readme may be slightly out-of-date, please report bugs.)


Introduction
------------

To use the ffi, it's easiest to load Ffi/ffi-load.sch and then evaluate

	(load-ffi <path>)

where <path> is the name of the Ffi directory, with a trailing directory
separator.  <tt>Load-ffi</tt> loads all the FFI files and loads the
appropriate libc.  It returns three values: an architecture identifier,
a callout-abi structure, and a callback-abi structure.

If you take a look in Ffi/examples.sch, you'll see an example of this.
The code in that file should in any case be very instructive.


Structure
---------

Larceny's FFI has two levels.  At the lower level, defined by tramp.sch
and ffi-lower.sch, the central procedures are

	ffi/make-callout
	ffi/make-callback
	ffi/convert-arg-descriptor
	ffi/convert-ret-descriptor
	ffi/apply

At the higher level, defined by ffi-upper.sch, the central procedures
are

	ffi/libraries
	ffi/foreign-procedure

In addition, there are the ABI objects, defined by ffi-sunos4.sch and
ffi-sunos5.sch

	ffi/SPARC-sunos4-C-callout-stdabi
	ffi/SPARC-sunos4-C-callback-stdabi
	ffi/SPARC-sunos5-C-callout-stdabi
	ffi/SPARC-sunos5-C-callback-stdabi

and also two utility procedures that return the name of libc on the
particular system:

	ffi/sun4-sunos4-libc
	ffi/sun4-sunos5-libc

It is easiest to work on the higher level, or to define abstractions on
top of the high level.  Auxlib/std-ffi.sch is one such abstraction.


Files
-----

The FFI consists of seven files, where six are OS-independent and the
remaining one is not.  The OS-indepentent files are, from lowest level
to highest:

memory.sch

  This file defines non-moving memory allocation primitives,
  pointer-to-integer conversion, and a gcprotect mechanism.  The code in
  this file is highly experimental and it would be (ahem) unwise to rely
  too heavily on it at this point in time.

  The following procedures are probably of general interest:

	make-nonrelocatable-bytevector : length -> bytevector
	cons-nonrelocatable : object * object -> pair
	make-nonrelocatable-vector : length -> vector
	make-nonrelocatable-vector : length * init -> vector
	ffi/gcprotect : object -> handle
	ffi/gcprotect-increment : handle -> ()
	ffi/gcunprotect : handle -> ()

tramp.sch

  This file defines low-level common code for trampoline construction and 
  manipulation.  The main functions are 

	ffi/make-callout :
		ABI * func-address * arg-desc * ret-desc -> trampoline

	  The ABI is an ABI object as defined in the description of
	  ffi-sparc.sch and extended in ffi-sunos*.sch, below.
	  The func-address is the address of the foreign function.
	  Arg-desc is a list of argument descriptor symbols, and ret-desc
	  is a return type descriptor symbol.

	ffi/make-callback :
		ABI * procedure * arg-desc * ret-desc -> trampoline

	  The ABI is an ABI object (see above).  Procedure is a Scheme
	  procedure.  Arg-desc is a list of argument descriptor symbols,
	  and ret-desc is a return type descriptor symbol.

ffi-sparc.sch	

  This file defines most of the callout and callback ABI objects for C/C++
  calling conventions on the SPARC architecture.  An ABI object is
  represented as a Scheme procedure that takes a selector argument and
  returns an operator procedure that implements the operation.  The first
  argument to the operator is always a trampoline object (see the
  description of tramp.sch, above); other arguments depend on the
  operation.  The ABI object definitions are extended by operating-system
  specific operators in the files ffi-sunos4.sch and ffi-sunos5.sch,
  below.

  Callout ABI.

  The callout ABI object protocol functions are listed below.  Argument
  operators should be called in left-to-right order followed by the
  'done' operator, and then the 'done-pasteup' operator.  The return
  operators can be called any time before the 'done' operator.

  The callout ABI is used to create a procedure that can be called from
  C and that will call a C procedure with arguments as specified
  by the operations on the trampoline object, and that will extract the
  specified return value.  The conversion is handled outside the
  trampoline; see the source code for a detailed explanation.

	alloc : () -> trampoline
          Allocates a fresh trampoline.

        arg-word : trampoline -> ()
	  Generates code to pass a word (unsigned32 or signed32) argument.

        arg-ieee32 : trampoline -> ()
	  Generates code to pass a single-precision float argument.

        arg-ieee64 : trampoline -> ()
	  Generates code to pass a double-precision float argument.

	ret-word : trampoline -> ()
	  Generates code to capture a word (unsigned32 or signed32) return
	  value.

	ret-ieee32 : trampoline -> ()
	  Generates code to capture a single-precision float return value.

	ret-ieee64 : trampoline -> ()
	  Generates code to capture a double-precision float return value.

	ret-void : trampoline -> ()
	  Generates code to ignore the return value.

	done : trampoline -> ()
	  Finishes up code generation.

        done-pasteup : trampoline -> ()
	  Usually called by the pasteup procedure in the trampoline driver
	  after the code vector has been generated and locked in memory,
	  this operator performs instruction cache flushing on the code
	  addresses if required by the architecture.


  Callback ABI.

  The callback ABI object protocol functions are listed below.  The callback
  object is used to construct a C procedure that will receive some C data
  and invoke a Scheme procedure after first converting the arguments.

  Argument operators should be called in the left-to-right order of the
  constructed C function's argument list, followed by 'done' and 
  'done-pasteup'.  'Arg-types' and 'ret-types' should be called before 'done'.

	alloc : () -> trampoline
          Allocates a fresh trampoline.

        arg-word : trampoline -> ()
	  Generates code to pass a word (unsigned32 or signed32) argument.

        arg-ieee32 : trampoline -> ()
	  Generates code to pass a single-precision float argument.

        arg-ieee64 : trampoline -> ()
	  Generates code to pass a double-precision float argument.

        proc-addr : trampoline * address -> ()
	  Defines the procedure address of a Scheme procedure.

        arg-types : trampoline * bytevector -> ()
	  Defines an argument descriptor that will be decoded by static 
	  code in Larceny's run-time system to convert C values to Scheme
	  values.  Normally, the bytevector is generated by 
	  ffi/convert-arg-descriptor, in ffi-lower.sch (see description
	  below).  The bytevector should NOT be altered subsequently.

        ret-type : trampoline * symbol * integer -> ()
	  The integer code defines the foreign type to which the Scheme
	  return value must be converted upon return from the callback.
	  The symbol is the name of the return type.  The return
	  encoding is usually constructed by ffi/convert-ret-descriptor,
	  in ffi-lower.sch (see description below).

        done : trampoline * integer -> ()
	  Finishes up code generation.  the integer argument denotes the
	  number of arguments that the caller will pass to the callback.

        done-pasteup : trampoline -> ()
	  Usually called by the pasteup procedure in the trampoline driver
	  after the code vector has been generated and locked in memory,
	  this operator performs instruction cache flushing on the code
	  addresses if required by the architecture.

ffi-lower.sch

  This file defines primitives for loading, linking, argument and return
  type encoding, and foreign function invocation.

	ffi/dlopen : string -> { integer, #f }
	  The string names a dynamic library.  The library is brought into
	  memory (loaded) in some implementation-specific manner; on SunOS4
	  and SunOS5, the dlopen(3X) procedure is used.  If #f is
	  returned, then the linking failed; otherwise, the return value
	  denotes a handle on the dynamic library.

	  NOTE: Under SunOS4, handing the name of a nonexistent library to
	  dlopen() will result in termination of the process.  I don't
	  know how to fix this.

	  NOTE: I don't think the current implementation works on SunOS5,
	  because the code is commented out in Rts/Sys/ffi.c.

	ffi/dlsym : integer * string -> { integer, #f }
	  The integer argument is a dynamic library handle, and the string
	  names the symbol whose address we wish to obtain.  If an
	  integer is returned, it is the address of the linked-in entity;
	  otherwise, the linking failed.

	ffi/convert-arg-descriptor : ABI * arg-descriptor -> arg-encoding
	  Given a list of argument representation types, returns a
	  bytevector that encodes these types in a format more suitable
	  for decoding by low-level code (i.e., the FFI procedures in 
	  the C run-time).
          A better name would have been ffi/encode-argument-types.

	ffi/convert-ret-descriptor : ABI * ret-descriptor -> ret-encoding
	  Given a return value representation type, returns an integer
	  that encodes that type.
          A better name would have been ffi/encode-return-type.

	ffi/apply : trampoline * arg-encoding * ret-encoding * actuals 
		-> ( boolean, value )
	  Given a trampoline that encodes a call-out, and argument and
	  return encodings as produced by ffi/convert-arg-descriptor and
	  ffi/convert-ret-descriptor, and a list of actual Scheme arguments
	  for which conversion rules to C encodings exist: convert the
	  arguments, invoke the foreign function, and convert the return
	  value.  If the call-out did not signal an error, then the
	  first return value is #t and the second is the value returned.
	  if the call-out did signal an error, the the first return value
	  is #f and the second is an error code (not currently defined).
	  
ffi-upper.sch

  This file defines most of the useful high-level abstractions on top of the
  lower-level functions.  Normally, the programmer interacts with the FFI
  at this (or a higher!) level, by defining the libraries with ffi/libraries
  and then linking in procedures with ffi/foreign-procedure.

	ffi/libraries : () -> string list
	ffi/libraries : string list -> string list
	  Called with no arguments, ffi/libraries returns the list
	  of currently installed libraries.  Called with one argument,
	  which must be the full list of libraries, it installs that
	  list as the list of libraries to use.  It then returns that
	  list.

	ffi/load-libraries : ABI -> library list
	  Called with an ABI, ffi/load-libraries loads the libraries in
	  the currently installed list that have not yet been loaded,
	  sets up some globals to record the changes, and returns
	  an association list (also kept in the global *ffi/loaded-libraries*)
	  that maps library names to their handles.

	ffi/link-procedure : ABI * string -> integer
	  Given an ABI and a procedure name, loads the named procedure
	  from the installed libraries, trying them in an unspecified
	  order.  If the procedure can't be found, an error is signalled.

	ffi/foreign-procedure : ABI * string * arg-desc * ret-desc -> procedure
	  Given an ABI, a procedure name, an argument type list, and a return
	  type, returns a Scheme procedure that will call the foreign 
	  function with some arguments, and return its return value.

ffi-util.sch

  This file contains utility functions for data conversions.

	ffi/string->asciiz : string -> bytevector
	ffi/asciiz-length : bytevector -> integer
	ffi/asciiz->string : bytevector -> string


OS dependencies.

  A full ABI object extends the definition given above by adding the
  operators listed below.  These operators implement operating-system and
  compiler specific functionality in the ABI objects: loading, linking,
  name mangling, and an aspect of the callback protocol.

  Callout ABI protocol

	load-lib : string -> { integer, #f }
	  Loads the library and returns its handle, or #f if the library
	  could not be loaded (and the loader did not kill the process).

	link-proc : integer * string -> { integer, #f }
	  Given a library handle and a string, try to obtain the address
	  of the entity named by the string, returning the address if
	  found and #f otherwise.

  Callback ABI protocol

	callback-addr : () -> integer
	  Return the address of a C procedure that will invoke a Scheme
	  procedure.  The details are deeply implementation-specific and
	  purposely left undocumented.  The address is only useful during
	  construction of a callback trampoline, and the operator is public 
	  only so that we can abstract the trampoline construction away from
	  some implementation details.  Perhaps it should be placed 
	  somewhere else.

Currently, two OS-dependent files exist:

ffi-sunos4.sch

  This file defines two full ABI objects:
	ffi/SPARC-sunos4-C-callout-stdabi
	ffi/SPARC-sunos4-C-callback-stdabi
  These ABI objects implement standard name mangling for C on SunOS 4,
  namely, prepending "_" to the name.

  Also, it defines a procedure
	ffi/sun4-sunos4-libc : () -> string
	  Returns the correct file name for libc.so on the system.  On
	  SunOS 4, this may involve a minor search because the libc
	  has a version number in its name.

ffi-sunos5.sch

  This file defines two full ABI objects:
	ffi/SPARC-sunos5-C-callout-stdabi
	ffi/SPARC-sunos5-C-callback-stdabi
  These ABI objects implement standard name mangling for C on SunOS 4,
  namely, there is no name mangling.

  Also, it defines a procedure
	ffi/sun4-sunos5-libc : () -> string
	  Returns the correct file name for libc.so on the system.
