Building Larceny


WHAT YOU NEED

Most users of Larceny will never need to build Larceny from source,
and can ignore this documentation.  See README-FIRST.txt for a more
detailed explanation.

If you want to recompile the Scheme sources entirely from source,
you will need the source code plus an R5RS Scheme system that can
host Larceny's build process.  At this time, the R5RS systems we
recommend are plt-r5rs (distributed as part of Racket) or another,
possibly older instance of Larceny itself [1].  (Please don't ask
Larceny to overwrite itself during the build process.  That doesn't
work.)

If you want to build Larceny or Petit Larceny entirely from source,
you will also need a C compiler (such as gcc).  For x86-32 targets,
you will also need the NASM assembler [2] and the necessary 32-bit
library packages [3].

To build Common Larceny, see the Common Larceny documentation in
doc/CommonLarceny/user-manual.txt.

Larceny's build process was streamlined in Larceny 2.0.  Most of
what follows below assumes you are building a 32-bit version of
Larceny (v1.5 or earlier).  To build a 32-bit version of Larceny,
you will need to start with a different pre-built copy of Larceny.
(Using plt-r5rs instead has been tested in Larceny v2.0, but only
for building a 64-bit version of Larceny.)


BUILDING ON APPLE SILICON (macOS/arm64): THE SHORT VERSION

From the Larceny root directory,

    % src/Build/bootstrap.sh --native --r7rs      # native Larceny
    % src/Build/bootstrap.sh --petit --r7rs       # Petit Larceny

builds everything with Racket as the host (--larceny PATH-TO-LARCENY
uses an existing Larceny instead; --clean removes the build products
of another configuration first).  The rest of this section describes
the steps the script performs.


BUILDING 64-BIT PETIT LARCENY ON APPLE SILICON (macOS/arm64)

Petit Larceny can be built as a 64-bit system (64-bit words, 62-bit
fixnums) for macOS on Apple Silicon, using Racket (v8 or later, with
its plt-r5rs language) to host the build instead of a prebuilt
Larceny.  The C compiler is Apple's clang (invoked as cc/gcc).

 1. From the Larceny root directory, run plt-r5rs with --no-prim and
    build the bootstrap system:

        % plt-r5rs --no-prim
        > (load "setup.sch")
        > (setup 'scheme: 'racket 'host: 'macosx-arm64)
        > (build-config-files)
        > (load-compiler)
        > (build-heap)
        > (build-runtime)
        > (build-executable)
        > (build-larceny-files)
        > (build-larceny-heap)
        > (exit)

    (Racket's plt-r5rs is in the bin directory of the Racket
    installation, e.g. "/Applications/Racket v9.3/bin/plt-r5rs".)

 2. (build-larceny-heap) creates the heap image; it runs

        % ./petit-larceny.bin -stopcopy -heap petit.heap \
              -- src/Build/petit-larceny-heap.sch

 3. Compile the R7RS/R6RS/SRFI libraries as in step 4 below:

        % ./larceny
        > (require 'r7rsmode)
        > (larceny:compile-r7rs-runtime)
        > (exit)

The 64-bit system uses the same object representations as the 32-bit
system, scaled to 8-byte words, except that flonums and compnums have
no pad word after the header, and fixnums have 62 bits.  Heap memory
is allocated from a reserved region of virtual address space (64 GB by
default; set LARCENY_ARENA_GB to change it).  Petit Larceny's write
barrier now follows the same protocol as native Larceny's, so the
regional collector (-rrof) works.  Code compiled by earlier versions
of Petit Larceny must be recompiled.  The FFI supports callouts (made
from C, in src/Rts/Sys/ffi.c, on arm64 and x86-64) but not callbacks.
The non-predictive collector (-np) is not implemented.


BUILDING NATIVE LARCENY ON APPLE SILICON (macOS/arm64)

Native Larceny for AArch64 uses the Fence/Cant code generator (see
src/Asm/Fence/pass5p2-arm64.sch) and, like the 64-bit Petit Larceny,
uses 64-bit words and 62-bit fixnums.  It is bootstrapped with Racket
(see "Build configurations", below, if the tree has been used for other
builds).  The script

        % src/Build/bootstrap-arm64-native.sh

performs steps 1 and 2 below; or, by hand:

 1. From the Larceny root directory:

        % plt-r5rs --no-prim
        > (load "setup.sch")
        > (setup 'scheme: 'racket 'host: 'macosx-arm64 'native)
        > (build-config-files)
        > (load-compiler)
        > (build-heap)
        > (build-runtime)
        > (build-executable)
        > (build-larceny-files)
        > (build-larceny-heap)
        > (exit)

 2. (build-larceny-heap) creates the heap image; it runs

        % ./larceny.bin -stopcopy -heap arm64.heap \
              -- src/Build/arm64-macosx-larceny-heap.sch

 3. Compile the R7RS/R6RS/SRFI libraries as in step 4 below.

MacOS X on Apple Silicon does not allow memory that is both writable
and executable, so compiled code lives in a separate code space: a
semispace of MAP_JIT memory that belongs to the oldest non-static
generation and is garbage collected, by copying, together with that
generation.  (The regional collector, -rrof, has no such generation:
there the code space belongs to the static generation, and code is
collected by a separate pass over the heap when much code has been
created.)  Heap images keep code in their text segment.  The code
space's address space can be set with LARCENY_CODE_GB (default 1 GB).


BUILDING 64-BIT PETIT LARCENY ON x86-64 LINUX

Petit Larceny builds on x86-64 Linux as on Apple Silicon (same 64-bit
representations), with gcc, and Racket's plt-r5rs (v8 or later) or an
existing Larceny as the host.  For native Larceny, see below.

        % src/Build/bootstrap.sh --petit --r7rs

or by hand as in steps 1 to 3 of the macOS section, with

        > (setup 'scheme: 'racket 'host: 'linux-x86-64)

The runtime is compiled with -std=gnu17: gcc 15 defaults to C23, in
which bool is a keyword, but the runtime defines its own int-sized
bool.  On a small machine (one CPU, 1.6 GB of memory, some swap) the
Racket-hosted build takes about nine minutes, and compiling the
R7RS/R6RS libraries about as long; neither ran short of memory.


BUILDING NATIVE LARCENY ON x86-64 LINUX

Native Larceny for x86-64 uses the Fence/Cant code generator with the
x86-64 back end (src/Asm/Fence/pass5p2-x86-64.sch), and like the
AArch64 system uses 64-bit words, 62-bit fixnums, and a code space
(here mapped read-write-execute; heap images keep code in their text
segment).  With Racket as the host,

        % src/Build/bootstrap.sh --native --r7rs

or by hand as for AArch64, with

        > (setup 'scheme: 'racket 'host: 'linux-x86-64 'native)

The heap is built from x86-64.heap by
src/Build/x86-64-linux-larceny-heap.sch.  On a small machine the build
takes about four minutes and compiling the R7RS/R6RS libraries about a
minute and a half.

Forward branches always take 32-bit displacements (the assembler makes
one pass); see TODO.md.  Twobit has only 8 registers on x86-64.


BUILD CONFIGURATIONS

Build products are written into the source tree, and must not be mixed
between configurations (Petit vs native, 32 vs 64 bits, and so on).
setup records the configuration in the file build-configuration, and
refuses to proceed if the tree was set up for a different one, or holds
build products but no record.  Then either

    > (setup ... 'clean)

removes the build products (using git to find them) before proceeding,
or

    > (setup ... 'keep-build-products)

keeps them and records the new configuration.  The Scheme that hosts
the build is not part of the configuration, so, for example, a tree
built with Racket as the host can be rebuilt with Larceny as the host.


WHAT TO DO

Building Larceny has four major steps:

1. Set up the Larceny development environment.

Run your host Scheme (a prebuilt version of Larceny [1]), and change
to the root directory of the Larceny tree.

    > (load "setup.sch")
    > (setup 'scheme: ${SCHEME} 'host: ${HOST} [ 'sassy | 'common ]
             'string-rep: ${STREP})

The call to setup (above) has several options that let you
declare which implementation of Scheme you want to use for
the build process (SCHEME) and which OS you are using as
both the host and target of the build (HOST):

        SCHEME      ::= 'larceny | 'racket | 'mzscheme | 'petite | 'chez
        HOST        ::= 'macosx | 'macosx-el | 'macosx-arm64 | 'linux86
                      | 'win32 | 'linux-arm-el
        STRING-REP  ::= 'flat4  |  'flat1

The HOST option macosx-el is for Intel-based Macintosh computers.
The HOST option macosx-arm64 is for Apple Silicon Macintosh computers;
it builds a 64-bit Petit Larceny (see above).
The HOST option macosx is for PowerPC-based Macintosh computers.

The HOST option linux-arm-el is for most ARM-based Linux systems.

The optional 'sassy flag will build native Larceny (x86-32) on
Linux, Windows, or MacOS X.  The 'sassy flag is necessary if
you want to use dynamic loading on Win32, and is strongly
recommended for that operating system.  There are several
other optional flags that change the compiler's behavior [4].

The STRING-REP option determines the representation used
for strings.  The flat4 representation uses 4 bytes per
character, is the default on all versions of Larceny, and
is the only representation that can be used when building
Petit Larceny or Common Larceny.  The flat1 representation
of strings uses 1 byte per character, which limits it to
strings of Latin-1 characters.  Native Larceny for x86-32
can be built to use either the flat4 or flat1 representations;
be sure to perform a full build when switching from one
representation to the other.  At present, native Larceny
on ARM can only be built with the flat4 representation.

Some useful examples which are known to work:

To build native Larceny:
    > (setup 'scheme: 'larceny 'host: 'win32 'sassy 'always-source)
    > (setup 'scheme: 'larceny 'host: 'linux86 'sassy)
    > (setup 'scheme: 'larceny 'host: 'linux86 'sassy 'string-rep: 'flat1)
    > (setup 'scheme: 'larceny 'host: 'macosx-el 'sassy)

To build native Larceny for little-endian ARMv7 running Linux
an explicit target: will be required [5]:

    > (setup 'scheme: 'larceny
             'host: 'linux-arm-el
             'target: 'linux-arm-el-hardfp 'native)

To build native Larceny for ARMv7 target platforms that reserve
the ARM platform register, specify 'linux-arm-el-hardfp0 instead
of 'linux-arm-el-hardfp.

To build Petit Larceny:
    > (setup 'scheme: 'larceny 'host: 'linux86 'always-source)


2. Build Larceny components

After the development environment is setup, you should compile the core
system:

    > (build-config-files)              ; [6]
    > (load-compiler)
    > (build-heap)
    > (build-runtime)
    > (build-executable)

You will now have a working Larceny interpreter and minimal heap image.

To build a complete Larceny user system, including the Twobit compiler:

    > (build-larceny-files)

You may want the Twobit compiler development environment; on
Petit, you must build it separately:

    > (build-twobit)

After all that, it's time to exit from your host Scheme:

    > (exit)

For Windows, you'll need to mark the larceny.bin.exe as incompatible
with NX:

    editbin.exe /NXCOMPAT:NO larceny.bin.exe

Larceny will not run under Windows with DEP set to AlwaysOn, but
Larceny will run with DEP set to OptOut, OptIn, or AlwaysOff.


3. Create heap images

Step 2 above creates a bootstrap heap (petit.heap, arm.heap, or
sasstrap.heap).  To create the user and development heap images, exit
the host Scheme back to your command shell.  Then perform one of the
following three sets of commands (omitting those for twobit unless
you want the Twobit compiler development environment):

On IAssassin Larceny (Intel) [7]:

    % "./larceny.bin" -stopcopy -- src/Build/iasn-larceny-heap.fasl
    > (exit)

    % "./larceny.bin" -stopcopy -- src/Build/iasn-twobit-heap.fasl
    > (exit)

On Native Larceny (ARM):

    % ./larceny.bin -stopcopy -- src/Build/arm-el-hardfp-larceny-heap.fasl
    > (exit)

    % ./larceny.bin -stopcopy -- src/Build/arm-el-hardfp-twobit-heap.fasl
    > (exit)

On Petit Larceny:

    % ./petit-larceny.bin -stopcopy -- src/Build/petit-larceny-heap.sch
    > (exit)

    % ./twobit.bin -stopcopy -- src/Build/petit-twobit-heap.sch
    > (exit)

Then, if you want the Twobit compiler development environment:

  On *NIX:
    % cp larceny twobit

  On Windows:
    > copy larceny.bat twobit.bat

On Windows machines, you will probably need to tell DEP to allow
larceny.bin to opt-out.

You can now run "./larceny" (and maybe "./twobit") in its default
-r5rs mode, but the -r7rs and -r6rs modes won't work until you
do step 4.



4. Compile the R7RS/R6RS/SRFI standard libraries.

Run Larceny in its default -r5rs mode, and compile the
R7RS/R6RS/SRFI standard libraries as follows.

    % ./larceny
    Larceny v0.98 "Funny in the Head" (...)

    > (require 'r7rsmode)
    > (larceny:compile-r7rs-runtime)
    > (exit)

Larceny should now run correctly in all modes.  If you want
to use Larceny to run Scheme scripts on a Unix system, then
you should make sure that the Larceny root directory is in
your execution path, ahead of any other directories that
may contain a file named scheme-script.



----------------------------------------------------------------

NOTES

[1] For the build process, you need a prebuilt version of Larceny
itself, or you can use Racket's plt-r5rs.

[2] http://www.nasm.us/

For Windows and Ubuntu Linux, we use NASM version 2.  For MacOS X,
we use NASM version 0.98.40, which is installed with the Apple
Developer command line tools.

For the 'sassy backend on 'win32, Microsoft's development tools
(including NMAKE) and NASM must both be in your search path.

[3] To build or run 32-bit Larceny On Linux machines, you may need
to do the following:

    sudo apt-get install lib32z1
    sudo apt-get install libc6-i386
    sudo apt-get install libc6-dev-i386

On MacOS X machines, you may need to install the Apple Developer
command line tools.

On Windows, the Microsoft Visual Studio Community 2013 download
contains everything you need (and a lot more than you need).  We
use the "VS2013 x86 Native Tools Command Prompt".

[4] Larceny requires CPU support for floating point instructions.
Several sub-ABIs exist for floating point on the ARM architecture.
The two APIs are known as "soft" (hardware instructions are generated but
the calling convention is as for software floating point; gcc calls
this "softfp") and "hard" (hardware instructions are generated and the
calling convention uses FPU registers).  Larceny needs to know the ABI
when it compiles C code, and the target name therefore contains either
"softfp" or "hardfp".  Generally the operating system sets the ABI and
all programs on the system use it; check your local documentation.

[5] Some other potentially useful flags, given optionally at the end of
the call to setup:

    'code-cov
        This enables loading of the instrumented files generated by the
        stcov tool when loading the compiler. Use this to test code
        coverage of the compiler testsuite

    'rebuild-code-cov
        This causes the stcov files to be regenerated before loading.
        This implies 'code-cov.

    'always-source
        Don't load compiled fasl files when loading the development
        environment under Larceny.  This is useful under Petit Larceny
        if the executable gets out of sync with the fasl files.

    'nasm
        Build a Petit Larceny which targets the NASM assembler (IA32)
        rather than Standard C.  This option is deprecated in favor of
        the Sassy backend; indeed, we suspect that it no longer builds
        correctly now that we have added the string-rep: parameter to
        setup.

[6] This is only strictly necessary the first time and when the options
given to setup have changed, but it should be safe to do every time.

[7] The quotation marks around "./larceny.bin" are a uniform syntax that
works on both Microsoft Windows and UNIX-like shells.  The reason is
left as an exercise for the reader.

