feat(xsnap): Initial checkin from 34dfa4a

packages/xsnap/doc/XS Metering.md

@@ -0,0 +1,144 @@
+# XS Metering
+Revised: November 10, 2020
+
+Warning: These notes are preliminary. Omissions and errors are likely. If you encounter problems, please ask for assistance.
+
+## Introduction
+
+The objective is to allow runtime to constraint how much computation a machine can do.
+
+The technique is to count how many byte codes are executed and to ask the runtime if the limit has been reached.
+
+Asking the runtime at every byte code would be prohibitively slow. So XS only asks the runtime if the limit has been reached:
+
+- when branching backwards,
+- when calling a function,
+- when returning from a function,
+- when catching an exception,
+- when iterating a generator,
+- when resuming an async function.
+
+To be faster, the runtime can also set a metering *interval*, a number of byte codes to wait for before asking again.
+
+When the runtime tells XS that the limit has been reached, XS aborts with a "*too much computation*" exit. Like for other exits ("*not enough memory*", "*unhandled exception*", etc), the runtime can then decide what to do:
+
+- throwing an exception,
+- exiting the machine,
+- exiting the process.
+
+> Both exiting the machine and exiting the process cannot be caught by the executed JavaScript code.
+
+## Programming Interface
+
+To begin metering use the `xsBeginMetering` macro:
+
+	xsBeginMetering(xsMachine* machine,
+					xsBooleanValue (*ask)(xsMachine* the, xsUnsignedValue index),
+					xsUnsignedValue interval)
+
+- `machine`: the machine to meter.
+- `ask`: the C function that XS will call to ask if the limit has been reached.
+- `interval`: the metering interval.
+
+The macro uses `setjmp` and must be balanced with the `xsEndMetering` macro:
+
+	xsEndMetering(xsMachine* machine)
+
+- `machine`: the metered machine.
+
+The `ask` callback gets the metered machine and the current index. It returns `1` to tell XS to continue, `0` to tell XS to abort.
+
+## Built-ins
+
+To fine tune the metering, runtimes can patch built-ins functions.
+
+	xsPatchHostFunction(xsSlot function, xsCallback patch)
+
+- `function`: the function to patch.
+- `patch`: the callback that replaces the original callback of the function.
+
+Patches must conclude by running their original callback with
+
+	xsMeterHostFunction(xsUnsignedValue count)
+
+- `count`: the number to add to the metering index.
+
+### Example
+
+Here is a patch for `Array.prototype` functions that adds the length of the array to the metering index.
+
+	void fx_Array_prototype_meter(xsMachine* the)
+	{
+		xsIntegerValue length = xsToInteger(xsGet(xsThis, xsID("length")));
+		xsMeterHostFunction(length);
+	}
+
+The same patch can be installed into several `Array.prototype` functions, for instance here `Array.prototype.reverse` and `Array.prototype.sort`
+
+	xsBeginHost(machine);
+	xsVars(2);
+	xsVar(0) = xsGet(xsGlobal, xsID("Array"));
+	xsVar(0) = xsGet(xsVar(0), xsID("prototype"));
+	xsVar(1) = xsGet(xsVar(0), xsID("reverse"));
+	xsPatchHostFunction(xsVar(1), fx_Array_prototype_meter);
+	xsVar(1) = xsGet(xsVar(0), xsID("sort"));
+	xsPatchHostFunction(xsVar(1), fx_Array_prototype_meter);
+	xsEndHost(machine);
+
+## Usage
+
+Here is the typical runtime sequence:
+
+	static xsBooleanValue ask(xsMachine* machine, xsUnsignedValue index)
+	{
+		if (index > 10000) {
+			fprintf(stderr, "too much computation\n");
+			return 0;
+		}
+		return 1;
+	}
+
+	int main(int argc, char* argv[]) 
+	{
+		//...
+		xsMachine* machine xsCreateMachine(creation, "main", NULL);
+		xsBeginMetering(machine, ask, 1000);
+		{
+			xsBeginHost(machine);
+			{
+				// execute scripts or modules
+			}
+			xsEndHost(machine);
+		}
+		xsEndMetering(machine);
+		xsDeleteMachine(machine);
+		//...
+	}
+
+The fxAbort function has to be supplied by all runtimes based on XS. Here the `xsTooMuchComputationExit` case exits the machine.
+
+	void fxAbort(xsMachine* the, int exit)
+	{
+		if (exit == xsTooMuchComputationExit) {
+			fxExitToHost(the);
+		}
+		//...
+	}
+
+### In JavaScript
+
+The runtime must provide a C function for the `ask` callback. However the `ask` callback can use the XS in C programming interface to call a JavaScript function. Like a system callback, the `ask` callback has to use `xsBeginHost` and `xsEndHost`.
+
+	static xsBooleanValue ask(xsMachine* machine, xsUnsignedValue index)
+	{
+		xsBooleanValue result;
+		xsBeginHost(machine);
+		{
+			result = xsToBoolean(xsCall1(xsGlobal, xsID_ask, xsNumber(index));
+		}
+		xsEndHost(machine);
+		return result;
+	}
+
+The metering is suspended during the `ask` callback.
+

packages/xsnap/doc/XS Snapshots.md

@@ -0,0 +1,135 @@
+# XS Snapshots
+Revised: September 24, 2020
+
+Warning: These notes are preliminary. Omissions and errors are likely. If you encounter problems, please ask for assistance.
+
+## Format
+
+XS snapshots are atom based. Inside the `XS_M` container, there are nine atoms:
+
+- `VERS`: The version of XS and the architecture. 
+- `SIGN`: The runtime signature. 
+- `CREA`: The parameters to create the machine: block initial and incremental size, heap initial and incremental size, stack size, etc.
+- `BLOC`: The chunks in the blocks.
+- `HEAP`: The slots in the heaps.
+- `STAC`: The slots in the stack.
+- `KEYS`: The keys table.
+- `NAME`: The names table.
+- `SYMB`: The symbols table.
+
+XS snapshots are bound to:
+
+- the major and minor version numbers of XS,
+- the architecture: 32-bit vs 64-bit, big endian vs little endian, 
+- the runtime signature.
+
+## Programming Interface
+
+The `xsSnapshot` structure is used to read and write snapshots.
+
+	typedef struct {
+		char* signature;
+		int signatureLength;
+		xsCallback* callbacks;
+		int callbacksLength;
+		int (*read)(void* stream, void* ptr, size_t size);
+		int (*write)(void* stream, void* ptr, size_t size);
+		void* stream;
+		int error;
+		void* data[3];
+	} xsSnapshot;
+
+
+- `signature`: bytes to identify the runtime.
+- `signatureLength`: the number of bytes in `signature`.
+- `callbacks`: array of XS callbacks implemented by the runtime.
+- `callbacksLength`: the number of XS callbacks in `callbacks`.
+- `read`: the function that `xsReadSnapshot` will call to read the snapshot.
+- `write`: the function that `xsWriteSnapshot` will call to write the snapshot.
+- `stream`: the parameter passed to the `read` and `write` functions.
+- `error`: the error that occurred when reading or writing the snapshot.
+- `data`: pointers used internally by XS, must be set to `NULL`.
+
+The `signature` and `callbacks` fields are related. Runtimes must change the `signature` if the `callbacks` become incompatible. 
+
+If the `stream` is a binary `FILE*`, the `read` and `write` functions are trivial:
+
+	int read(void* stream, void* ptr, size_t size)
+	{
+		return (fread(ptr, size, 1, stream) == 1) ? 0 : errno;
+	}
+
+	int write(void* stream, void* ptr, size_t size)
+	{
+		return (fwrite(ptr, size, 1, stream) == 1) ? 0 : errno;
+	}
+
+### Writing Snapshot
+
+Here is the typical runtime sequence:
+
+- create a new machine,
+- run modules or scripts,
+- wait for all jobs to complete, 
+- write the snapshot.
+
+To write the snapshot, fill the `xsSnapshot` structure and call  `xsWriteSnapshot`.
+
+	extern int xsWriteSnapshot(xsMachine* the, xsSnapshot* snapshot);
+
+- `xsWriteSnapshot` must be called outside of XS callbacks and outside of `xsBeginHost` `xsEndHost` blocks. 
+- There must be no host instances.
+
+`xsWriteSnapshot` returns `1` if successful, otherwise `xsWriteSnapshot` returns `0` and sets the `error` field.
+
+### Reading Snapshot
+
+Once you have a snapshot, instead of creating a new machine with `xsCreateMachine`, you can create a new machine with `xsReadSnapshot`. The new machine will be in the same state as the machine that was saved by `xsWriteSnapshot`.
+
+To read the snapshot, fill the `xsSnapshot` structure and call  `xsReadSnapshot`.
+
+	xsMachine* xsReadSnapshot(xsSnapshot* snapshot, xsStringValue name, void* context);
+
+- `snapshot`: The snapshot structure.
+- `name`: The name of the machine to be displayed in **xsbug**.
+- `context`: The initial context of the machine, or `NULL`
+
+`xsReadSnapshot` returns a machine if successful, otherwise `xsReadSnapshot` returns `NULL` and sets the `error` field.
+
+## Implementation Details
+
+To be able to write a snapshot, everything must be in chunk blocks, slot heaps and slot stack. There cannot be pointers to host data.
+
+That was mostly the case, except for a few optimizations that create JS strings pointing to C data. Such optimizations are now skipped if `mxSnapshot` is defined. We should investigate if such optimizations are still necessary.
+
+Interestingly enough snapshots are completely orthogonal to the shared machine prepared by the XS linker to flash micro-controllers. The strategy there is to have as many pointers to host data as possible...
+
+### Callbacks
+
+The only pointers that cannot be avoided are XS callbacks, i.e. pointers to host code.
+
+The major and minor version numbers of XS change when byte codes evolve and when built-ins get new features. New features are always implemented with new callbacks. 
+
+Snapshots are obviously bound to major and minor version numbers of XS. For the sake of snapshots, XS maintains an array of callbacks. It is then enough to project XS callbacks into array indexes. 
+
+Similarly, thanks to a signature and an array of callbacks, runtimes can add new callbacks that will be projected into array indexes.
+
+### Strictly Deterministic
+
+If two machines with the same allocations perform the same operations with the same results in the same order, their snapshots will be the same.  
+
+The XS garbage collector is always complete and introduces no variations.
+
+But asynchronous features can of course alter the order. Then the snapshots will not be the same, even if they are functionally equivalent.
+
+### Tests
+
+A lot of tests remain to be done to verify how various built-ins survive the snapshot process.	
+
+
+
+
+
+
+
+

packages/xsnap/makefiles/lin/makefile

@@ -0,0 +1,10 @@
+.NOTPARALLEL:
+
+all: debug release
+
+debug:
+	make -f xsnap.mk
+
+release:
+	make GOAL=release -f xsnap.mk
+