[GR-20912] [GR-29807] [GR-32443] Support value sharing across context…

…s; Support using the embedding API from guest languages.

PullRequest: graal/9194

docs/reference-manual/embedding/embed-languages.md

@@ -16,10 +16,12 @@ permalink: /reference-manual/embed-languages/
 * [Access Restrictions](#access-restrictions)
 * [Build Native Images from Polyglot Applications](#build-native-images-from-polyglot-applications)
 * [Code Caching Across Multiple Contexts](#code-caching-across-multiple-contexts)
+* [Embed languages in Guest Languages](#embed-languages-in-guest-languages)
 * [Step Through with Execution Listeners](#step-through-with-execution-listeners)
 * [Build a Shell for Many Languages](#build-a-shell-for-many-languages)
 * [Configure Sandbox Resource Limits](#configure-sandbox-resource-limits)
 
+
 The GraalVM Polyglot API lets you embed and run code from guest languages in JVM-based host applications.
 
 Throughout this section, you will learn how to create a host application in Java that
@@ -401,9 +403,7 @@ Caching may be disabled explicitly by setting [cached(boolean cached)](https://
 
 Consider the following code snippet as an example:
 
-```java
-import org.graalvm.polyglot.*;
-
+```
 public class Main {
     public static void main(String[] args) {
         try (Engine engine = Engine.create()) {
@@ -434,6 +434,46 @@ with "js" language, which is the language identifier for JavaScript.
 an explicit engine assigned to it. All contexts associated with an engine share the code.
 - `context.eval(source).asInt()` evaluates the source and returns the result as `Value` instance.
 
+## Embed Guest languages in Guest Languages
+
+The GraalVM Polyglot API can be used from within a guest language using Java interoperability.
+This can be useful if a script needs to run isolated from the parent context.
+In Java as a host language a call to `Context.eval(Source)` returns an instance of `Value`, but since we executing this code as part of a guest language we can use the language-specific interoperability API instead.
+It is therefore possible to use values returned by contexts created inside of a language, like regular values of the language.
+In the example below we can conveniently write `value.data` instead of `value.getMember("data")`.
+Please refer to the individual language documentation for details on how to interoperate with foreign values.
+More information on value sharing between multiple contexts can be found [here](https://www.graalvm.org/sdk/javadoc/org/graalvm/polyglot/Context.Builder.html#allowValueSharing-boolean-).
+
+Consider the following code snippet as an example:
+
+```java
+import org.graalvm.polyglot.*;
+
+public class Main {
+    public static void main(String[] args) {
+        try (Context outer = Context.newBuilder()
+                                   .allowAllAccess(true)
+                               .build()) {            
+            outer.eval("js", "inner = Java.type('org.graalvm.polyglot.Context').create()");
+            outer.eval("js", "value = inner.eval('js', '({data:42})')");
+            int result = outer.eval("js", "value.data").asInt();
+            outer.eval("js", "inner.close()");
+
+            System.out.println("Valid " + (result == 42));
+        }
+    }
+}
+```
+
+In this code:

+- `Context.newBuilder().allowAllAccess(true).build()` builds a new outer context with all privileges.
+- `outer.eval` evaluates a JavaScript snippet in the outer context.
+- `inner = Java.type('org.graalvm.polyglot.Context').create()` the first JS script line looks up the Java host type Context and creates a new inner context instance with no privileges (default).
+- `inner.eval('js', '({data:42})');` evaluates the JavaScript code `({data:42})` in the inner context and returns stores the result.
+- `"value.data"` this line reads the member `data` from the result of the inner context. Note that this result can only be read as long as the inner context is not yet closed.
+- `context.eval("js", "c.close()")` this snippet closes the inner context. Inner contexts need to be closed manually and are not automatically closed with the parent context.
+- Finally the example is expected to print `Valid true` to the console.
+
 ## Build a Shell for Many Languages
 
 With just a few lines of code, the GraalVM Polyglot API lets you build

sdk/CHANGELOG.md

@@ -2,6 +2,9 @@
 
 This changelog summarizes major changes between GraalVM SDK versions. The main focus is on APIs exported by GraalVM SDK.
 
+## Version 21.3.0
+* Added the ability to share values between contexts. Please see  `Context.Builder.allowValueSharing(boolean)` for further details. 
+
 ## Version 21.2.0
 * `AllowVMInspection` is enabled in the native launchers, `SIGQUIT` can be used to generate thread dumps. Performance counters are disabled by default, they can be enabled in the graalvm enterprise by the `--vm.XX:+UsePerfData` option.
 * Changed behavior of `Value.as(TypeLiteral<Function<Object, Object>>).apply()`: When the function is called with an `Object[]` argument, it is passed through as a single argument rather than an array of arguments.

sdk/src/org.graalvm.polyglot/snapshot.sigtest

@@ -139,7 +139,7 @@ meth public void leave()
 meth public void resetLimits()
 meth public void safepoint()
 supr java.lang.Object
-hfds ALL_HOST_CLASSES,EMPTY,NO_HOST_CLASSES,UNSET_HOST_LOOKUP,impl
+hfds ALL_HOST_CLASSES,EMPTY,NO_HOST_CLASSES,UNSET_HOST_LOOKUP,currentAPI,dispatch,engine,receiver
 
 CLSS public final org.graalvm.polyglot.Context$Builder
  outer org.graalvm.polyglot.Context
@@ -157,6 +157,7 @@ meth public org.graalvm.polyglot.Context$Builder allowHostClassLookup(java.util.
 meth public org.graalvm.polyglot.Context$Builder allowIO(boolean)
 meth public org.graalvm.polyglot.Context$Builder allowNativeAccess(boolean)
 meth public org.graalvm.polyglot.Context$Builder allowPolyglotAccess(org.graalvm.polyglot.PolyglotAccess)
+meth public org.graalvm.polyglot.Context$Builder allowValueSharing(boolean)
 meth public org.graalvm.polyglot.Context$Builder arguments(java.lang.String,java.lang.String[])
 meth public org.graalvm.polyglot.Context$Builder currentWorkingDirectory(java.nio.file.Path)
 meth public org.graalvm.polyglot.Context$Builder engine(org.graalvm.polyglot.Engine)
@@ -178,7 +179,7 @@ meth public org.graalvm.polyglot.Context$Builder resourceLimits(org.graalvm.poly
 meth public org.graalvm.polyglot.Context$Builder serverTransport(org.graalvm.polyglot.io.MessageTransport)
 meth public org.graalvm.polyglot.Context$Builder timeZone(java.time.ZoneId)
 supr java.lang.Object
-hfds allowAllAccess,allowCreateProcess,allowCreateThread,allowExperimentalOptions,allowHostAccess,allowHostClassLoading,allowIO,allowNativeAccess,arguments,currentWorkingDirectory,customFileSystem,customLogHandler,environment,environmentAccess,err,hostAccess,hostClassFilter,hostClassLoader,in,messageTransport,onlyLanguages,options,out,polyglotAccess,processHandler,resourceLimits,sharedEngine,zone
+hfds allowAllAccess,allowCreateProcess,allowCreateThread,allowExperimentalOptions,allowHostAccess,allowHostClassLoading,allowIO,allowNativeAccess,allowValueSharing,arguments,currentWorkingDirectory,customFileSystem,customLogHandler,environment,environmentAccess,err,hostAccess,hostClassFilter,hostClassLoader,in,messageTransport,onlyLanguages,options,out,polyglotAccess,processHandler,resourceLimits,sharedEngine,zone
 
 CLSS public final org.graalvm.polyglot.Engine
 innr public final Builder
@@ -195,7 +196,7 @@ meth public static org.graalvm.polyglot.Engine$Builder newBuilder()
 meth public void close()
 meth public void close(boolean)
 supr java.lang.Object
-hfds EMPTY,JDK8_OR_EARLIER,impl
+hfds EMPTY,JDK8_OR_EARLIER,currentAPI,dispatch,receiver
 hcls APIAccessImpl,ImplHolder,PolyglotInvalid
 
 CLSS public final org.graalvm.polyglot.Engine$Builder
@@ -288,7 +289,7 @@ meth public java.lang.String getName()
 meth public java.lang.String getVersion()
 meth public org.graalvm.options.OptionDescriptors getOptions()
 supr java.lang.Object
-hfds impl
+hfds dispatch,receiver
 
 CLSS public final org.graalvm.polyglot.Language
 meth public boolean isInteractive()
@@ -300,7 +301,7 @@ meth public java.lang.String getVersion()
 meth public java.util.Set<java.lang.String> getMimeTypes()
 meth public org.graalvm.options.OptionDescriptors getOptions()
 supr java.lang.Object
-hfds impl
+hfds dispatch,receiver
 
 CLSS public final org.graalvm.polyglot.PolyglotAccess
 fld public final static org.graalvm.polyglot.PolyglotAccess ALL
@@ -348,7 +349,7 @@ meth public void printStackTrace(java.io.PrintStream)
 meth public void printStackTrace(java.io.PrintWriter)
 meth public void setStackTrace(java.lang.StackTraceElement[])
 supr java.lang.RuntimeException
-hfds impl
+hfds dispatch,impl
 
 CLSS public final org.graalvm.polyglot.PolyglotException$StackFrame
  outer org.graalvm.polyglot.PolyglotException
@@ -366,13 +367,13 @@ CLSS public final org.graalvm.polyglot.ResourceLimitEvent
 meth public java.lang.String toString()
 meth public org.graalvm.polyglot.Context getContext()
 supr java.lang.Object
-hfds impl
+hfds context
 
 CLSS public final org.graalvm.polyglot.ResourceLimits
 innr public final Builder
 meth public static org.graalvm.polyglot.ResourceLimits$Builder newBuilder()
 supr java.lang.Object
-hfds EMPTY,impl
+hfds EMPTY,receiver
 
 CLSS public final org.graalvm.polyglot.ResourceLimits$Builder
  outer org.graalvm.polyglot.ResourceLimits
@@ -421,7 +422,7 @@ meth public static org.graalvm.polyglot.Source$Builder newBuilder(java.lang.Stri
 meth public static org.graalvm.polyglot.Source$Builder newBuilder(java.lang.String,java.net.URL)
 meth public static org.graalvm.polyglot.Source$Builder newBuilder(java.lang.String,org.graalvm.polyglot.io.ByteSequence,java.lang.String)
 supr java.lang.Object
-hfds EMPTY,IMPL,impl
+hfds DISPATCH,EMPTY,receiver
 
 CLSS public org.graalvm.polyglot.Source$Builder
  outer org.graalvm.polyglot.Source
@@ -460,7 +461,7 @@ meth public java.lang.CharSequence getCode()
 meth public java.lang.String toString()
 meth public org.graalvm.polyglot.Source getSource()
 supr java.lang.Object
-hfds IMPL,impl,source
+hfds DISPATCH,receiver,source
 
 CLSS public abstract org.graalvm.polyglot.TypeLiteral<%0 extends java.lang.Object>
 cons protected init()
@@ -571,7 +572,6 @@ meth public void writeBufferInt(java.nio.ByteOrder,long,int)
 meth public void writeBufferLong(java.nio.ByteOrder,long,long)
 meth public void writeBufferShort(java.nio.ByteOrder,long,short)
 supr java.lang.Object
-hfds impl,receiver
 
 CLSS public abstract interface org.graalvm.polyglot.io.ByteSequence
 meth public abstract byte byteAt(int)

sdk/src/org.graalvm.polyglot/src/org/graalvm/polyglot/Context.java

@@ -264,6 +264,10 @@
  * like ASTs or optimized code by specifying a single underlying engine. See {@link Engine} for more
  * details about code sharing.
  *
+ * <p>
+ * Context can be configured to allow value sharing between multiple contexts (allowed by default).
+ * See {@link Builder#allowValueSharing(boolean)} for details.
+ *
  * <h3>Proxies</h3>
  *
  * The {@link Proxy proxy interfaces} allow to mimic guest language objects, arrays, executables,
@@ -1000,6 +1004,7 @@ public final class Builder {
         private Boolean allowHostClassLoading;
         private Boolean allowExperimentalOptions;
         private Boolean allowHostAccess;
+        private boolean allowValueSharing = true;
         private PolyglotAccess polyglotAccess;
         private HostAccess hostAccess;
         private FileSystem customFileSystem;
@@ -1297,6 +1302,44 @@ public Builder allowPolyglotAccess(PolyglotAccess accessPolicy) {
             return this;
         }
 
+        /**
+         * Enables or disables sharing of any {@link Value value} between contexts. Value sharing is
+         * enabled by default and is not affected by {@link #allowAllAccess(boolean)}.
+         * <p>
+         * If this option is set to <code>true</code> (default) then any value that is associated
+         * with one context will be automatically migrated when passed to another context.
+         * Primitive, {@link Value#isHostObject() host} and {@link Value#isHostObject() proxy}
+         * values can be migrated without limitation. When guest language values are migrated, they
+         * capture and remember their original context. Guest language objects need to be accessed
+         * when their respective context is {@link Context#enter() entered}, therefore before any
+         * access their original context is entered and subsequently left. Entering the original
+         * context may fail, for example when the context of a original context value only allows
+         * single threaded access to values or if it was {@link Context#close() closed} in the mean
+         * time.
+         * <p>
+         * If this option is set to <code>false</code> then any value passed from one context to
+         * another will fail with an error indicating that sharing is disallowed. Turning sharing
+         * off can be useful when strict safety is required and it would be considered an error if a
+         * value of one context is passed to another.
+         * <p>
+         * Values of a guest language that are passed from one context to another are restricted to
+         * using the interoperability protocol only. In practice this often leads to slight changes
+         * and incompatibilities in behavior. For example, the prototype of a JavaScript object
+         * passed from one context to another is not writable, as the interoperability protocol does
+         * not allow such an operation (yet). A typical use-case is passing big immutable data
+         * structures that are infrequently accessed from one context to another, without copying
+         * them. In practice, passing values from one context to another should be avoided if
+         * possible, as their access is slower and their language compatibility reduced. This
+         * feature was introduced in 21.3. Older versions fail when guest values are passed from one
+         * context to another.
+         *
+         * @since 21.3
+         */
+        public Builder allowValueSharing(boolean enabled) {
+            this.allowValueSharing = enabled;
+            return this;
+        }
+
         /**
          * Sets a class filter that allows to limit the classes that are allowed to be loaded by
          * guest languages. If the filter returns <code>true</code>, then the class is accessible,
@@ -1701,8 +1744,19 @@ public Context build() {
             String localCurrentWorkingDirectory = currentWorkingDirectory == null ? null : currentWorkingDirectory.toString();
             Engine engine = this.sharedEngine;
             Context ctx;
+            OutputStream contextOut;
+            OutputStream contextErr;
+            InputStream contextIn;
+            Map<String, String> contextOptions;
             if (engine == null) {
                 org.graalvm.polyglot.Engine.Builder engineBuilder = Engine.newBuilder().options(options == null ? Collections.emptyMap() : options);
+                // for bound engines we just pass all the options to the engine so they can be
+                // processed in one step.
+                contextOptions = Collections.emptyMap();
+                contextOut = null;
+                contextErr = null;
+                contextIn = null;
+
                 if (out != null) {
                     engineBuilder.out(out);
                 }
@@ -1723,21 +1777,20 @@ public Context build() {
                 engineBuilder.allowExperimentalOptions(experimentalOptions);
                 engineBuilder.setBoundEngine(true);
                 engine = engineBuilder.build();
-                ctx = engine.dispatch.createContext(engine.receiver, null, null, null, hostClassLookupEnabled, hostAccess, polyglotAccess, nativeAccess, createThread,
-                                io, hostClassLoading, experimentalOptions,
-                                localHostLookupFilter, Collections.emptyMap(), arguments == null ? Collections.emptyMap() : arguments,
-                                onlyLanguages, customFileSystem, customLogHandler, createProcess, processHandler, environmentAccess, environment, zone, limits,
-                                localCurrentWorkingDirectory, hostClassLoader);
             } else {
                 if (messageTransport != null) {
                     throw new IllegalStateException("Cannot use MessageTransport in a context that shares an Engine.");
                 }
-                ctx = engine.dispatch.createContext(engine.receiver, out, err, in, hostClassLookupEnabled, hostAccess, polyglotAccess, nativeAccess, createThread,
-                                io, hostClassLoading, experimentalOptions,
-                                localHostLookupFilter, options == null ? Collections.emptyMap() : options, arguments == null ? Collections.emptyMap() : arguments,
-                                onlyLanguages, customFileSystem, customLogHandler, createProcess, processHandler, environmentAccess, environment, zone, limits,
-                                localCurrentWorkingDirectory, hostClassLoader);
+                contextOptions = options == null ? Collections.emptyMap() : options;
+                contextOut = out;
+                contextErr = err;
+                contextIn = in;
             }
+            ctx = engine.dispatch.createContext(engine.receiver, contextOut, contextErr, contextIn, hostClassLookupEnabled, hostAccess, polyglotAccess, nativeAccess, createThread,
+                            io, hostClassLoading, experimentalOptions,
+                            localHostLookupFilter, contextOptions, arguments == null ? Collections.emptyMap() : arguments,
+                            onlyLanguages, customFileSystem, customLogHandler, createProcess, processHandler, environmentAccess, environment, zone, limits,
+                            localCurrentWorkingDirectory, hostClassLoader, allowValueSharing);
             return ctx;
         }
 

sdk/src/org.graalvm.polyglot/src/org/graalvm/polyglot/impl/AbstractPolyglotImpl.java

@@ -501,7 +501,8 @@ public abstract Context createContext(Object receiver, OutputStream out, OutputS
                         boolean allowNativeAccess, boolean allowCreateThread, boolean allowHostIO, boolean allowHostClassLoading, boolean allowExperimentalOptions, Predicate<String> classFilter,
                         Map<String, String> options,
                         Map<String, String[]> arguments, String[] onlyLanguages, FileSystem fileSystem, Object logHandlerOrStream, boolean allowCreateProcess, ProcessHandler processHandler,
-                        EnvironmentAccess environmentAccess, Map<String, String> environment, ZoneId zone, Object limitsImpl, String currentWorkingDirectory, ClassLoader hostClassLoader);
+                        EnvironmentAccess environmentAccess, Map<String, String> environment, ZoneId zone, Object limitsImpl, String currentWorkingDirectory, ClassLoader hostClassLoader,
+                        boolean allowValueSharing);
 
         public abstract String getImplementationName(Object receiver);
 
@@ -622,7 +623,7 @@ protected AbstractHostAccess(AbstractPolyglotImpl impl) {
             Objects.requireNonNull(impl);
         }
 
-        public abstract Object toGuestValue(Object internalContext, Object parentNode, Object hostValue);
+        public abstract Object toGuestValue(Object internalContext, Object hostValue);
 
         public abstract <T> List<T> toList(Object internalContext, Object guestValue, boolean implementFunction, Class<T> elementClass, Type elementType);
 
@@ -710,14 +711,12 @@ public abstract void initializeHostContext(Object internalContext, Object contex
 
         public abstract boolean isHostProxy(Object value);
 
-        public abstract Object migrateHostObject(Object newContext, Object value);
-
-        public abstract Object migrateHostProxy(Object newContext, Object value);
-
         public abstract Error toHostResourceError(Throwable hostException);
 
         public abstract int findNextGuestToHostStackTraceElement(StackTraceElement firstElement, StackTraceElement[] hostStack, int nextElementIndex);
 
+        public abstract Object migrateValue(Object hostContext, Object value, Object valueContext);
+
     }
 
     public abstract static class AbstractValueDispatch {