Skip to content

Commit

Permalink
Add "registers" for pod data; add send! function (see #339)
Browse files Browse the repository at this point in the history
Test case:

    #!/usr/bin/env boot
    (require '[boot.pod :as pod])
    (def p (pod/make-pod))
    (def x (into [] (range 0 1000000)))
    (prn (pod/with-eval-in p (count ~x)))

This fails with "RuntimeException: Class file too large!"

    #!/usr/bin/env boot
    (require '[boot.pod :as pod])
    (def p (pod/make-pod))
    (def x (into [] (range 0 1000000)))
    (prn (pod/with-eval-in p (count ~(pod/send! x))))

This works, no problem.
  • Loading branch information
micha committed Nov 12, 2015
1 parent 5717632 commit 8f4eabd
Show file tree
Hide file tree
Showing 2 changed files with 25 additions and 3 deletions.
18 changes: 16 additions & 2 deletions boot/base/src/main/java/boot/App.java
Original file line number Diff line number Diff line change
Expand Up @@ -49,8 +49,22 @@ public class App {
public static String getBootVersion() { return bootversion; }
public static String getClojureName() { return cljname; }

private static final WeakHashMap<ClojureRuntimeShim, Object> pods = new WeakHashMap<>();
public static WeakHashMap<ClojureRuntimeShim, Object> getPods() { return pods; }
private static final ConcurrentHashMap<String, String> podRegisters = new ConcurrentHashMap<>();
private static final WeakHashMap<ClojureRuntimeShim, Object> pods = new WeakHashMap<>();

public static WeakHashMap<ClojureRuntimeShim, Object>
getPods() {
return pods; }

public static String
getRegister(String name) throws Exception {
String ret = podRegisters.get(name);
podRegisters.remove(name);
return ret; }

public static void
setRegister(String name, String value) throws Exception {
podRegisters.put(name, value); }

public static class
Exit extends Exception {
Expand Down
10 changes: 9 additions & 1 deletion boot/pod/src/boot/pod.clj
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
[dynapath.dynamic-classpath :as cp])
(:import
[java.util.jar JarFile]
[java.util Properties]
[java.util Properties UUID]
[java.net URL URLClassLoader URLConnection]
[java.util.concurrent ConcurrentLinkedQueue LinkedBlockingQueue TimeUnit]
[java.io File]
Expand Down Expand Up @@ -182,6 +182,14 @@
(.offer @shutdown-hooks f)
(->> f Thread. (.addShutdownHook (Runtime/getRuntime)))))

(defn send!
"This is ALPHA status, it may change, be renamed, or removed."
[form]
(let [uuid (str (UUID/randomUUID))
form (binding [*print-meta* true] (pr-str form))]
(boot.App/setRegister uuid form)
`(read-string (boot.App/getRegister ~uuid))))

(defn eval-fn-call
[[f & args]]
(when-let [ns (namespace f)] (require (symbol ns)))
Expand Down

2 comments on commit 8f4eabd

@martinklepsch
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey :)
Did the streaming approach not work for some reason?
I was wondering if using something like nippy might also be a solution?
👍

@micha
Copy link
Contributor Author

@micha micha commented on 8f4eabd Nov 12, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is that eval can't handle expressions over a certain size. It's not a problem of serializing the expression; it's a problem of the expression being too large when it gets to the other side and is deserialized and passed to eval. The solution here is to stash the form somewhere and pass a reference to it inside the expression that will be passed to eval. This way the expression itself is bounded in size.

Please sign in to comment.