Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pod: NPE in upload module #141

Closed
hamann opened this issue Mar 10, 2022 · 5 comments
Closed

pod: NPE in upload module #141

hamann opened this issue Mar 10, 2022 · 5 comments
Labels
bug Something isn't working

Comments

@hamann
Copy link
Contributor

hamann commented Mar 10, 2022 

(require '[babashka.pods :as pods])
(pods/load-pod 'epiccastle/spire "0.1.0-alpha.17")

(ns playground
  (:require [pod.epiccastle.spire.module.sudo :as spire.sudo]
            [pod.epiccastle.spire.module.upload :as spire.upload]
            [pod.epiccastle.spire.transport :as spire.transport]))

(spire.transport/ssh
 {:username "vagrant" :hostname "localhost"
  :port 54321 :strict-host-key-checking false
  :accept-host-key true :agent-forwarding true}

 (spire.sudo/sudo-user
  {:username "root"}
  (spire.upload/upload {:content "foo" :dest "/tmp/foo"})))

gives

#error {
 :cause nil
 :via
 [{:type java.lang.NullPointerException
   :message nil
   :at [spire.nio$relativise invokeStatic nio.clj 21]}]
 :trace
 [[spire.nio$relativise invokeStatic nio.clj 21]
  [spire.nio$relativise invoke nio.clj 16]
  [clojure.lang.AFn applyToHelper AFn.java 156]
  [clojure.lang.AFn applyTo AFn.java 144]
  [clojure.core$apply invokeStatic core.clj 667]
  [spire.pod.core$main$fn__27553 invoke core.clj 1215]
  [spire.pod.core$main invokeStatic core.clj 1208]
  [spire.core$_main invokeStatic core.clj 67]
  [spire.core$_main doInvoke core.clj 59]
  [clojure.lang.RestFn invoke RestFn.java 397]
  [clojure.lang.AFn applyToHelper AFn.java 152]
  [clojure.lang.RestFn applyTo RestFn.java 132]
  [spire.core main nil -1]]}
#error {
 :cause "clojure.lang.PersistentVector cannot be cast to byte[]"
 :via
 [{:type java.lang.ClassCastException
   :message "clojure.lang.PersistentVector cannot be cast to byte[]"
   :at [babashka.pods.impl$bytes__GT_string invokeStatic "impl.clj" 28]}]
 :trace
 [[babashka.pods.impl$bytes__GT_string invokeStatic "impl.clj" 28]
  [babashka.pods.impl$processor invokeStatic "impl.clj" 186]
  [babashka.pods.sci$load_pod$fn__30764 invoke "sci.clj" 72]
  [sci.impl.vars$binding_conveyor_fn$fn__398 invoke "vars.cljc" 154]
  [clojure.core$binding_conveyor_fn$fn__5788 invoke "core.clj" 2035]
  [clojure.lang.AFn call "AFn.java" 18]
  [java.util.concurrent.FutureTask run "FutureTask.java" 264]
  [java.util.concurrent.ThreadPoolExecutor runWorker "ThreadPoolExecutor.java" 1128]
  [java.util.concurrent.ThreadPoolExecutor$Worker run "ThreadPoolExecutor.java" 628]
  [java.lang.Thread run "Thread.java" 829]
  [com.oracle.svm.core.thread.JavaThreads threadStartRoutine "JavaThreads.java" 596]
  [com.oracle.svm.core.posix.thread.PosixJavaThreads pthreadStartRoutine "PosixJavaThreads.java" 192]]}

No issues with running equivalent code with spire binary
@retrogradeorbit retrogradeorbit added the bug Something isn't working label Mar 26, 2022
@retrogradeorbit
Copy link
Member

@hamann having trouble replicating this. Works on my machine to real ssh box. tested with vagrant, also worked.

Going to need more info. What host platform are you using?

It looks like one of the calls when being invoked in the pod is producing unserialisable return data. And for some reason it's the path relativize nio func? Very strange. Is there something interesting about the vagrant box? Does it actually have a /tmp folder? Does vagrant user have a home directory?

Is it possible to share a minimal vagrant file that can reproduce this bug?

@hamann
Copy link
Contributor Author

hamann commented Mar 29, 2022 •
edited
Loading

This is on macOS 11.6.4 with babashka 0.7.8

Vagrantfile

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/impish64"

  config.vm.provider "virtualbox" do |v|
    v.memory = 4096
    v.cpus = 2
  end

  config.vm.provider "vmware_desktop" do |v|
    v.vmx["memsize"] = "4096"
    v.vmx["numvcpus"] = "2"
  end

  config.vm.network "forwarded_port", id: "ssh", host: 54321, guest: 22
end

But in the meantime the stacktrace changed (maybe due to a bb update and/or using alpha.18?), now it's

spire-as-pod.clj:16 (spire.upload/upload {:content "foo", :dest "/tmp/foo"})
----- Error --------------------------------------------------------------------
Type:     java.lang.IllegalArgumentException
Message:  No matching clause: ["java.io.PipedOutputStream" #{}]

----- Exception ----------------------------------------------------------------
java.lang.IllegalArgumentException: No matching clause: ["java.io.PipedOutputStream" #{}]
 at babashka.impl.proxy$proxy_fn.invokeStatic (proxy.clj:16)
    babashka.impl.proxy$proxy_fn.invoke (proxy.clj:14)
    sci.impl.proxy$proxy_STAR_.invokeStatic (proxy.clj:25)
    sci.impl.proxy$proxy_STAR_.invoke (proxy.clj:21)
    sci.impl.vars.SciVar.invoke (vars.cljc:333)
    sci.impl.analyzer$return_needs_ctx_call$reify__9497.eval (analyzer.cljc:1100)
    sci.impl.fns$fun$arity_1__7879.invoke (fns.cljc:106)
    sci.impl.vars.SciVar.invoke (vars.cljc:325)
    sci.impl.analyzer$return_call$reify__9555.eval (analyzer.cljc:1149)
    sci.impl.analyzer$return_if$reify__9276.eval (analyzer.cljc:653)
    sci.impl.evaluator$eval_let$fn__7066.invoke (evaluator.cljc:65)
    sci.impl.evaluator$eval_let.invokeStatic (evaluator.cljc:56)
    sci.impl.analyzer$analyze_let_STAR_$reify__9227.eval (analyzer.cljc:492)
    sci.impl.evaluator$eval_let.invokeStatic (evaluator.cljc:74)
    sci.impl.analyzer$analyze_let_STAR_$reify__9227.eval (analyzer.cljc:492)
    sci.impl.fns$fun$arity_3__7394.doInvoke (fns.cljc:86)
    clojure.lang.RestFn.invoke (RestFn.java:2482)
    sci.impl.vars.SciVar.invoke (vars.cljc:357)
    sci.impl.analyzer$return_call$reify__9619.eval (analyzer.cljc:1149)
    sci.impl.analyzer$return_if$reify__9274.eval (analyzer.cljc:643)
    sci.impl.evaluator$eval_let$fn__7066.invoke (evaluator.cljc:65)
    sci.impl.evaluator$eval_let.invokeStatic (evaluator.cljc:56)
    sci.impl.analyzer$analyze_let_STAR_$reify__9227.eval (analyzer.cljc:492)
    sci.impl.evaluator$eval_let.invokeStatic (evaluator.cljc:74)
    sci.impl.analyzer$analyze_let_STAR_$reify__9227.eval (analyzer.cljc:492)
    sci.impl.analyzer$return_if$reify__9276.eval (analyzer.cljc:652)
    sci.impl.analyzer$return_if$reify__9276.eval (analyzer.cljc:653)
    sci.impl.evaluator$eval_let$fn__7066.invoke (evaluator.cljc:65)
    sci.impl.evaluator$eval_let.invokeStatic (evaluator.cljc:56)
    sci.impl.analyzer$analyze_let_STAR_$reify__9227.eval (analyzer.cljc:492)
    sci.impl.analyzer$return_if$reify__9276.eval (analyzer.cljc:652)
    sci.impl.analyzer$return_if$reify__9276.eval (analyzer.cljc:653)
    sci.impl.analyzer$return_if$reify__9276.eval (analyzer.cljc:653)
    sci.impl.analyzer$return_if$reify__9276.eval (analyzer.cljc:653)
    sci.impl.evaluator$eval_let.invokeStatic (evaluator.cljc:74)
    sci.impl.analyzer$analyze_let_STAR_$reify__9227.eval (analyzer.cljc:492)
    sci.impl.analyzer$return_or$reify__8613.eval (analyzer.cljc:163)
    sci.impl.evaluator$eval_let$fn__7066.invoke (evaluator.cljc:65)
    sci.impl.evaluator$eval_let.invokeStatic (evaluator.cljc:56)
    sci.impl.analyzer$analyze_let_STAR_$reify__9227.eval (analyzer.cljc:492)
    sci.impl.fns$fun$arity_0__7370.doInvoke (fns.cljc:83)
    clojure.lang.RestFn.invoke (RestFn.java:457)
    sci.impl.vars.SciVar.invoke (vars.cljc:331)
    sci.impl.analyzer$return_call$reify__9567.eval (analyzer.cljc:1149)
    sci.impl.evaluator$eval_let$fn__7066.invoke (evaluator.cljc:65)
    sci.impl.evaluator$eval_let.invokeStatic (evaluator.cljc:56)
    sci.impl.analyzer$analyze_let_STAR_$reify__9227.eval (analyzer.cljc:492)
    sci.impl.evaluator$eval_try.invokeStatic (evaluator.cljc:125)
    sci.impl.analyzer$analyze_try$reify__9310.eval (analyzer.cljc:741)
    sci.impl.analyzer$return_do$reify__8590.eval (analyzer.cljc:133)
    sci.impl.evaluator$eval_let$fn__7066.invoke (evaluator.cljc:65)
    sci.impl.evaluator$eval_let.invokeStatic (evaluator.cljc:56)
    sci.impl.analyzer$analyze_let_STAR_$reify__9227.eval (analyzer.cljc:492)
    sci.impl.evaluator$eval_try.invokeStatic (evaluator.cljc:125)
    sci.impl.analyzer$analyze_try$reify__9310.eval (analyzer.cljc:741)
    sci.impl.analyzer$return_do$reify__8590.eval (analyzer.cljc:133)
    sci.impl.evaluator$eval_let.invokeStatic (evaluator.cljc:74)
    sci.impl.analyzer$analyze_let_STAR_$reify__9227.eval (analyzer.cljc:492)
    sci.impl.analyzer$return_do$reify__8590.eval (analyzer.cljc:133)
    sci.impl.evaluator$eval_let.invokeStatic (evaluator.cljc:74)
    sci.impl.analyzer$analyze_let_STAR_$reify__9227.eval (analyzer.cljc:492)
    sci.impl.analyzer$return_do$reify__8592.eval (analyzer.cljc:133)
    sci.impl.evaluator$eval_let.invokeStatic (evaluator.cljc:74)
    sci.impl.analyzer$analyze_let_STAR_$reify__9227.eval (analyzer.cljc:492)
    sci.impl.analyzer$return_do$reify__8590.eval (analyzer.cljc:133)
    sci.impl.evaluator$eval_try.invokeStatic (evaluator.cljc:125)
    sci.impl.analyzer$analyze_try$reify__9310.eval (analyzer.cljc:741)
    sci.impl.analyzer$return_do$reify__8590.eval (analyzer.cljc:133)
    sci.impl.evaluator$eval_let.invokeStatic (evaluator.cljc:74)
    sci.impl.analyzer$analyze_let_STAR_$reify__9227.eval (analyzer.cljc:492)
    sci.impl.evaluator$eval_let.invokeStatic (evaluator.cljc:74)
    sci.impl.analyzer$analyze_let_STAR_$reify__9227.eval (analyzer.cljc:492)
    sci.impl.evaluator$eval_try.invokeStatic (evaluator.cljc:125)
    sci.impl.analyzer$analyze_try$reify__9310.eval (analyzer.cljc:741)
    sci.impl.evaluator$eval_let.invokeStatic (evaluator.cljc:74)
    sci.impl.analyzer$analyze_let_STAR_$reify__9227.eval (analyzer.cljc:492)
    sci.impl.interpreter$eval_form.invokeStatic (interpreter.cljc:39)
    sci.impl.interpreter$eval_string_STAR_.invokeStatic (interpreter.cljc:61)
    sci.core$eval_string_STAR_.invokeStatic (core.cljc:235)
    babashka.main$exec$fn__35764$fn__35765.invoke (main.clj:877)
    babashka.main$exec$fn__35764.invoke (main.clj:877)
    babashka.main$exec.invokeStatic (main.clj:867)
    babashka.main$main.invokeStatic (main.clj:954)
    babashka.main$main.doInvoke (main.clj:933)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    clojure.core$apply.invokeStatic (core.clj:667)
    babashka.main$_main.invokeStatic (main.clj:987)
    babashka.main$_main.doInvoke (main.clj:979)
    clojure.lang.RestFn.applyTo (RestFn.java:137)
    babashka.main.main (:-1)

@retrogradeorbit
Copy link
Member

Oh boy. This is totally my fault! That latest exception I remember from doing the pod conversion work! Let me explain.

When I was doing the spire pod conversion I modified babashka. Initially I used what I had modded it for quite a bit in the spire code, but eventually I rewrote a lot of that and didn't need it. So I assumed I didn't need it. But there was a lot of complexity going on. Over time I forgot all about the change I made to bb.

Gradually things changed. It looks like I did end up reintroducing spire code that depended on that bb change. So I couldn't reproduce the issue because I was using the modified version of bb! 🤦 As soon as I installed the official bb I got the exception. This will need a bb patch.

The issue is the proxy use of java.io.PipedInputStream and java.io.PipedOutputStream in code on the bb side. Babashka only supports a small subset of java classes from proxying.

@retrogradeorbit retrogradeorbit mentioned this issue Mar 31, 2022
Closed
@retrogradeorbit
Copy link
Member

PR underway here babashka/babashka#1225

@retrogradeorbit
Copy link
Member

fixed in 0.1.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@hamann @retrogradeorbit