Fn Expressions

Benchmark and state fns may be given as expression data. This is a convenience feature that is best used sparingly (if at all). For an example of why, we'll look at a sample jmh.edn file in a hypothetical project.

{:benchmarks [{:fn my.core/addition
               :args [:int :int]}]
 :states {:int {:fn (fn [] (rand-int 42))}}}

The above defines a simple addition benchmark (the benchmark code itself is not important for these examples). The state fn is specified inline as data. We'll run our benchmark using the lein-jmh plugin:

$ lein jmh
# => ({:score [6.3E-5 "s/op"], :fn my.core/addition #_...})

Now what if we wanted to change our :int state to use a custom random number generator:

{#_...
 :states {:int {:fn (fn [] (my.util/gen-random 42))}}}

When we run our benchmark we now get an exception:

$ lein jmh
# => clojure.lang.ExceptionInfo: error while evaluating fn expression form ...
#    ...
#    Caused by: java.lang.ClassNotFoundException: my.util ...

To support fn expressions, jmh-clojure must eval-uate the fn data code in an isolated JMH subprocess. The my.util namespace above has not yet been required at this point and we get an exception.

We could fix the error in this case by dynamically loading the namespace, like so:

{#_...
 :states {:int {:fn (fn []
                      ((ns-resolve (doto 'my.util require) 'gen-random) 42))}}}

This works, but at this point we are likely better off using a parameter:

{#_...
 :params {:max 42}
 :states {:int {:fn my.util/gen-random
                :args [:max]}}}

Using expressions can also make debugging more difficult. Going back to the first example, what if we wanted to use the same parameter with rand-int:

{#_...
 :params {:max 42}
 :states {:int {:fn (fn [] (rand-int 42))
                :args [:max]}}}

Above, we updated the :args but we forgot to update the fn argument vector and code. This leads to a vague exception:

$ lein jmh
# => clojure.lang.ArityException: Wrong number of args (1) passed to: core/eval12/fn--13

Also, you may have noticed that we are now wrapping with a redundant fn. We should be using the bare var as a symbol:

{#_...
 :states {:int {:fn rand-int, :args [:max]}}}

If we get argument counts wrong now, we will get an informative message. If we remove :args from above, for example:

$ lein jmh
# => java.lang.RuntimeException: clojure.core/rand-int does not support arity 0

In general, its best to just use var symbols, possibly via an auxiliary namespace. For example, now we want to specify a random range. We'll create a my.states namespace:

(ns my.states)

(defn random-range [min max]
  (+ min (rand-int (- max min))))

Our jmh.edn file now looks like this:

{#_...
 :params {:min 17, :max 42}
 :states {:int {:fn my.states/random-range
                :args [:min :max]}}}

Finally, sometimes fn expressions are sufficiently simple that its debatable whether using a var is really necessary. For example, generating a vector of a given size:

{:benchmarks [{:fn my.core/process-vector
               :args [:vec]}]
 :params {:size 100000}
 :states {:vec {:fn (comp vec range)
                :args [:size]}}}