Yadriggy Py

An embedded DSL similar to the Python language. It is a front-end of PyCall Ruby; this DSL code is translated into Python code and sent to the Python interpreter for execution.

This DSL was designed for using Python libraries from Ruby as easy as possible, ideally, by just copying a sample program written in Python. So the DSL code might look like Python code. This is a deliberate design. The DSL code is not normal Ruby code; it is syntactically valid but it cannot be interpreted as normal Ruby code. It is executed with different semantics from Ruby's.

Overview

This DSL allows programmers to write a Python-like code in Ruby and execute it by the Python interpreter. The Python interpreter is invoked by command python. To change this command, set the environment variable PYTHON to an appropriate command name.

The code written in this DSL must be a block given to Yadriggy::Py::run. For example,

require 'yadriggy/py'
r = Yadriggy::Py::run { -(3 + 4) }    # r == -7

The DSL code is -(3 + 4). It is passed to the Python interpreter. The code shown above is equivalent to:

require 'pycall'
PyCall::eval("-(3 + 4)")

A significant difference is PyCall::eval receives a string object but Yadriggy::Py::run receives a block. So the DSL code for Yadriggy::Py is properly syntax-highlighted.

The block given to Yadriggy::Py::run has to be a syntactically valid Ruby block but it does not have to be semantically correct as Ruby code. So Python-like list comprehension is available in the DSL code:

a = Yadriggy::Py::run { [for i in range(0,3) do i end] })

This results in an array [0, 1, 2]. In Ruby, the code block would cause an error because range would not be defined. Even when range is defined, the code block would not work as list comprehension. However, that code block is translated by Yadriggy::Py into proper Python code like:

[i for i in range(0,3)]

and executed by the Python interpreter. The resulting list is sent back to Ruby by PyCall. The code block above borrows its syntax from Ruby but has its own semantics different from Ruby.

Free variables

The code block given to Yadriggy::Py::run may contain a variable name that denotes a variable declared out of the block. Such a variable is often called a free variable.

Yadriggy::Py supports a free variable if its value is a number, a string, or an array of these values. For example,

require 'yadriggy/py'

a = [1, 2, 3]
r = Yadriggy::Py::run { a[0] + a[1] }    # r == 3

The name a in the code block denotes to the variable a out of the block. It holds an array object [1, 2, 3]. When the code block is sent to executed by the Python interpreter, the value of a is also copied to Python. So the code block can access the array elements in Python but updates on the array is not reflected on the original array in Ruby. For example,

a = [1, 2, 3]
Yadriggy::Py::run { a[0] = 10 }
puts a[0]    # not 10 but 1

Since the assignment a[0] = 10 is executed in Python, the last line prints 1, which is the value of a[0] in Ruby. The array updated in the code block { a[0] = 10 } is a copy of the original array.

Functions

The code block given to Yadriggy::Py::run may contain a call to a Ruby function. When the code block containing such a call is sent to Python, the callee Ruby function is also sent to Python. The body of the callee function is interpreted by this DSL.

def sum_numbers(n)
  return sum([for i in range(0,n) do i end])
end

s = Yadriggy::Py::run { sum_numbers(10) }
puts s    # 45

Since the sum_numbers function is called from the code block passed to run, it is also sent to Python. The body is written in the DSL. No definitions of sum and range are necessary since they are built-in functions in Python. When the function name is not found in the current scope in Ruby, it is recognized as the name of a Python function.

Import

To import a Python module, call pyimport:

require 'yadriggy/py'
include Yadriggy::Py::PyImport

pyimport('math')        # import math
puts Yadriggy::Py::run { math.cos(0) }

pyfrom(:random).import(:randint).as(:rand)
# from random import randint as rand
puts Yadriggy::Py::run { rand(1, 6) }

pyimport, pyfrom, import, and as take either a string object or a symobol. A call to import method can follow any calls to pyimport, pyfrom, import, or as. A call to as method can follow only a call to import. Hence the following code is valid:

pyimport('math').import('random')
# import math, random

pyfrom('math').import('cos').import('sin')
# from math import cos, sin

PyImport defines utility functions pyimport and pyfrom, which corresponds to import and from in Python, respectively. Hence the code snippet above is equivalent to the following code:

Yadriggy::Py::Import.import('math')
Yadriggy::Py::Import.from(:random).import(:randint).as(:rand)

Differences from Python

Although Ruby has flexible syntax, there are several differences between this DSL and Python.

Function definition

A function is defined as in Ruby.

def add(a, b)
  return a + b
end

Only the required arguments are allowed. The default arguments, keyword arguments, or variable number of arguments are not supported for function definitions.

Keyword arguments are available for function calls. For example,

puts Yadriggy::Py::run { add(b=1, a=3) }    # 4

Return statement

Although the syntax of function definitions is the same as Ruby's, the function body has to include a return statement when it returns a value. Unlike Ruby, the value of the function call is not the value of the expression evaluated last in the function body. If the function body does not include a return statement, the function does not return any value.

Lambda

A lambda expression is written in the same syntax as Ruby's.

Yadriggy::Py::run do
  f = -> (x) { x + 1}
  g = lambda {|x| x + 20 }
  f(3) + g(10)
end

Tuple

To construct a tuple, call tuple function.

Yadriggy::Py::run do
  t1 = tuple()        # ()
  t2 = tuple(1,)      # (1,)
  t3 = tuple(1, 2)    # (1, 2)
  print(t1, t2, t3)
end

Slicing

Slicing is partly supported. Use .. instead of :.

Yadriggy::Py::run do
  a = [1, 2, 3, 4]
  print(a[1..3])    # a[1:3]
  print(a[1.._])    # a[1:]
  print(a[_..2])    # a[:2]
end

List comprehension

A list comprehension is expressed by a for statement enclosed by [ ].

Yadriggy::Py::run { [for i in range(0, 3) do i end] })

This is equivalent to:

[ i for i in range(0, 3)]

The expression before for is expressed by the do ... end part.

Dictionary

A dictionary is expressed by a hash literal.

Yadriggy::Py::run do
  h = { 'one' => 1, 'two' => 2 }
  g = { 'one': 1, 'two': 2 }
  h['one'] + g['two']
end

Note that a dictionary key on the left of : is not a label but a value in Python. So,

Yadriggy::Py::run do
  key1 = 'one'
  key2 = 'two'
  h = { key1: 1, key2: 2 }
  h['one']
end

The dictionary h has two entries and the first key is 'one' and the second key is 'two'. They are not 'key1' or 'key2'.

with statement

The with statement is written as follows:

Yadriggy::Py::run do
  with open('foo.txt') => f do
    print(f.read())
  end
end

This code is equivalent to the following Python code:

with open('foo.txt') as f:
  print(f.read())

The with keyword can be followed by multiple expressions. For example, the following Ruby code is valid:

Yadriggy::Py::run do
  with open('foo.txt') => f, open('bar.txt') => b do
    print(f.read())
    print(b.read())
  end
end

Others

Some kinds of code have to be written in a different style from Python.

Yadriggy::Py	Python	Example
true	True	b = true
false	False	b = false
True	True	b = True
False	False	b = False
nil	None	x = nil
None	None	x = None
!	not	!true
&&	and	x > 3 && y > 2
||	or	x > 3 || y > 2
.in	in	x .in [1, 2, 3]
		x.in([1, 2, 3])
.not_in	not in	x .not_in [1, 2, 3]
.idiv	//	x .idiv 3
i..j	range(i,j)	for i in 0..n do sum += i end
range(i,j)	range(i,j)	for i in range(0, n) do sum += i end
? :	if else	x > 0 ? x + 1 : -x + 1

Tips

You might think Yadriggy::Py::run is too long. Then you might want to define your utility function:

def py_run(&block)
  Yadriggy::Py::run(block)
end

and call it like:

r = py_run { -(3 + 4) }    # r == -7

This works since Yadriggy::Py::run is a normal Ruby method and takes a normal Ruby block.

To change the Python interpreter, set the environment variable PYTHON. For example,

export PYTHON=/usr/local/bin/python3

This tells PyCall Ruby to use python3.

Comparison

This DSL works as a front end of PyCall Ruby. Although PyCall enables a call to Python function, this DSL sends a code block to Python for execution. It causes fewer data exchanges between Ruby and Python, and enables a wider range of computation in Python, such as a function definition and list comprehension.

Although the idea of this DSL is similar to the technique known as the string embedding, the DSL code is embedded in Ruby as not a string literal but a syntactically-correct code block in Ruby (note that the code block may be semantically incorrect). Thus, the code written in this DSL will be syntactically highlighted and a syntax error will be reported when the code is loaded by the Ruby interpreter (not when the DSL code is sent to the Python interpreter during runtime).