Skip to content

snmsts/burgled-batteries3

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

burgled-batteries3: A Common Lisp / Python Bridge

burgled-batteries3 provides a shim between Python3 (specifically, the CPython implementation of Python) and Common Lisp.

Synopsis

(asdf:load-system "burgled-batteries3")
(in-package #:burgled-batteries3)
(startup-python)

(run "1+1") ; => 2

(import "feedparser")
(defpyfun "feedparser.parse" (thing))
(documentation 'feedparser.parse 'function)
; => "Parse a feed from a URL, file, stream, or string"
(feedparser.parse "http://pinterface.livejournal.com/data/atom")
; => #<HASH-TABLE>

(shutdown-python)

Why burgled-batteries3

CLPython is great when it works. However, if you’re using a low-resource computer—perhaps an underpowered VPS, or a Linux box salvaged from the 90s—, or need access to a Python library written in C, or there’s a bug and you can’t be bothered to narrow it down to a small test case, CLPython can’t help you. Two out of three of those are more your problem than CLPython’s, but hey, I’m not here to judge.

While a number of other Python-by-FFI options exist, burgled-batteries3 aims for a CLPython-esque level of integration. In other words, deep integration. You shouldn’t have to care that the library you’re using was written in Python—it should Just Work.

Certainly, b-b is not there yet. It may never be there completely. But we’ll try, dagnabbit.

Basic Type Mapping

Python objects are converted to a Lisp object where possible. Where a conversion is unknown, a pointer to the CPython object is returned (or, if inside a refcnt barrier, a wrapper around the pointer which will become invalid upon exiting the barrier). In general, this mapping follows the lead of CLPython.

Python TypeLisp Type
Boolean(member T NIL)
Integer, LongInteger
FloatDouble-float
DictHashtable
UnicodeString
ListAdjustable vector
TupleList
ComplexComplex
ByteArrayOctet vector
ExceptionCondition
<Unknown><pointer>

CPython FFI

Anything dealing with the CPython API can be found in the PYTHON.CFFI package. See the docstring for that package for more information, as well as Python’s C API.

Calling Python from Lisp

IMPORT
Equivalent to “import <foo>” in Python.
RUN
Mimicks the RUN function from CLPython.
DEFPYFUN
Similar to CFFI’s DEFCFUN—defines a Lisp function which calls a Python function. Ideally, this will eventually be obviated by something which uses Python introspection to figure out an apropriate argument list so you don’t have to, but it’ll get you going for now.

Calling Lisp from Python

Not yet supported, but see ffi-callbacks.lisp for some experimentation and notes along those lines.

Avoid the Hassle of Reference Counts

Because dealing with reference counts is Just No Fun and Not Lispy At All, as well as Inevitable—at some point there will be an object for which no translation is known—, b-b provides multiple options to avoid dealing with refcnts for those untranslatable pointers. See the macro CPYTHON:WITH-UNKNOWN-TRANSLATION-POLICY.

Note that this policy also also affects the EXCEPTION-* slots of PYTHON-CONDITION, and so they may or may not be available for inspection depending on the translation policy in effect and the manner of handling.

For example, under the default policy of :DISCARD, you would see something like:

(defun reveal-effect (c)
  (format t "~A~%" (slot-value c 'exception-type)))

(handler-bind ((python-condition #'reveal-effect))
  (burgled-batteries3:run "1/0"))
; prints #.(SB-SYS:INT-SAP #X?????)

(handler-case (burgled-batteries3:run "1/0")
  (python-condition (c) (reveal-effect c)))
; prints NIL

If you’d like access to Python types without a known translation, :BARRIER or :FINALIZE are highly recommended over :PASS-THROUGH. They do, however, come with some caveats which you should be aware of.

Requirements

Burgled-batteries3 links with C code, and accordingly requires the appropriate headers and library files. On Debian-based systems, you can get these via:

sudo apt-get install python3-dev

Testing

(asdf:test-system "burgled-batteries3")

Tests have been run under SBCL, Clozure CL, and CLISP.

To Do

  • Output redirection
  • Callbacks
  • Whole-module import (into own package?)
  • Python object <-> CLOS mappings
  • Provide facilities for user code to define own Lisp-Python mappings
  • Better integrate Quickcheck tests, so LIFT knows about quickcheck failures
  • Pathname support (requires FILE* support)

Prior Art / Other Solutions

Python-on-Lisp
It was a good try back in 2006, but unfortunately has experienced significant bitrot. It did manage to provide callbacks and writing to Lisp streams, however (both of which are still on b-b’s TODO list). Very handy features!
pythononlisp-ex
A fork of Python-on-Lisp which shimmies things between Lisp and Python using JSON. As you might expect, this falls down as soon as you hit something which can’t be represented in JSON, which covers quite a lot of types.
Pyffi
A more streamlined, and less featureful, Python FFI which came after PoL. Technically, burgled-batteries began life as some patches to Pyffi because it seemed to be the best place to start. (Almost nothing of Pyffi remains.)
CLPython
A very fine Python compiler written in Common Lisp. It requires a somewhat beefier machine than CPython. Unfortunately, it doesn’t work with Python libraries written in C (e.g., numpy), and so is unable to handle the full gamut of Python libraries one might wish to borrow.

About

A bridge between Python and Lisp (FFI bindings, etc.)

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Common Lisp 98.5%
  • Python 1.5%