Skip to content

Latest commit

 

History

History
103 lines (72 loc) · 3.25 KB

README.md

File metadata and controls

103 lines (72 loc) · 3.25 KB
Raw

An implementation of Google Protobuf for Smalltalk/Pharo

This repository contains support for Google Protobuf and bindings to GRPC.

Google Protobuf

The code includes an encoder/decoder for binary Google Protobuf and a code generator that takes Googles protoc definition and can generate Pharo code.

Use protoc to generate a binary definition

protoc -o tf.pb --include_imports tensorflow/core/framework/graph.proto

Generate Smalltalk code from the description

| descriptor nameTable generator |
"1. Decode the 'tf.pb' file into a descriptor"
descriptor := GPBFileDescriptorSet
	  materializeFrom: 'tf.pb' asFileReference  binaryReadStream.

"2. Set-up a type/name table and apply a custom prefix to all types"
nameTable := GPBTypeNamesVisitor new.
nameTable customPrefix: 'TF_'.

"3. Set-up the class/type generator and specify the target package for the code"
generator := GPBGeneratingVisitor new
  typeNames: nameTable;
  targetPackage: 'Tensorflow-Definitions'.
  
"4. Generate types and code"
descriptor visit: nameTable.
generator visit: descriptor.

Use the generated code

TF_GraphDef materializeFrom: 'inception_v3_2016_08_28_frozen.pb' asFileReference binaryReadStream

gRPC code

The repository includes work-in-progress bindings to the gRPC C library. This requires at least Pharo8.0 to include FFI fixes.

State of simple things

| channel queue stub |
 "Create a new gRPC channel to a destination."
 channel := GrpcChannel newInsecure: 'localhost:50051'.
 channel check_connectivity: 1.

 "Create a gRPC client stub for the global completionQueue."
 queue := GrpcLibrary uniqueInstance completionQueue.
 stub := GrpcClientStub initWith: channel queue: queue.

 "Make an unary call to 'SayHello' and encode a protobuf on the fly
 and wait up to one second for the completion. E.g. start the C++
 server on the other side"
 stub genericUnaryCall: '/helloworld.Greeter/SayHello' asByteArray param: (GRPC_EX_HLWHelloRequest new name: 'Smalltalk'; materialize) resultClass: GRPC_EX_HLWHelloReply timeout: 1 seconds

What

This was tested on macOs against v1.19.x of gRPC (4566c2a29ebec0835643b972eb99f4306c4234a3) installed to /usr/local/lib/libgrpc.dylib.

Known issues

  • ProtoBuf slots are not properly initialized on load. Fix this by running PBTypeName allSubclassesDo: #resolveEncodingType after everything is loaded.

  • GrpcLibrary startUp: true must be called once on start up. This is fixed by having a baseline.

  • GrpcLibrary uniqueInstance completionQueue polls for completion (limiting throughput and adds fixed latency cost).

  • Test failure on Linux with v1.27.x of gRPC in the completion queue.

Todos

  • Create BaselineOfGrpc and load packages and run resolveEncodingType on load.
  • Enable CI tests on travis-ci/github actions.
  • Only GrpcClientStub>>#genericUnaryCall:... is implemented.
  • Basic server support is missing.
  • Streaming client/server is missing.
  • Add completion by calling a block (in addition to the waiting on a semaphore)
  • Introspection for the server (which interfaces supported)
  • Windows and Linux support
  • Remove polling for a plugin..
  • Image suspend/resume with outstanding gRPC calls.
  • Generate server and client stubs based on the service definition.