Skip to content

Latest commit

 

History

History
247 lines (175 loc) · 7.25 KB

protobuf-format.md

File metadata and controls

247 lines (175 loc) · 7.25 KB
Raw

Protobuf Event Format for CloudEvents - Version 1.0.3-wip

Abstract

Protocol Buffers is a mechanism for marshalling structured data, this document defines how CloudEvents are represented using version 3 of that specification.

In this document the terms Protocol Buffers, protobuf, and proto are used interchangeably.

Table of Contents

  1. Introduction
  2. Attributes
  3. Data
  4. Transport
  5. Batch Format
  6. Examples

1. Introduction

CloudEvents is a standardized and protocol-agnostic definition of the structure and metadata description of events. This specification defines how the elements defined in the CloudEvents specification are are represented using a protobuf schema.

The Attributes section describes the naming conventions and data type mappings for CloudEvent attributes for use as protobuf message properties.

The Data section describes how the event payload is carried.

1.1. Conformance

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC2119.

1.2 Content-Type

There is no official IANA media-type designation for protobuf, as such this specification uses 'application/protobuf' to identify such content.

2. Attributes

This section defines how CloudEvents attributes are represented in the protobuf schema.

2.1 Type System

The CloudEvents type system is mapped to protobuf as follows :

CloudEvents protobuf
Boolean boolean
Integer int32
String string
Binary bytes
URI string following RFC 3986 §4.3
URI-reference string following RFC 3986 §4.1
Timestamp Timestamp

2.3 REQUIRED Attributes

REQUIRED attributes are represented explicitly as protobuf fields.

2.4 OPTIONAL Attributes & Extensions

OPTIONAL and extension attributes are represented using a map construct enabling direct support of the CloudEvent type system.

map<string, CloudEventAttributeValue> attributes = 1;

message CloudEventAttributeValue {

    oneof attr {
      bool ce_boolean = 1;
      int32 ce_integer = 2;
      string ce_string = 3;
      bytes ce_binary = 4;
      string ce_uri = 5;
      string ce_uri_reference = 6;
      google.protobuf.Timestamp ce_timestamp = 7;
    }
}

In this model an attribute's name is used as the map key and is associated with its value stored in the appropriately typed property.

This approach allows attributes to be represented and transported with no loss of type information.

3. Data

The specification allows for data payloads of the following types to be explicitly represented:

  • string
  • bytes
  • protobuf object/message
oneof data {

    // Binary data
    bytes binary_data = 2;

    // String data
    string text_data = 3;

    // Protobuf Message data
    google.protobuf.Any proto_data = 4;
}

  • Where the data is a protobuf message it MUST be stored in the proto_data property.

    • datacontenttype MAY be populated with application/protobuf
    • dataschema SHOULD be populated with the type URL of the protobuf data message.

  • When the type of the data is text, the value MUST be stored in the text_data property.

    • datacontenttype SHOULD be populated with the appropriate media-type.

  • When the type of the data is binary the value MUST be stored in the binary_data property.

    • datacontenttype SHOULD be populated with the appropriate media-type.

4. Transport

Transports that support content identification MUST use the following designation:

   application/cloudevents+protobuf

5. Batch Format

In the Protobuf Batch Format several CloudEvents are batched into a single Protobuf message. The message contains a repeated field filled with independent CloudEvent messages in the structured mode Protobuf event format.

5.1 Envelope

The enveloping container is a CloudEventBatch protobuf message containing a repeating set of CloudEvent message(s):

message CloudEventBatch {
  repeated CloudEvent events = 1;
}

5.2 Batch Media Type

A compliant protobuf batch representation is identifed using the following media-type

   application/cloudevents-batch+protobuf

6. Examples

The following code-snippets show how proto representations might be constructed assuming the availability of some convenience methods.

6.1 Plain Text event data

public static CloudEvent plainTextExample() {
  CloudEvent.Builder ceBuilder = CloudEvent.newBuilder();

  ceBuilder
    //-- REQUIRED Attributes.
    .setId(UUID.randomUUID().toString())
    .setSpecVersion("1.0")
    .setType("io.cloudevent.example")
    .setSource("producer-1")

    //-- Data.
    .setTextData("This is a plain text message");

  //-- OPTIONAL Attributes
  withCurrentTime(ceBuilder, "time");
  withAttribute(ceBuilder, "datacontenttype", "text/plain");

  // Build it.
  return ceBuilder.build();
}

6.2 Proto message as event data

Where the event data payload is itself a protobuf message (with its own schema) a protocol buffer idiomatic method can be used to carry the data.

private static Spec.CloudEvent protoExample() {

  //-- Build an event data protobuf object.
  Test.SomeData.Builder dataBuilder = Test.SomeData.newBuilder();

  dataBuilder
    .setSomeText("this is an important message")
    .setIsImportant(true);

  //-- Build the CloudEvent.
  CloudEvent.Builder ceBuilder = Spec.CloudEvent.newBuilder();

  ceBuilder
    .setId(UUID.randomUUID().toString())
    .setSpecVersion("1.0")
    .setType("io.cloudevent.example")
    .setSource("producer-2")

    // Add the proto data into the CloudEvent envelope.
    .setProtoData(Any.pack(dataBuilder.build()));

  // Add the protto type URL
  withAttribute(ceBuilder, "dataschema", ceBuilder.getProtoData().getTypeUrl());

  // Set Content-Type (OPTIONAL)
  withAttribute(ceBuilder, "datacontenttype", "application/protobuf");

  //-- Done.
  return ceBuilder.build();

}

References