CloudEvents integration #404

RaeesBhatti · 2018-03-29T12:19:01Z

No description provided.

codecov · 2018-03-29T12:56:45Z

Codecov Report

Merging #404 into master will decrease coverage by 0.93%.
The diff coverage is 70.37%.

@@            Coverage Diff             @@
##           master     #404      +/-   ##
==========================================
- Coverage   65.13%   64.19%   -0.94%     
==========================================
  Files          26       29       +3     
  Lines        1592     1648      +56     
==========================================
+ Hits         1037     1058      +21     
- Misses        512      547      +35     
  Partials       43       43

Impacted Files	Coverage Δ
event/http.go	`0% <0%> (ø)`
internal/http/headers.go	`100% <100%> (ø)`
router/event.go	`84.44% <100%> (-2.13%)`	⬇️
event/event.go	`69.35% <75%> (ø)`
router/router.go	`59.27% <83.33%> (-2.99%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3ffe17c...f4ae257. Read the comment docs.

ganeshvlrk · 2018-03-29T17:08:28Z

@mthenw @elsteelbrain I don't see any tests that validate the event format. What are the plans to add the same

alexdebrie

First review of changes based on shifting CloudEvents spec.

alexdebrie · 2018-03-29T21:50:16Z

event/event.go

-	ReceivedAt uint64      `json:"receivedAt"`
-	Data       interface{} `json:"data"`
-	DataType   string      `json:"dataType"`
+	Namespace          string `json:"namespace"`


Updates from today's CloudEvents meeting: Namespace was removed as a property.

alexdebrie · 2018-03-29T21:52:21Z

event/event.go

+	Source             struct {
+		SourceType string `json:"source-type"`
+		SourceID   string `json:"source-id"`
+	} `json:"source"`


Updates from today's CloudEvents meeting: The source object was changed to a string that's a URI.

alexdebrie · 2018-03-29T21:57:19Z

event/event.go

+		SourceID   string `json:"source-id"`
+	} `json:"source"`
+	EventID     string                 `json:"event-id"`
+	EventTime   uint64                 `json:"event-time"`


event-time needs to be in an RFC3339 format. I wonder if it's better to store as a time.Time instance? cc @mthenw

Yup, it should be time.Time. time.Time marshals to RFC 3339 by default.

alexdebrie · 2018-03-29T21:58:19Z

event/event.go

+	} `json:"source"`
+	EventID     string                 `json:"event-id"`
+	EventTime   uint64                 `json:"event-time"`
+	ContentType string                 `json:"content-type"`


There's a schema-url property that's been added to the spec. It's optional and must be a URI.

alexdebrie · 2018-03-29T22:01:05Z

event/event.go

+		EventType:   eventType,
+		EventID:     uuid.NewV4().String(),
+		EventTime:   uint64(time.Now().UnixNano() / int64(time.Millisecond)),
+		ContentType: mime,


New() should include a value for cloud-events-version (0.1) and source (TBD),

mthenw

Few things that are missing:

update readme with info that we now use Cloud Events format.
add integration test (tests package) that checks the CloudEvents format.
update source for http and custom event.

RaeesBhatti · 2018-04-02T15:08:27Z

@alexdebrie What would be the value for the source field?

alexdebrie · 2018-04-02T17:42:15Z

@elsteelbrain Hmm, I'm not sure. The new spec requires a URI, and one example shows http://example.com/repomanager.

@mthenw @ganeshvlrk Any ideas on what would be a good value for source here?

Full description:

Type: URI
Description: This describes the event producer. Often this will include
information such as the type of the event source, the organization
publishing the event, and some unique idenfitiers. The exact syntax and
semantics behind the data encoded in the URI is event producer defined.
Constraints:
- REQUIRED

ganeshvlrk · 2018-04-02T17:54:23Z

Based on the definition, I am not sure we can provide sane ootb values here except for cases where we eventually configure the event sources. Should we consider leaving this empty for now?

mthenw · 2018-04-03T08:13:55Z

docs/api.md

-* `receivedAt` - `number` - the time (milliseconds) when the Event was received by the Event Gateway (provided by the event gateway)
-* `data` - type depends on `dataType` - the event payload
-* `dataType` - `string` - the mime type of `data` payload
+* `event-type` - `string` - the event name


Let's remove list of fields from here and leave only example and link to original spec.

Do you want me to remove the example too or just this list of fields?

mthenw · 2018-04-03T08:14:08Z

docs/api.md

@@ -16,23 +16,27 @@ for invoking function. By default Events API runs on `:4000` port.

 ### Event Definition

-All data that passes through the Event Gateway is formatted as an Event, based on our default Event schema:
+All data that passes through the Event Gateway is formatted as an CloudEvent, based on CloudEvent schema:


"as a CloudEvent"

mthenw · 2018-04-03T08:14:21Z

docs/api.md

@@ -16,23 +16,27 @@ for invoking function. By default Events API runs on `:4000` port.

 ### Event Definition

-All data that passes through the Event Gateway is formatted as an Event, based on our default Event schema:
+All data that passes through the Event Gateway is formatted as an CloudEvent, based on CloudEvent schema:


Please specify spec version here.

mthenw · 2018-04-03T08:15:23Z

internal/zap/strings.go


 // Strings is a string array that implements MarshalLogArray.
 type Strings []string

+// MapStringInterface is a map that implements MarshalLogObject.


Please move it under MarshalLogArray function

Updated as requested

mthenw · 2018-04-10T13:11:06Z

event/event.go

+	EventTypeVersion   string                 `json:"eventTypeVersion"`
+	CloudEventsVersion string                 `json:"cloudEventsVersion" validate:"required"`
+	Source             string                 `json:"source" validate:"url,required"`
+	EventID            string                 `json:"eventId" validate:"required"`


eventID, not eventId

mthenw · 2018-04-10T13:11:48Z

event/event.go

+			event.Extensions = zap.MapStringInterface{
+				"eventgateway": map[string]interface{}{
+					"transformed": true,
+					"transformation-version": "0.1",


0.1 string is used in few places, please defined it as const

TransformationVersion

mthenw · 2018-04-10T13:25:55Z

router/event.go

-			event.Data = string(body)
-		}
+	var sourceURL string
+	if r.TLS != nil {


After thinking more about this one I think it won't work in all cases.

mthenw · 2018-04-11T09:33:12Z

event/event.go

+	event := &Event{
+		EventType:          eventType,
+		CloudEventsVersion: "0.1",
+		Source:             source + "#transformationVersion=0.1",


"https://serverless.com/event-gateway#transformationVersion=" + TransformationVersion

mthenw · 2018-04-11T09:33:20Z

event/event.go

+			event.Extensions = zap.MapStringInterface{
+				"eventgateway": map[string]interface{}{
+					"transformed": true,
+					"transformation-version": "0.1",


TransformationVersion

mthenw · 2018-04-11T12:14:20Z

docs/api.md

-* `receivedAt` - `number` - the time (milliseconds) when the Event was received by the Event Gateway (provided by the event gateway)
-* `data` - type depends on `dataType` - the event payload
-* `dataType` - `string` - the mime type of `data` payload
+All data that passes through the Event Gateway is formatted as a CloudEvent, based on [CloudEvents v0.1 schema](https://github.com/cloudevents/spec/blob/master/spec.md):


@alexdebrie can you talk a final look here. Should we add something?

@mthenw @elsteelbrain Docs are fine by me

mthenw · 2018-04-11T12:14:41Z

docs/api.md

+  "eventType": "myapp.user.created",
+  "eventID": "66dfc31d-6844-42fd-b1a7-a489a49f65f3",
+  "cloudEventsVersion": "0.1",
+  "source": "https://slsgateway.com/",


@elsteelbrain please update the example

mthenw · 2018-04-11T12:15:53Z

event/event.go

+	event := &Event{
+		EventType:          eventType,
+		CloudEventsVersion: "0.1",
+		Source:             "https://serverless.com/event-gateway/#transformationVersion=" + TransformationVersion,


https://serverless.com/event-gateway without trailing /

The correct address for the page is https://serverless.com/event-gateway/ with the slash.

mthenw · 2018-04-11T12:16:49Z

event/event.go

+	mimeJSON           = "application/json"
+	mimeFormMultipart  = "multipart/form-data"
+	mimeFormURLEncoded = "application/x-www-form-urlencoded"
+	TransformationVersion = "0.1"


let's define it outside of this mime const, under Type consts.

mthenw · 2018-04-11T12:18:04Z

event/event_test.go

+		eventpkg.Event{
+			EventType:          eventpkg.Type("user.created"),
+			CloudEventsVersion: eventpkg.TransformationVersion,
+			Source:             "https://slsgateway.com#transformationVersion=0.1",


Tests have to be updated.

Changes made as requested

RaeesBhatti added 4 commits March 29, 2018 16:16

Update Event type struct

cf400e5

go fmt

81d60cb

Fix linting

51df7bd

Fix a type

c0bdb89

ganeshvlrk requested review from mthenw and alexdebrie March 29, 2018 17:08

alexdebrie previously requested changes Mar 29, 2018

View reviewed changes

mthenw suggested changes Mar 30, 2018

View reviewed changes

RaeesBhatti added 7 commits March 30, 2018 13:40

Update the schema

a2ce635

Update README to reflect the CloudEvent

905fde5

Merge branch 'master' into cloudevents

03fd349

Write test for CloudEvents

a5e7cea

Merge branch 'master' into cloudevents

e9df992

Update README to move definitons for API

8ec5042

Add reference links to API doc

3279ff6

mthenw previously requested changes Apr 3, 2018

View reviewed changes

RaeesBhatti added 3 commits April 3, 2018 12:47

Reformat a bit

762cff0

Update docs

ce49e51

Docs: Remove CloudEvents definition fields

4fb5991

Add link to CloudEvents

7060f07

RaeesBhatti force-pushed the cloudevents branch from db9fe00 to 7060f07 Compare April 3, 2018 16:20

RaeesBhatti added 2 commits April 3, 2018 21:20

var notation

579655f

Try to parse custom events as CloudEvent

46f2a6d

Add extensions field if custom event not cloudevent

a44628c

RaeesBhatti force-pushed the cloudevents branch from a485f5b to a44628c Compare April 6, 2018 07:58

RaeesBhatti added 6 commits April 9, 2018 13:59

Update field names to camelcase

8e008f5

Update some field names

a2f5f24

Update event example

bfbaeec

Accept source param in new event func

22a27b0

Fix linter warning

614e9c3

Fix test to include tranformationversion anchor

9b48a3e

mthenw suggested changes Apr 10, 2018

View reviewed changes

RaeesBhatti added 8 commits April 10, 2018 18:27

Change a field name

e4984f4

Declare transformationVersion as const

f1df8b5

Fix a loop

a5441c4

Fix tests

ba7969c

Use const

e4c9714

Use a fixed value for Source

736037a

Use pkg constant in test

cb04faa

Update source value in tests

e025f7e

serverless deleted a comment from RaeesBhatti Apr 11, 2018

mthenw previously requested changes Apr 11, 2018

View reviewed changes

RaeesBhatti added 3 commits April 11, 2018 18:13

Update source in docs

9d0aa38

Move TransformationVersion const up

00a1ca2

Ignore test generated files

f1df44c

Maciej Winnicki added 5 commits April 12, 2018 10:53

add more tests

b9b2efc

minor cleanup

afc1ba0

add omitempty tag for optional fields

7de8e5f

don't log empty fields

f7242a3

rename var

f4ae257

mthenw merged commit 10ba8a1 into master Apr 12, 2018

mthenw deleted the cloudevents branch April 12, 2018 10:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CloudEvents integration #404

CloudEvents integration #404

RaeesBhatti commented Mar 29, 2018

codecov bot commented Mar 29, 2018 •

edited

Loading

ganeshvlrk commented Mar 29, 2018

alexdebrie left a comment

alexdebrie Mar 29, 2018

alexdebrie Mar 29, 2018

alexdebrie Mar 29, 2018

mthenw Mar 30, 2018

alexdebrie Mar 29, 2018

alexdebrie Mar 29, 2018

mthenw left a comment •

edited

Loading

RaeesBhatti commented Apr 2, 2018

alexdebrie commented Apr 2, 2018

ganeshvlrk commented Apr 2, 2018

mthenw Apr 3, 2018

RaeesBhatti Apr 3, 2018

mthenw Apr 3, 2018

mthenw Apr 3, 2018

mthenw Apr 3, 2018

mthenw Apr 10, 2018

mthenw Apr 10, 2018

mthenw Apr 11, 2018

mthenw Apr 10, 2018

RaeesBhatti Apr 10, 2018

mthenw Apr 11, 2018

mthenw Apr 11, 2018

mthenw Apr 11, 2018

alexdebrie Apr 11, 2018

mthenw Apr 11, 2018

mthenw Apr 11, 2018

RaeesBhatti Apr 11, 2018

mthenw Apr 11, 2018

mthenw Apr 11, 2018

CloudEvents integration #404

CloudEvents integration #404

Conversation

RaeesBhatti commented Mar 29, 2018

codecov bot commented Mar 29, 2018 • edited Loading

Codecov Report

ganeshvlrk commented Mar 29, 2018

alexdebrie left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mthenw left a comment • edited Loading

Choose a reason for hiding this comment

RaeesBhatti commented Apr 2, 2018

alexdebrie commented Apr 2, 2018

ganeshvlrk commented Apr 2, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Mar 29, 2018 •

edited

Loading

mthenw left a comment •

edited

Loading