Skip to content

Commit

Permalink
New: Added an asJSON-option to always populate array fields, even if …
Browse files Browse the repository at this point in the history
…defaults=false, see #597
  • Loading branch information
dcodeIO committed Dec 30, 2016
1 parent 961dd03 commit 40074bb
Show file tree
Hide file tree
Showing 17 changed files with 102 additions and 73 deletions.
55 changes: 27 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -237,25 +237,16 @@ There is also an [example for streaming RPC](https://github.com/dcodeIO/protobuf

### Usage with TypeScript

Under node.js:

```ts
import * as protobuf from "protobufjs";
import * as Long from "long"; // optional
...
```

In the browser:

```ts
/// <reference path="path/to/protobuf/index.d.ts" />
import * as protobuf from "path/to/protobuf.js";
...
```

If you need long support, there is also a [TypeScript definition](https://github.com/DefinitelyTyped/DefinitelyTyped/blob/types-2.0/long/index.d.ts) for that (on npm: [@types/long](https://www.npmjs.com/package/@types/long)).

See also: [Generating your own TypeScript definitions](https://github.com/dcodeIO/protobuf.js#generating-typescript-definitions-from-static-modules)

Additional configuration might be necessary when not utilizing node, i.e. reference [protobuf.js.d.ts](https://github.com/dcodeIO/protobuf.js/blob/master/index.d.ts) and [long.js.d.ts](https://github.com/DefinitelyTyped/DefinitelyTyped/blob/types-2.0/long/index.d.ts).

Module Structure
----------------
The library exports a flat `protobuf` namespace including but not restricted to the following members, ordered by category:
Expand Down Expand Up @@ -483,37 +474,45 @@ The package includes a benchmark that tries to compare performance to native JSO
```
benchmarking encoding performance ...
Type.encode to buffer x 521,803 ops/sec ±0.84% (88 runs sampled)
JSON.stringify to string x 300,362 ops/sec ±1.11% (86 runs sampled)
JSON.stringify to buffer x 169,413 ops/sec ±1.49% (86 runs sampled)
Type.encode to buffer x 553,499 ops/sec ±0.33% (91 runs sampled)
JSON.stringify to string x 313,354 ops/sec ±0.84% (89 runs sampled)
JSON.stringify to buffer x 177,932 ops/sec ±0.78% (88 runs sampled)
Type.encode to buffer was fastest
JSON.stringify to string was 42.6% slower
JSON.stringify to buffer was 67.7% slower
JSON.stringify to string was 43.7% slower
JSON.stringify to buffer was 68.0% slower
benchmarking decoding performance ...
Type.decode from buffer x 1,325,308 ops/sec ±1.46% (88 runs sampled)
JSON.parse from string x 283,907 ops/sec ±1.39% (86 runs sampled)
JSON.parse from buffer x 255,372 ops/sec ±1.28% (88 runs sampled)
Type.decode from buffer x 1,352,868 ops/sec ±0.66% (89 runs sampled)
JSON.parse from string x 293,883 ops/sec ±0.55% (92 runs sampled)
JSON.parse from buffer x 267,287 ops/sec ±0.83% (91 runs sampled)
Type.decode from buffer was fastest
JSON.parse from string was 78.6% slower
JSON.parse from buffer was 80.7% slower
JSON.parse from string was 78.3% slower
JSON.parse from buffer was 80.3% slower
benchmarking combined performance ...
Type to/from buffer x 269,719 ops/sec ±0.87% (91 runs sampled)
JSON to/from string x 122,878 ops/sec ±1.59% (87 runs sampled)
JSON to/from buffer x 89,310 ops/sec ±1.01% (88 runs sampled)
Type to/from buffer x 267,534 ops/sec ±0.88% (91 runs sampled)
JSON to/from string x 129,143 ops/sec ±0.66% (92 runs sampled)
JSON to/from buffer x 91,789 ops/sec ±0.73% (87 runs sampled)
Type to/from buffer was fastest
JSON to/from string was 54.8% slower
JSON to/from buffer was 66.9% slower
JSON to/from string was 51.6% slower
JSON to/from buffer was 65.6% slower
benchmarking verifying performance ...
Type.verify x 5,857,856 ops/sec ±0.82% (91 runs sampled)
Type.verify x 6,431,917 ops/sec ±0.49% (91 runs sampled)
benchmarking converting performance ...
Message.from x 629,785 ops/sec ±0.62% (94 runs sampled)
Message#asJSON x 609,017 ops/sec ±0.74% (93 runs sampled)
Message.from was fastest
Message#asJSON was 3.4% slower
```

Note that JSON is a native binding nowadays and as such is about as fast as it possibly can get. So, how can protobuf.js be faster?
Expand Down
24 changes: 18 additions & 6 deletions dist/noparse/protobuf.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion dist/noparse/protobuf.js.map

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions dist/noparse/protobuf.min.js

Large diffs are not rendered by default.

Binary file modified dist/noparse/protobuf.min.js.gz
Binary file not shown.
2 changes: 1 addition & 1 deletion dist/noparse/protobuf.min.js.map

Large diffs are not rendered by default.

24 changes: 18 additions & 6 deletions dist/protobuf.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion dist/protobuf.js.map

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions dist/protobuf.min.js

Large diffs are not rendered by default.

Binary file modified dist/protobuf.min.js.gz
Binary file not shown.
2 changes: 1 addition & 1 deletion dist/protobuf.min.js.map

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion dist/runtime/protobuf.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion dist/runtime/protobuf.min.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Binary file modified dist/runtime/protobuf.min.js.gz
Binary file not shown.
12 changes: 7 additions & 5 deletions index.d.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
// $> pbts --name protobuf --out index.d.ts src
// Generated Fri, 30 Dec 2016 15:32:08 UTC
// Generated Fri, 30 Dec 2016 16:29:57 UTC

export = protobuf;
export as namespace protobuf;
Expand Down Expand Up @@ -122,24 +122,26 @@ declare namespace protobuf {
* JSON conversion options as used by {@link Message#asJSON} with {@link convert}.
* @typedef JSONConversionOptions
* @type {Object}
* @property {boolean} [fieldsOnly=false] Keeps only properties that reference a field
* @property {boolean} [fieldsOnly=false] If `true`, keeps only properties that reference a field.
* @property {*} [longs] Long conversion type. Only relevant with a long library.
* Valid values are `String` and `Number` (the global types).
* Defaults to a possibly unsafe number without, and a `Long` with a long library.
* @property {*} [enums=Number] Enum value conversion type.
* Valid values are `String` and `Number` (the global types).
* Defaults to the numeric ids.
* Defaults to `Number`, which sets the numeric ids.
* @property {*} [bytes] Bytes value conversion type.
* Valid values are `Array` and `String` (the global types).
* Defaults to return the underlying buffer type.
* @property {boolean} [defaults=false] Also sets default values on the resulting object
* Defaults to return the underlying buffer type, which usually is an `Uint8Array` in the browser and a `Buffer` under node.
* @property {boolean} [defaults=false] If `true`, sets default values on the resulting object including empty arrays for repeated fields
* @property {boolean} [arrays=false] If `true`, always initializes empty arrays for repeated fields. Only relevant with `defaults=false`.
*/
interface JSONConversionOptions {
fieldsOnly?: boolean;
longs?: any;
enums?: any;
bytes?: any;
defaults?: boolean;
arrays?: boolean;
}

/**
Expand Down
11 changes: 6 additions & 5 deletions src/convert.js
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ function convert(type, source, destination, options, converter) {
value = source[key];
if (field) {
if (field.repeated) {
if (value || options.defaults) {
if (value || options.defaults || options.arrays) {
destination[key] = [];
if (value)
for (var j = 0, l = value.length; j < l; ++j)
Expand All @@ -62,17 +62,18 @@ function convert(type, source, destination, options, converter) {
* JSON conversion options as used by {@link Message#asJSON} with {@link convert}.
* @typedef JSONConversionOptions
* @type {Object}
* @property {boolean} [fieldsOnly=false] Keeps only properties that reference a field
* @property {boolean} [fieldsOnly=false] If `true`, keeps only properties that reference a field.
* @property {*} [longs] Long conversion type. Only relevant with a long library.
* Valid values are `String` and `Number` (the global types).
* Defaults to a possibly unsafe number without, and a `Long` with a long library.
* @property {*} [enums=Number] Enum value conversion type.
* Valid values are `String` and `Number` (the global types).
* Defaults to the numeric ids.
* Defaults to `Number`, which sets the numeric ids.
* @property {*} [bytes] Bytes value conversion type.
* Valid values are `Array` and `String` (the global types).
* Defaults to return the underlying buffer type.
* @property {boolean} [defaults=false] Also sets default values on the resulting object
* Defaults to return the underlying buffer type, which usually is an `Uint8Array` in the browser and a `Buffer` under node.
* @property {boolean} [defaults=false] If `true`, sets default values on the resulting object including empty arrays for repeated fields
* @property {boolean} [arrays=false] If `true`, always initializes empty arrays for repeated fields. Only relevant with `defaults=false`.
*/
/**/
convert.toJson = function toJson(field, value, options) {
Expand Down
25 changes: 14 additions & 11 deletions tests/typescript.ts
Original file line number Diff line number Diff line change
@@ -1,13 +1,18 @@
import * as protobuf from "..";
import * as Long from "long";

export const proto = {
nested: {
Hello: {
fields: {
value: {
aString: {
rule: "required",
type: "string",
id:1
id: 1
},
aLong: {
type: "uint64",
id: 2
}
}
}
Expand All @@ -20,17 +25,15 @@ export class Hello extends protobuf.Message {
constructor (properties?: { [k: string]: any }) {
super(properties);
}

foo() {
this["value"] = "hi";
return this;
}
}
protobuf.Class.create(root.lookup("Hello") as protobuf.Type, Hello);
protobuf.Class.create(root.lookupType("Hello"), Hello);

var hello = new Hello();
var hello = new Hello({
aString: "hi",
aLong: Long.fromNumber(123)
});

var buf = Hello.encode(hello.foo()).finish();
var buf = Hello.encode(hello).finish();

var hello2 = Hello.decode(buf) as Hello;
console.log(hello2.foo().asJSON());
console.log(hello2.asJSON());

1 comment on commit 40074bb

@robin-anil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not enough. var keys = Object.keys(options.defaults ? type.fields : source); should be

var keys = Object.keys(options.defaults || options.arrays ? type.fields : source);

Please sign in to comment.