From c968a3f1d43bb13f053f85cff974069e9fe75b37 Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Fri, 8 Nov 2024 17:54:44 -0500 Subject: [PATCH 01/13] Migration guide for v2 Signed-off-by: Steve Ayers --- README.md | 128 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 128 insertions(+) diff --git a/README.md b/README.md index 30c26101..4d5b95cf 100644 --- a/README.md +++ b/README.md @@ -68,6 +68,134 @@ You can serve your Connect RPCs with vanilla Node.js, or use our [server plugins for **Fastify**, **Next.js**, and **Express**. We support Node.js v18.14.1 and later with the builtin `http` and `http2` modules. +## Migrating from version 1 + +Version 2 provides many new features and simplifies some common APIs. In addition, +it makes use of all the enhancements of Protobuf-ES v2. To upgrade, you'll need +to update your dependencies, re-generate code, and update call sites in your application. + +The first thing to be aware of is that the plugin `protoc-gen-connect-es` has been +removed in v2. Connect now relies on descriptors generated by Protobuf-ES v2. +When upgrading, you will first need to remove any usage of this plugin. For most +users, this means a combination of any of the following steps: + +* Remove `protoc-gen-connect-es` from their `package.json` either manually or + via a package manager, i.e. `npm uninstall @connectrpc/protoc-gen-connect-es`. + +* Remove any usage of the plugin in `buf.gen.yaml`. + +* Remove any dependency on Generated SDKs for Connect from `package.json` using + the same approach used for removing the plugin. Users should then rely on + types from the `@bufbuild_es` plugin instead. This will also necessitate a code + change, which is usually a simple import path update from *_connect to *_pb. + +Next, as mentioned, Connect uses Protobuf-ES v2, so you will need to upgrade any +dependencies you have on its relevant packages: + +```shellsession +npm install @bufbuild/protobuf@^2.0.0 @bufbuild/protoc-gen-es@^2.0.0 +``` + +Finally, update the [packages](#packages) you use with your package manager of choice. +For example, with NPM: + +```shellsession +npm install @connectrpc/connect@^2.0.0 @connectrpc/connect-web@^2.0.0 +``` + +### Update your code + +Next, you'll need to account for any breaking changes. Following is a list to +be aware of: + +* `createGrpcTransport` now requires HTTP/2. + + If you are using `createGrpcTransport` and specifying an `httpVersion`, it will + now fail compilation. Remove the `httpVersion` property to use the default. + Note that if you were relying on HTTP/1.1 as part of your gRPC strategy, this + may require bigger architectural changes, but the hope is that this is not a + common problem. + +* The deprecated `createPromiseClient` has been removed. + + Promise clients are now the default. Any callsites using `createPromiseClient` + should be updated to use `createClient`. + +* Node 16 is no longer supported. + + Connect v2 now supports Node versions 18.14.1 and up. + +* Requests with similar shapes can no longer be used interchangeably. + + Previously, Connect allowed request objects with matching shapes to be passed + to API calls interchangeably as long as the passed object was a superset of the + target type. For example, given the following proto definitions: + + ```protobuf + syntax = "proto3"; + package example.v1; + message MessageA { + string field_a = 1; + } + message MessageB { + string field_a = 1; + int64 field_b = 2; + } + service ExampleService { + rpc RequestA(MessageA) returns (Empty) {} + rpc RequestB(MessageB) returns (Empty) {} + } + ``` + + The following would have passed TypeScript compilation: + + ```ts + client.requestA(new MessageA()) + client.requestA(new MessageB()) + ``` + + This was an unintended bug and not a feature and in Connect v2, only the specified + target type will pass compilation. + +* `fetch`-related options `credentials` and `init` have been removed. + + We have removed the `credentials` option from transports as well as the `init` + option in interceptors. These two options were used to customize `fetch` routines. + Users should now rely on the `fetch` option in transports as a way to perform + these customizations. For example: + + ```ts + createConnectTransport({ + baseUrl: "/", + fetch: (input, init) => fetch(input, {...init, credentials: "include"}), + }); + ``` + + In addition, as a replacement to determine whether an incoming request is a + Connect GET request in server-side interceptors, the property `requestMethod: string` + has been added to intercepted requests. This property is symmetrical to `HandlerContext.requestMethod`. + +* The access pattern of `MethodDescriptor` has slightly changed. + + In v2, the Protobuf-ES plugin generates service descriptors and changes the + access pattern for method descriptors. Instead of the plural form `methods`, + it now uses the singular `method`. + + For example: `ElizaService.methods.Say` becomes `ElizaService.method.Say`. + +* Errors details are a pair of desc and init values + + TBD + +* MethodDescriptor is self sufficient + + TBD + +* Interceptors always use stream types + + TBD + + ## Other platforms Would you like to use Connect on other platforms like Bun, Deno, Vercel’s Edge Runtime, From 4fa79617ec255da30f44bce4167ad39ebb7b5908 Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Mon, 11 Nov 2024 11:29:51 -0500 Subject: [PATCH 02/13] Updates Signed-off-by: Steve Ayers --- README.md | 89 ++++++++++++++++++++++++++++++++++++++----------------- 1 file changed, 61 insertions(+), 28 deletions(-) diff --git a/README.md b/README.md index 4d5b95cf..841a5cfc 100644 --- a/README.md +++ b/README.md @@ -108,26 +108,21 @@ npm install @connectrpc/connect@^2.0.0 @connectrpc/connect-web@^2.0.0 Next, you'll need to account for any breaking changes. Following is a list to be aware of: -* `createGrpcTransport` now requires HTTP/2. +* The gRPC Transport now requires HTTP/2. If you are using `createGrpcTransport` + and specifying an `httpVersion`, it will fail compilation. Remove the + `httpVersion` property to use the default. - If you are using `createGrpcTransport` and specifying an `httpVersion`, it will - now fail compilation. Remove the `httpVersion` property to use the default. Note that if you were relying on HTTP/1.1 as part of your gRPC strategy, this may require bigger architectural changes, but the hope is that this is not a common problem. -* The deprecated `createPromiseClient` has been removed. - - Promise clients are now the default. Any callsites using `createPromiseClient` +* Promise clients are now the default and the previously-deprecated + `createPromiseClient` has been removed. Any callsites using `createPromiseClient` should be updated to use `createClient`. -* Node 16 is no longer supported. - - Connect v2 now supports Node versions 18.14.1 and up. - -* Requests with similar shapes can no longer be used interchangeably. +* Node 16 is no longer supported. Connect v2 now supports Node versions 18.14.1 and up. - Previously, Connect allowed request objects with matching shapes to be passed +* Previously, Connect allowed request objects with matching shapes to be passed to API calls interchangeably as long as the passed object was a superset of the target type. For example, given the following proto definitions: @@ -157,9 +152,7 @@ be aware of: This was an unintended bug and not a feature and in Connect v2, only the specified target type will pass compilation. -* `fetch`-related options `credentials` and `init` have been removed. - - We have removed the `credentials` option from transports as well as the `init` +* We have removed the `credentials` option from transports as well as the `init` option in interceptors. These two options were used to customize `fetch` routines. Users should now rely on the `fetch` option in transports as a way to perform these customizations. For example: @@ -175,26 +168,66 @@ be aware of: Connect GET request in server-side interceptors, the property `requestMethod: string` has been added to intercepted requests. This property is symmetrical to `HandlerContext.requestMethod`. -* The access pattern of `MethodDescriptor` has slightly changed. - - In v2, the Protobuf-ES plugin generates service descriptors and changes the +* The access pattern of `MethodDescriptor` has slightly changed. In v2, the + Protobuf-ES plugin generates service descriptors and changes the access pattern for method descriptors. Instead of the plural form `methods`, it now uses the singular `method`. - For example: `ElizaService.methods.Say` becomes `ElizaService.method.Say`. - -* Errors details are a pair of desc and init values - - TBD - -* MethodDescriptor is self sufficient + For example: `ElizaService.methods.Say` now becomes `ElizaService.method.Say`. + +* Errors details are now a pair of desc and init values. In Connect v1, error + details were specified as message instances. In v2, error + details are now an object that specifies both a schema and initialization + object which are both passed to the `create` function of Protobuf-ES. For example: + + ```diff + - import { LocalizedMessage } from "./gen/google/rpc/error_details_pb"; + - const details = [ + - new LocalizedMessage({ + - locale: "fr-CH", + - message: "Je n'ai plus de mots.", + - }), + - ]; + + import { LocalizedMessageSchema } from "./gen/google/rpc/error_details_pb"; + + const details = [ + + { + + desc: LocalizedMessageSchema, + + value: { + + locale: "fr-CH", + + message: "Je n'ai plus de mots.", + + } + + }, + + ]; + const metadata = new Headers({ + "words-left": "none" + }); + throw new ConnectError( + "I have no words anymore.", + Code.ResourceExhausted, + metadata, + details + ); + ``` - TBD +* `MethodDescriptor` is now self-sufficient. In v1, method descriptors always + had to be used alongside the service descriptors. In v2, they can now be used + standalone. This means that all callsites that previously passed a service + and method descriptor can now just pass the method descriptor. -* Interceptors always use stream types + A notable example is when using a Connect `Router` on the server-side: - TBD + ```diff + const routes = ({rpc}: Router) => { + - rpc(ElizaService, ElizaService.say, impl); + + rpc(ElizaService.say, impl); + } + ``` +* Interceptors for streaming RPCs now use appropriate stream types. In + v1, the server used more exact types in interceptors, for example `UnaryRequest` + for server-streaming rpcs while the client always used streaming variants. + This was unintended behavior and has been fixed in v2. Now all streaming RPCs + use the `StreamRequest` and `StreamResponse` on the server as well. ## Other platforms From 29763e1101916dbcbac0d43cf1cd9db3dc77c597 Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Mon, 11 Nov 2024 11:33:47 -0500 Subject: [PATCH 03/13] Cleanup Signed-off-by: Steve Ayers --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 841a5cfc..f39275ad 100644 --- a/README.md +++ b/README.md @@ -87,7 +87,7 @@ users, this means a combination of any of the following steps: * Remove any dependency on Generated SDKs for Connect from `package.json` using the same approach used for removing the plugin. Users should then rely on types from the `@bufbuild_es` plugin instead. This will also necessitate a code - change, which is usually a simple import path update from *_connect to *_pb. + change, which is usually a simple import path update from `*_connect` to `*_pb`. Next, as mentioned, Connect uses Protobuf-ES v2, so you will need to upgrade any dependencies you have on its relevant packages: @@ -198,6 +198,7 @@ be aware of: + } + }, + ]; + const metadata = new Headers({ "words-left": "none" }); From 9c9d49795feb953cf9f74bb6d937f3cfb970fac2 Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Mon, 11 Nov 2024 11:41:24 -0500 Subject: [PATCH 04/13] Cleanup Signed-off-by: Steve Ayers --- README.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index f39275ad..7f50199d 100644 --- a/README.md +++ b/README.md @@ -70,16 +70,16 @@ the builtin `http` and `http2` modules. ## Migrating from version 1 -Version 2 provides many new features and simplifies some common APIs. In addition, +Version 2 provides new features and simplifies some common APIs. In addition, it makes use of all the enhancements of Protobuf-ES v2. To upgrade, you'll need -to update your dependencies, re-generate code, and update call sites in your application. +to update your dependencies, re-generate code, and update callsites in your application. The first thing to be aware of is that the plugin `protoc-gen-connect-es` has been removed in v2. Connect now relies on descriptors generated by Protobuf-ES v2. When upgrading, you will first need to remove any usage of this plugin. For most users, this means a combination of any of the following steps: -* Remove `protoc-gen-connect-es` from their `package.json` either manually or +* Remove `protoc-gen-connect-es` from `package.json` either manually or via a package manager, i.e. `npm uninstall @connectrpc/protoc-gen-connect-es`. * Remove any usage of the plugin in `buf.gen.yaml`. @@ -96,8 +96,8 @@ dependencies you have on its relevant packages: npm install @bufbuild/protobuf@^2.0.0 @bufbuild/protoc-gen-es@^2.0.0 ``` -Finally, update the [packages](#packages) you use with your package manager of choice. -For example, with NPM: +Finally, update the Connect [packages](#packages) you use with your package +manager of choice. For example, with NPM: ```shellsession npm install @connectrpc/connect@^2.0.0 @connectrpc/connect-web@^2.0.0 @@ -110,7 +110,7 @@ be aware of: * The gRPC Transport now requires HTTP/2. If you are using `createGrpcTransport` and specifying an `httpVersion`, it will fail compilation. Remove the - `httpVersion` property to use the default. + `httpVersion` property to use the default of HTTP/2. Note that if you were relying on HTTP/1.1 as part of your gRPC strategy, this may require bigger architectural changes, but the hope is that this is not a @@ -149,7 +149,7 @@ be aware of: client.requestA(new MessageB()) ``` - This was an unintended bug and not a feature and in Connect v2, only the specified + This was an unintended bug and not a feature. In Connect v2, only the specified target type will pass compilation. * We have removed the `credentials` option from transports as well as the `init` @@ -228,7 +228,7 @@ be aware of: v1, the server used more exact types in interceptors, for example `UnaryRequest` for server-streaming rpcs while the client always used streaming variants. This was unintended behavior and has been fixed in v2. Now all streaming RPCs - use the `StreamRequest` and `StreamResponse` on the server as well. + use the `StreamRequest` and `StreamResponse` types on the server as well. ## Other platforms From a2dbba68cd224eca278eccacf6dc41303a67b49a Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Mon, 11 Nov 2024 11:43:10 -0500 Subject: [PATCH 05/13] Format Signed-off-by: Steve Ayers --- README.md | 62 +++++++++++++++++++++++++++---------------------------- 1 file changed, 31 insertions(+), 31 deletions(-) diff --git a/README.md b/README.md index 7f50199d..0f2d8f8f 100644 --- a/README.md +++ b/README.md @@ -70,23 +70,23 @@ the builtin `http` and `http2` modules. ## Migrating from version 1 -Version 2 provides new features and simplifies some common APIs. In addition, -it makes use of all the enhancements of Protobuf-ES v2. To upgrade, you'll need +Version 2 provides new features and simplifies some common APIs. In addition, +it makes use of all the enhancements of Protobuf-ES v2. To upgrade, you'll need to update your dependencies, re-generate code, and update callsites in your application. The first thing to be aware of is that the plugin `protoc-gen-connect-es` has been -removed in v2. Connect now relies on descriptors generated by Protobuf-ES v2. +removed in v2. Connect now relies on descriptors generated by Protobuf-ES v2. When upgrading, you will first need to remove any usage of this plugin. For most users, this means a combination of any of the following steps: -* Remove `protoc-gen-connect-es` from `package.json` either manually or +- Remove `protoc-gen-connect-es` from `package.json` either manually or via a package manager, i.e. `npm uninstall @connectrpc/protoc-gen-connect-es`. -* Remove any usage of the plugin in `buf.gen.yaml`. +- Remove any usage of the plugin in `buf.gen.yaml`. -* Remove any dependency on Generated SDKs for Connect from `package.json` using - the same approach used for removing the plugin. Users should then rely on - types from the `@bufbuild_es` plugin instead. This will also necessitate a code +- Remove any dependency on Generated SDKs for Connect from `package.json` using + the same approach used for removing the plugin. Users should then rely on + types from the `@bufbuild_es` plugin instead. This will also necessitate a code change, which is usually a simple import path update from `*_connect` to `*_pb`. Next, as mentioned, Connect uses Protobuf-ES v2, so you will need to upgrade any @@ -96,7 +96,7 @@ dependencies you have on its relevant packages: npm install @bufbuild/protobuf@^2.0.0 @bufbuild/protoc-gen-es@^2.0.0 ``` -Finally, update the Connect [packages](#packages) you use with your package +Finally, update the Connect [packages](#packages) you use with your package manager of choice. For example, with NPM: ```shellsession @@ -108,21 +108,21 @@ npm install @connectrpc/connect@^2.0.0 @connectrpc/connect-web@^2.0.0 Next, you'll need to account for any breaking changes. Following is a list to be aware of: -* The gRPC Transport now requires HTTP/2. If you are using `createGrpcTransport` - and specifying an `httpVersion`, it will fail compilation. Remove the - `httpVersion` property to use the default of HTTP/2. +- The gRPC Transport now requires HTTP/2. If you are using `createGrpcTransport` + and specifying an `httpVersion`, it will fail compilation. Remove the + `httpVersion` property to use the default of HTTP/2. Note that if you were relying on HTTP/1.1 as part of your gRPC strategy, this may require bigger architectural changes, but the hope is that this is not a common problem. -* Promise clients are now the default and the previously-deprecated +- Promise clients are now the default and the previously-deprecated `createPromiseClient` has been removed. Any callsites using `createPromiseClient` should be updated to use `createClient`. -* Node 16 is no longer supported. Connect v2 now supports Node versions 18.14.1 and up. +- Node 16 is no longer supported. Connect v2 now supports Node versions 18.14.1 and up. -* Previously, Connect allowed request objects with matching shapes to be passed +- Previously, Connect allowed request objects with matching shapes to be passed to API calls interchangeably as long as the passed object was a superset of the target type. For example, given the following proto definitions: @@ -145,14 +145,14 @@ be aware of: The following would have passed TypeScript compilation: ```ts - client.requestA(new MessageA()) - client.requestA(new MessageB()) + client.requestA(new MessageA()); + client.requestA(new MessageB()); ``` This was an unintended bug and not a feature. In Connect v2, only the specified target type will pass compilation. -* We have removed the `credentials` option from transports as well as the `init` +- We have removed the `credentials` option from transports as well as the `init` option in interceptors. These two options were used to customize `fetch` routines. Users should now rely on the `fetch` option in transports as a way to perform these customizations. For example: @@ -160,23 +160,23 @@ be aware of: ```ts createConnectTransport({ baseUrl: "/", - fetch: (input, init) => fetch(input, {...init, credentials: "include"}), + fetch: (input, init) => fetch(input, { ...init, credentials: "include" }), }); ``` - In addition, as a replacement to determine whether an incoming request is a + In addition, as a replacement to determine whether an incoming request is a Connect GET request in server-side interceptors, the property `requestMethod: string` has been added to intercepted requests. This property is symmetrical to `HandlerContext.requestMethod`. -* The access pattern of `MethodDescriptor` has slightly changed. In v2, the - Protobuf-ES plugin generates service descriptors and changes the +- The access pattern of `MethodDescriptor` has slightly changed. In v2, the + Protobuf-ES plugin generates service descriptors and changes the access pattern for method descriptors. Instead of the plural form `methods`, it now uses the singular `method`. For example: `ElizaService.methods.Say` now becomes `ElizaService.method.Say`. -* Errors details are now a pair of desc and init values. In Connect v1, error - details were specified as message instances. In v2, error +- Errors details are now a pair of desc and init values. In Connect v1, error + details were specified as message instances. In v2, error details are now an object that specifies both a schema and initialization object which are both passed to the `create` function of Protobuf-ES. For example: @@ -201,7 +201,7 @@ be aware of: const metadata = new Headers({ "words-left": "none" - }); + }); throw new ConnectError( "I have no words anymore.", Code.ResourceExhausted, @@ -210,10 +210,10 @@ be aware of: ); ``` -* `MethodDescriptor` is now self-sufficient. In v1, method descriptors always +- `MethodDescriptor` is now self-sufficient. In v1, method descriptors always had to be used alongside the service descriptors. In v2, they can now be used - standalone. This means that all callsites that previously passed a service - and method descriptor can now just pass the method descriptor. + standalone. This means that all callsites that previously passed a service + and method descriptor can now just pass the method descriptor. A notable example is when using a Connect `Router` on the server-side: @@ -224,11 +224,11 @@ be aware of: } ``` -* Interceptors for streaming RPCs now use appropriate stream types. In +- Interceptors for streaming RPCs now use appropriate stream types. In v1, the server used more exact types in interceptors, for example `UnaryRequest` - for server-streaming rpcs while the client always used streaming variants. + for server-streaming rpcs while the client always used streaming variants. This was unintended behavior and has been fixed in v2. Now all streaming RPCs - use the `StreamRequest` and `StreamResponse` types on the server as well. + use the `StreamRequest` and `StreamResponse` types on the server as well. ## Other platforms From 470f985f480e0b083133881f4e6ea4115b1ae1f8 Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Tue, 12 Nov 2024 13:22:13 -0500 Subject: [PATCH 06/13] Round 2 Signed-off-by: Steve Ayers --- MIGRATING.md | 292 +++++++++++++++++++++++++++++++++++++++++++++++++++ README.md | 160 +--------------------------- 2 files changed, 293 insertions(+), 159 deletions(-) create mode 100644 MIGRATING.md diff --git a/MIGRATING.md b/MIGRATING.md new file mode 100644 index 00000000..f0bf26c5 --- /dev/null +++ b/MIGRATING.md @@ -0,0 +1,292 @@ +# Migration Guide + +## Migrating from version 1 to version 2 + +Connect v2 provides new features and simplifies some common APIs. In addition, +it makes use of all the enhancements of Protobuf-ES v2. This document walks you +through all you need to know. + +### Migrating your code + +To help with the process of migrating, we provide a tool called +[@connectrpc/connect-migrate](https://www.npmjs.com/package/@connectrpc/connect-migrate) +which will take care of dependency and plugin updates as well as a few minor code +changes. So, as a first step, you will need to run the migration tool. To do so, +execute the following command: + +`npx @connectrpc/connect-migrate@v2.0.0` + +While the tool will do a lot of the dependency legwork for you, there are many +use cases in code that it does not cover. Below, we list everything that has changed in +v2 and whether it is fixed by the migration tool or whether it will require +manual intervention. + +In summary, the suggested way to migrate to Connect v2 is to first run the migration tool +and then go through the list below, fixing any other instances that weren't handled. + +### Updating dependencies + +Perhaps the biggest thing to be aware of is that the plugin `protoc-gen-connect-es` has been +removed in v2. Connect now relies on service descriptors generated by the Protobuf-ES v2 +`protoc-gen-es` and no longer generates code itself. + +So the first major part of the migration path is removing any usage of the old +`protoc-gen-connect-es` plugin. This means a combination of any +of the following: + +- Removing `protoc-gen-connect-es` from `package.json`. + +- Removing any usage of the plugin in `buf.gen.yaml`. + +- Removing any dependency on Generated SDKs for Connect from `package.json`. + +Next, upgrade any dependencies on [Protobuf-ES packages](https://github.com/bufbuild/protobuf-es?tab=readme-ov-file#packages) +to use v2. + +Finally, update the Connect [packages](#packages) in your project to use v2. + +:white_check_mark: The `connect-migrate` tool will handle all of these changes. + +### Generate code + +The next step is to re-generate code. The migration tool does not handle code +generation, so be sure to do so in whatever way your project is configured. For +example, `npx buf generate`, etc. + +There are a few things to be aware with the switch to the Protobuf-ES v2 plugin +as some of the options for that plugin have changed with its v2 release: + +* The `import_extension` option is now set to `none` by default, which means it doesn't + add a `.js` extension to import paths. If you use the plugin option + `import_extension=none`, you can delete it. If you require imports to have the + `.js` extension, use `import_extension=js`. +* The `ts_nocheck` option is now off by default. If you require a `// @ts-nocheck` + annotation at the top of generated code, use the option `ts_nocheck=true`. + +For more information, see the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md#migrating-from-version-1). + +When generating code, you should make sure to delete any now-unused `*_connect.ts` +files. An easy way to do this is to use the `clean` option provided by the Buf CLI, +introduced in v1.36.0. See the docs [here](https://buf.build/docs/configuration/v2/buf-gen-yaml/?h=clean) +for specifying `clean` in your config files. Note that you can also specify `--clean` +to any `buf generate` CLI commands you are running. + +### Update import paths + +Once your code is generated and the vestigial `*_connect` files are removed, +import paths should be updated. This is usually an update from `*_connect` to +`*_pb`: + +```diff +- import { ElizaService } from "./gen/eliza_connect.js"; ++ import { ElizaService } from "./gen/eliza_pb.js"; +``` + +:white_check_mark: The `connect-migrate` tool will handle this. + + +### Update usages with generated code + +As mentioned, Connect now solely uses the `protoc-gen-es` plugin provided by +Protobuf-ES v2. As a result, working with generated code has changed in some +areas. + +#### Creating messages + +To create a new instance, you now call the function `create()` provided by +Protobuf-ES and pass the generated schema: + +```diff +- import { SayRequest } from "./gen/eliza_connect.js"; +- import { SayRequestSchema } from "./gen/eliza_pb.js"; ++ import { create } from "@bufbuild/protobuf"; + +- const sayRequest = new SayRequest({ ++ const sayRequest = create(UserSchema, { + sentence: "Hello", +}); +``` + +#### Service descriptors + +Connect now relies on service descriptors generated by the Protobuf-ES v2 +`protoc-gen-es` plugin. The v2 service descriptors aren't much different from +the service descriptors previously generated by Connect v1. The same basic +information i.e. (typed metadata) is still generated, but it's generated by +`protoc-gen-es` now, along with additional information available. For example, +the generated code in v2 now provides the full descriptor, with +features such as [custom options](https://github.com/bufbuild/protobuf-es/blob/v2.2.2/MANUAL.md#custom-options), +which can be very useful in interceptors. + +One breaking change to be aware with service descriptors is that the access +pattern of the `MethodDescriptor` has slightly changed. + +Service descriptors in Connect v1 looked like this: + +```ts +// protoc-gen-connect-es v1 +import { ElizaService } from "./gen/eliza_connect"; +ElizaService.typeName; // "connectrpc.eliza.v1.ElizaService" +ElizaService.methods.say; +ElizaService.methods.say.name; // "Say" +ElizaService.methods.say.kind; // MethodKind.Unary +ElizaService.methods.say.idempotency; // MethodIdempotency.NoSideEffects +``` + +Compare this to what is now generated by `protoc-gen-es` v2: + +```ts +// protoc-gen-es v2 +import { ElizaService } from "./gen/eliza_pb"; +ElizaService.typeName; // "connectrpc.eliza.v1.ElizaService" +ElizaService.method.say; +ElizaService.method.say.name; // "Say" +ElizaService.method.say.methodKind; // "unary" +ElizaService.method.say.idempotency; // MethodOptions_IdempotencyLevel.NoSideEffects +``` + +Notice that instead of the plural form `methods`, it now uses the singular +`method`. + +Other than the above, the generated properties remain largely unchanged and +request and responses do not need any updates when initializers are used or +dot notation is used. However, a few noteworthy changes to be aware of: + +* The `toPlainMessage` function and the `PlainMessage` type are no longer + necessary. If you create a proto3 message with `create(UserSchema)`, the + returned object is already a plain object. You can replace the + `PlainMessage` types with `User`. The only difference is that `User` has + a property `$typeName`, which is a simple string with the full name of + the message like `"example.User"`. This property makes sure you don't pass the + wrong message to a function by accident. +* A message field using [`google.protobuf.Struct`](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md#googleprotobufstruct) + is now generated as more-convenient `JsonObject`. +* `proto2` fields with default values are no longer generated as optional + properties. +* All well-known types and helpers have been moved to an export under + `@bufbuild/protobuf/wkt`. For example, when converting a `google.protobuf.Timestamp` + to an ECMAScript Date object, it is no longer possible via a method. + +For more information on all Protobuf-ES v2 changes, see the +[Protobuf-ES v2 migration guide](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md#update-your-code). + +### Other breaking changes + +* Promise clients are now the default and the previously-deprecated + `createPromiseClient` has been removed. Any callsites using `createPromiseClient` + should be updated to use `createClient`. + + :white_check_mark: The `connect-migrate` tool will handle this. + +* The gRPC Transport now requires HTTP/2. If you are using `createGrpcTransport` + and specifying an `httpVersion`, it will fail compilation. Remove the + `httpVersion` property to use the default of HTTP/2. + + Note that if you were relying on HTTP/1.1 as part of your gRPC strategy, this + may require bigger architectural changes, but the hope is that this is not a + common problem. + +* Previously, Connect allowed request objects with matching shapes to be passed + to API calls interchangeably as long as the passed object was a superset of the + target type. For example, given the following proto definitions: + + ```protobuf + syntax = "proto3"; + package example.v1; + message MessageA { + string field_a = 1; + } + message MessageB { + string field_a = 1; + int64 field_b = 2; + } + service ExampleService { + rpc RequestA(MessageA) returns (Empty) {} + rpc RequestB(MessageB) returns (Empty) {} + } + ``` + + The following would have passed TypeScript compilation: + + ```ts + client.requestA(new MessageA()); + client.requestA(new MessageB()); + ``` + + This was an unintended bug and not a feature. In Connect v2, only the specified + target type will pass compilation. + +- We have removed the `credentials` option from transports as well as the `init` + option in interceptors. These two options were used to customize `fetch` routines. + Users should now rely on the `fetch` option in transports as a way to perform + these customizations. For example: + + ```ts + createConnectTransport({ + baseUrl: "/", + fetch: (input, init) => fetch(input, { ...init, credentials: "include" }), + }); + ``` + + In addition, as a replacement to determine whether an incoming request is a + Connect GET request in server-side interceptors, the property `requestMethod: string` + has been added to intercepted requests. This property is symmetrical to `HandlerContext.requestMethod`. + +* Errors details are now a pair of desc and init values. In Connect v1, error + details were specified as message instances. In v2, error + details are now an object that specifies both a schema and initialization + object which are both passed to the `create` function of Protobuf-ES. For example: + + ```diff + - import { LocalizedMessage } from "./gen/google/rpc/error_details_pb"; + - const details = [ + - new LocalizedMessage({ + - locale: "fr-CH", + - message: "Je n'ai plus de mots.", + - }), + - ]; + + import { LocalizedMessageSchema } from "./gen/google/rpc/error_details_pb"; + + const details = [ + + { + + desc: LocalizedMessageSchema, + + value: { + + locale: "fr-CH", + + message: "Je n'ai plus de mots.", + + } + + }, + + ]; + + const metadata = new Headers({ + "words-left": "none" + }); + throw new ConnectError( + "I have no words anymore.", + Code.ResourceExhausted, + metadata, + details + ); + ``` + +* `MethodDescriptor` is now self-sufficient. In v1, method descriptors always + had to be used alongside the service descriptors. In v2, they can now be used + standalone. This means that all callsites that previously passed a service + and method descriptor should now just pass the method descriptor. + + A notable example is when using a Connect `Router` on the server-side: + + ```diff + const routes = ({rpc}: Router) => { + - rpc(ElizaService, ElizaService.say, impl); + + rpc(ElizaService.say, impl); + } + ``` + +* Interceptors for streaming RPCs now use appropriate stream types. In + v1, the server used more exact types in interceptors, for example `UnaryRequest` + for server-streaming rpcs while the client always used streaming variants. + This was unintended behavior and has been fixed in v2. Now all streaming RPCs + use the `StreamRequest` and `StreamResponse` types on the server as well. + +* Node 16 is no longer supported. Connect v2 now supports Node versions 18.14.1 and up. + +* Connect v2 now requires at least TypeScript v4.9.5. diff --git a/README.md b/README.md index 0f2d8f8f..32e10e71 100644 --- a/README.md +++ b/README.md @@ -70,165 +70,7 @@ the builtin `http` and `http2` modules. ## Migrating from version 1 -Version 2 provides new features and simplifies some common APIs. In addition, -it makes use of all the enhancements of Protobuf-ES v2. To upgrade, you'll need -to update your dependencies, re-generate code, and update callsites in your application. - -The first thing to be aware of is that the plugin `protoc-gen-connect-es` has been -removed in v2. Connect now relies on descriptors generated by Protobuf-ES v2. -When upgrading, you will first need to remove any usage of this plugin. For most -users, this means a combination of any of the following steps: - -- Remove `protoc-gen-connect-es` from `package.json` either manually or - via a package manager, i.e. `npm uninstall @connectrpc/protoc-gen-connect-es`. - -- Remove any usage of the plugin in `buf.gen.yaml`. - -- Remove any dependency on Generated SDKs for Connect from `package.json` using - the same approach used for removing the plugin. Users should then rely on - types from the `@bufbuild_es` plugin instead. This will also necessitate a code - change, which is usually a simple import path update from `*_connect` to `*_pb`. - -Next, as mentioned, Connect uses Protobuf-ES v2, so you will need to upgrade any -dependencies you have on its relevant packages: - -```shellsession -npm install @bufbuild/protobuf@^2.0.0 @bufbuild/protoc-gen-es@^2.0.0 -``` - -Finally, update the Connect [packages](#packages) you use with your package -manager of choice. For example, with NPM: - -```shellsession -npm install @connectrpc/connect@^2.0.0 @connectrpc/connect-web@^2.0.0 -``` - -### Update your code - -Next, you'll need to account for any breaking changes. Following is a list to -be aware of: - -- The gRPC Transport now requires HTTP/2. If you are using `createGrpcTransport` - and specifying an `httpVersion`, it will fail compilation. Remove the - `httpVersion` property to use the default of HTTP/2. - - Note that if you were relying on HTTP/1.1 as part of your gRPC strategy, this - may require bigger architectural changes, but the hope is that this is not a - common problem. - -- Promise clients are now the default and the previously-deprecated - `createPromiseClient` has been removed. Any callsites using `createPromiseClient` - should be updated to use `createClient`. - -- Node 16 is no longer supported. Connect v2 now supports Node versions 18.14.1 and up. - -- Previously, Connect allowed request objects with matching shapes to be passed - to API calls interchangeably as long as the passed object was a superset of the - target type. For example, given the following proto definitions: - - ```protobuf - syntax = "proto3"; - package example.v1; - message MessageA { - string field_a = 1; - } - message MessageB { - string field_a = 1; - int64 field_b = 2; - } - service ExampleService { - rpc RequestA(MessageA) returns (Empty) {} - rpc RequestB(MessageB) returns (Empty) {} - } - ``` - - The following would have passed TypeScript compilation: - - ```ts - client.requestA(new MessageA()); - client.requestA(new MessageB()); - ``` - - This was an unintended bug and not a feature. In Connect v2, only the specified - target type will pass compilation. - -- We have removed the `credentials` option from transports as well as the `init` - option in interceptors. These two options were used to customize `fetch` routines. - Users should now rely on the `fetch` option in transports as a way to perform - these customizations. For example: - - ```ts - createConnectTransport({ - baseUrl: "/", - fetch: (input, init) => fetch(input, { ...init, credentials: "include" }), - }); - ``` - - In addition, as a replacement to determine whether an incoming request is a - Connect GET request in server-side interceptors, the property `requestMethod: string` - has been added to intercepted requests. This property is symmetrical to `HandlerContext.requestMethod`. - -- The access pattern of `MethodDescriptor` has slightly changed. In v2, the - Protobuf-ES plugin generates service descriptors and changes the - access pattern for method descriptors. Instead of the plural form `methods`, - it now uses the singular `method`. - - For example: `ElizaService.methods.Say` now becomes `ElizaService.method.Say`. - -- Errors details are now a pair of desc and init values. In Connect v1, error - details were specified as message instances. In v2, error - details are now an object that specifies both a schema and initialization - object which are both passed to the `create` function of Protobuf-ES. For example: - - ```diff - - import { LocalizedMessage } from "./gen/google/rpc/error_details_pb"; - - const details = [ - - new LocalizedMessage({ - - locale: "fr-CH", - - message: "Je n'ai plus de mots.", - - }), - - ]; - + import { LocalizedMessageSchema } from "./gen/google/rpc/error_details_pb"; - + const details = [ - + { - + desc: LocalizedMessageSchema, - + value: { - + locale: "fr-CH", - + message: "Je n'ai plus de mots.", - + } - + }, - + ]; - - const metadata = new Headers({ - "words-left": "none" - }); - throw new ConnectError( - "I have no words anymore.", - Code.ResourceExhausted, - metadata, - details - ); - ``` - -- `MethodDescriptor` is now self-sufficient. In v1, method descriptors always - had to be used alongside the service descriptors. In v2, they can now be used - standalone. This means that all callsites that previously passed a service - and method descriptor can now just pass the method descriptor. - - A notable example is when using a Connect `Router` on the server-side: - - ```diff - const routes = ({rpc}: Router) => { - - rpc(ElizaService, ElizaService.say, impl); - + rpc(ElizaService.say, impl); - } - ``` - -- Interceptors for streaming RPCs now use appropriate stream types. In - v1, the server used more exact types in interceptors, for example `UnaryRequest` - for server-streaming rpcs while the client always used streaming variants. - This was unintended behavior and has been fixed in v2. Now all streaming RPCs - use the `StreamRequest` and `StreamResponse` types on the server as well. +If you are migrating from v1 to v2, check out our [migration guide](./MIGRATING.md); ## Other platforms From 82b8600248071ce86ec4a6c87131b75a6bc4a63b Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Tue, 12 Nov 2024 13:38:09 -0500 Subject: [PATCH 07/13] Round 2 Signed-off-by: Steve Ayers --- MIGRATING.md | 44 +++++++++++++++++++++----------------------- 1 file changed, 21 insertions(+), 23 deletions(-) diff --git a/MIGRATING.md b/MIGRATING.md index f0bf26c5..b68e75c1 100644 --- a/MIGRATING.md +++ b/MIGRATING.md @@ -1,12 +1,11 @@ -# Migration Guide - -## Migrating from version 1 to version 2 +# Connect v1 to v2 migration guide Connect v2 provides new features and simplifies some common APIs. In addition, -it makes use of all the enhancements of Protobuf-ES v2. This document walks you -through all you need to know. +it makes use of all the enhancements of Protobuf-ES v2. If you're currently +using Connect v1, this document walks you through all you need to know to migrate +and start using it right away. -### Migrating your code +## Running the migration tool To help with the process of migrating, we provide a tool called [@connectrpc/connect-migrate](https://www.npmjs.com/package/@connectrpc/connect-migrate) @@ -18,36 +17,35 @@ execute the following command: While the tool will do a lot of the dependency legwork for you, there are many use cases in code that it does not cover. Below, we list everything that has changed in -v2 and whether it is fixed by the migration tool or whether it will require -manual intervention. +v2. Changes that are handled by the migration tool are noted as such. In summary, the suggested way to migrate to Connect v2 is to first run the migration tool and then go through the list below, fixing any other instances that weren't handled. -### Updating dependencies +## What's changed + +### Dependency updates Perhaps the biggest thing to be aware of is that the plugin `protoc-gen-connect-es` has been removed in v2. Connect now relies on service descriptors generated by the Protobuf-ES v2 `protoc-gen-es` and no longer generates code itself. So the first major part of the migration path is removing any usage of the old -`protoc-gen-connect-es` plugin. This means a combination of any -of the following: +`protoc-gen-connect-es` plugin. This could include: - Removing `protoc-gen-connect-es` from `package.json`. - - Removing any usage of the plugin in `buf.gen.yaml`. - - Removing any dependency on Generated SDKs for Connect from `package.json`. -Next, upgrade any dependencies on [Protobuf-ES packages](https://github.com/bufbuild/protobuf-es?tab=readme-ov-file#packages) +The next step is to upgrade any dependencies on [Protobuf-ES packages](https://github.com/bufbuild/protobuf-es?tab=readme-ov-file#packages) to use v2. -Finally, update the Connect [packages](#packages) in your project to use v2. +The final step is to update the Connect [packages](#packages) in your project +to use v2. :white_check_mark: The `connect-migrate` tool will handle all of these changes. -### Generate code +### Generated code The next step is to re-generate code. The migration tool does not handle code generation, so be sure to do so in whatever way your project is configured. For @@ -71,7 +69,7 @@ introduced in v1.36.0. See the docs [here](https://buf.build/docs/configuration/ for specifying `clean` in your config files. Note that you can also specify `--clean` to any `buf generate` CLI commands you are running. -### Update import paths +### Import paths Once your code is generated and the vestigial `*_connect` files are removed, import paths should be updated. This is usually an update from `*_connect` to @@ -84,8 +82,7 @@ import paths should be updated. This is usually an update from `*_connect` to :white_check_mark: The `connect-migrate` tool will handle this. - -### Update usages with generated code +### Working with generated code As mentioned, Connect now solely uses the `protoc-gen-es` plugin provided by Protobuf-ES v2. As a result, working with generated code has changed in some @@ -114,8 +111,7 @@ Connect now relies on service descriptors generated by the Protobuf-ES v2 the service descriptors previously generated by Connect v1. The same basic information i.e. (typed metadata) is still generated, but it's generated by `protoc-gen-es` now, along with additional information available. For example, -the generated code in v2 now provides the full descriptor, with -features such as [custom options](https://github.com/bufbuild/protobuf-es/blob/v2.2.2/MANUAL.md#custom-options), +v2 now provides the full descriptor, with features such as [custom options](https://github.com/bufbuild/protobuf-es/blob/v2.2.2/MANUAL.md#custom-options), which can be very useful in interceptors. One breaking change to be aware with service descriptors is that the access @@ -159,6 +155,8 @@ dot notation is used. However, a few noteworthy changes to be aware of: a property `$typeName`, which is a simple string with the full name of the message like `"example.User"`. This property makes sure you don't pass the wrong message to a function by accident. +* The `PartialMessage` type has been mostly replaced by `MessageInitShape`, which + extracts the init type from a message descriptor. * A message field using [`google.protobuf.Struct`](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md#googleprotobufstruct) is now generated as more-convenient `JsonObject`. * `proto2` fields with default values are no longer generated as optional @@ -167,8 +165,8 @@ dot notation is used. However, a few noteworthy changes to be aware of: `@bufbuild/protobuf/wkt`. For example, when converting a `google.protobuf.Timestamp` to an ECMAScript Date object, it is no longer possible via a method. -For more information on all Protobuf-ES v2 changes, see the -[Protobuf-ES v2 migration guide](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md#update-your-code). +For more information on all Protobuf-ES v2 changes, see the Protobuf-ES [manual](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md) +as well as the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md#update-your-code). ### Other breaking changes From 7ec6a2fb747bb38cea9a18522c036e5e7ca38276 Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Tue, 12 Nov 2024 13:41:37 -0500 Subject: [PATCH 08/13] Round 2 Signed-off-by: Steve Ayers --- MIGRATING.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/MIGRATING.md b/MIGRATING.md index b68e75c1..7e81519d 100644 --- a/MIGRATING.md +++ b/MIGRATING.md @@ -37,10 +37,9 @@ So the first major part of the migration path is removing any usage of the old - Removing any usage of the plugin in `buf.gen.yaml`. - Removing any dependency on Generated SDKs for Connect from `package.json`. -The next step is to upgrade any dependencies on [Protobuf-ES packages](https://github.com/bufbuild/protobuf-es?tab=readme-ov-file#packages) -to use v2. +The next step is to upgrade any dependencies on [Protobuf-ES packages](https://github.com/bufbuild/protobuf-es?tab=readme-ov-file#packages) to use v2. -The final step is to update the Connect [packages](#packages) in your project +The final step is to update the [Connect packages](#packages) in your project to use v2. :white_check_mark: The `connect-migrate` tool will handle all of these changes. @@ -72,7 +71,7 @@ to any `buf generate` CLI commands you are running. ### Import paths Once your code is generated and the vestigial `*_connect` files are removed, -import paths should be updated. This is usually an update from `*_connect` to +import paths will need to be updated. This is usually an update from `*_connect` to `*_pb`: ```diff @@ -95,11 +94,11 @@ Protobuf-ES and pass the generated schema: ```diff - import { SayRequest } from "./gen/eliza_connect.js"; -- import { SayRequestSchema } from "./gen/eliza_pb.js"; ++ import { SayRequestSchema } from "./gen/eliza_pb.js"; + import { create } from "@bufbuild/protobuf"; - const sayRequest = new SayRequest({ -+ const sayRequest = create(UserSchema, { ++ const sayRequest = create(SayRequestSchema, { sentence: "Hello", }); ``` From 1e1f03f78d6f5718b63faad6ac11e315a7f97fd0 Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Tue, 12 Nov 2024 13:48:00 -0500 Subject: [PATCH 09/13] Round 2 Signed-off-by: Steve Ayers --- MIGRATING.md | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/MIGRATING.md b/MIGRATING.md index 7e81519d..75e95399 100644 --- a/MIGRATING.md +++ b/MIGRATING.md @@ -140,15 +140,18 @@ ElizaService.method.say.methodKind; // "unary" ElizaService.method.say.idempotency; // MethodOptions_IdempotencyLevel.NoSideEffects ``` -Notice that instead of the plural form `methods`, it now uses the singular -`method`. +Note that: +* instead of the plural form `methods`, it now uses the singular `method`. +* the property `kind` for `MethodKind` has been renamed to `methodKind` and it now + returns a string. +* the enum type of the `idempotency` property has slightly changed. Other than the above, the generated properties remain largely unchanged and request and responses do not need any updates when initializers are used or dot notation is used. However, a few noteworthy changes to be aware of: * The `toPlainMessage` function and the `PlainMessage` type are no longer - necessary. If you create a proto3 message with `create(UserSchema)`, the + necessary. If you create a `proto3` message with `create(UserSchema)`, the returned object is already a plain object. You can replace the `PlainMessage` types with `User`. The only difference is that `User` has a property `$typeName`, which is a simple string with the full name of @@ -157,12 +160,13 @@ dot notation is used. However, a few noteworthy changes to be aware of: * The `PartialMessage` type has been mostly replaced by `MessageInitShape`, which extracts the init type from a message descriptor. * A message field using [`google.protobuf.Struct`](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md#googleprotobufstruct) - is now generated as more-convenient `JsonObject`. + is now generated as a more-convenient `JsonObject`. * `proto2` fields with default values are no longer generated as optional properties. * All well-known types and helpers have been moved to an export under `@bufbuild/protobuf/wkt`. For example, when converting a `google.protobuf.Timestamp` - to an ECMAScript Date object, it is no longer possible via a method. + to an ECMAScript `Date` object, it is no longer possible via a method and must + be done via helpers imported from this path. For more information on all Protobuf-ES v2 changes, see the Protobuf-ES [manual](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md) as well as the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md#update-your-code). @@ -284,6 +288,6 @@ as well as the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/prot This was unintended behavior and has been fixed in v2. Now all streaming RPCs use the `StreamRequest` and `StreamResponse` types on the server as well. -* Node 16 is no longer supported. Connect v2 now supports Node versions 18.14.1 and up. +* Node 16 is no longer supported. Connect v2 now supports Node versions **18.14.1** and up. -* Connect v2 now requires at least TypeScript v4.9.5. +* Connect v2 now requires at least TypeScript **v4.9.5**. From 5e388c6f1faf429e1ae80925280230c60a418443 Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Tue, 12 Nov 2024 13:51:36 -0500 Subject: [PATCH 10/13] Format Signed-off-by: Steve Ayers --- MIGRATING.md | 95 ++++++++++++++++++++++++++-------------------------- 1 file changed, 48 insertions(+), 47 deletions(-) diff --git a/MIGRATING.md b/MIGRATING.md index 75e95399..c4141693 100644 --- a/MIGRATING.md +++ b/MIGRATING.md @@ -1,22 +1,22 @@ # Connect v1 to v2 migration guide Connect v2 provides new features and simplifies some common APIs. In addition, -it makes use of all the enhancements of Protobuf-ES v2. If you're currently +it makes use of all the enhancements of Protobuf-ES v2. If you're currently using Connect v1, this document walks you through all you need to know to migrate and start using it right away. ## Running the migration tool -To help with the process of migrating, we provide a tool called -[@connectrpc/connect-migrate](https://www.npmjs.com/package/@connectrpc/connect-migrate) +To help with the process of migrating, we provide a tool called +[@connectrpc/connect-migrate](https://www.npmjs.com/package/@connectrpc/connect-migrate) which will take care of dependency and plugin updates as well as a few minor code -changes. So, as a first step, you will need to run the migration tool. To do so, +changes. So, as a first step, you will need to run the migration tool. To do so, execute the following command: `npx @connectrpc/connect-migrate@v2.0.0` While the tool will do a lot of the dependency legwork for you, there are many -use cases in code that it does not cover. Below, we list everything that has changed in +use cases in code that it does not cover. Below, we list everything that has changed in v2. Changes that are handled by the migration tool are noted as such. In summary, the suggested way to migrate to Connect v2 is to first run the migration tool @@ -30,7 +30,7 @@ Perhaps the biggest thing to be aware of is that the plugin `protoc-gen-connect- removed in v2. Connect now relies on service descriptors generated by the Protobuf-ES v2 `protoc-gen-es` and no longer generates code itself. -So the first major part of the migration path is removing any usage of the old +So the first major part of the migration path is removing any usage of the old `protoc-gen-connect-es` plugin. This could include: - Removing `protoc-gen-connect-es` from `package.json`. @@ -39,7 +39,7 @@ So the first major part of the migration path is removing any usage of the old The next step is to upgrade any dependencies on [Protobuf-ES packages](https://github.com/bufbuild/protobuf-es?tab=readme-ov-file#packages) to use v2. -The final step is to update the [Connect packages](#packages) in your project +The final step is to update the [Connect packages](#packages) in your project to use v2. :white_check_mark: The `connect-migrate` tool will handle all of these changes. @@ -53,30 +53,30 @@ example, `npx buf generate`, etc. There are a few things to be aware with the switch to the Protobuf-ES v2 plugin as some of the options for that plugin have changed with its v2 release: -* The `import_extension` option is now set to `none` by default, which means it doesn't - add a `.js` extension to import paths. If you use the plugin option - `import_extension=none`, you can delete it. If you require imports to have the +- The `import_extension` option is now set to `none` by default, which means it doesn't + add a `.js` extension to import paths. If you use the plugin option + `import_extension=none`, you can delete it. If you require imports to have the `.js` extension, use `import_extension=js`. -* The `ts_nocheck` option is now off by default. If you require a `// @ts-nocheck` +- The `ts_nocheck` option is now off by default. If you require a `// @ts-nocheck` annotation at the top of generated code, use the option `ts_nocheck=true`. For more information, see the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md#migrating-from-version-1). -When generating code, you should make sure to delete any now-unused `*_connect.ts` +When generating code, you should make sure to delete any now-unused `*_connect.ts` files. An easy way to do this is to use the `clean` option provided by the Buf CLI, -introduced in v1.36.0. See the docs [here](https://buf.build/docs/configuration/v2/buf-gen-yaml/?h=clean) +introduced in v1.36.0. See the docs [here](https://buf.build/docs/configuration/v2/buf-gen-yaml/?h=clean) for specifying `clean` in your config files. Note that you can also specify `--clean` to any `buf generate` CLI commands you are running. ### Import paths -Once your code is generated and the vestigial `*_connect` files are removed, +Once your code is generated and the vestigial `*_connect` files are removed, import paths will need to be updated. This is usually an update from `*_connect` to `*_pb`: ```diff - import { ElizaService } from "./gen/eliza_connect.js"; -+ import { ElizaService } from "./gen/eliza_pb.js"; ++ import { ElizaService } from "./gen/eliza_pb.js"; ``` :white_check_mark: The `connect-migrate` tool will handle this. @@ -87,7 +87,7 @@ As mentioned, Connect now solely uses the `protoc-gen-es` plugin provided by Protobuf-ES v2. As a result, working with generated code has changed in some areas. -#### Creating messages +#### Creating messages To create a new instance, you now call the function `create()` provided by Protobuf-ES and pass the generated schema: @@ -106,15 +106,15 @@ Protobuf-ES and pass the generated schema: #### Service descriptors Connect now relies on service descriptors generated by the Protobuf-ES v2 -`protoc-gen-es` plugin. The v2 service descriptors aren't much different from -the service descriptors previously generated by Connect v1. The same basic -information i.e. (typed metadata) is still generated, but it's generated by -`protoc-gen-es` now, along with additional information available. For example, -v2 now provides the full descriptor, with features such as [custom options](https://github.com/bufbuild/protobuf-es/blob/v2.2.2/MANUAL.md#custom-options), +`protoc-gen-es` plugin. The v2 service descriptors aren't much different from +the service descriptors previously generated by Connect v1. The same basic +information i.e. (typed metadata) is still generated, but it's generated by +`protoc-gen-es` now, along with additional information available. For example, +v2 now provides the full descriptor, with features such as [custom options](https://github.com/bufbuild/protobuf-es/blob/v2.2.2/MANUAL.md#custom-options), which can be very useful in interceptors. -One breaking change to be aware with service descriptors is that the access -pattern of the `MethodDescriptor` has slightly changed. +One breaking change to be aware with service descriptors is that the access +pattern of the `MethodDescriptor` has slightly changed. Service descriptors in Connect v1 looked like this: @@ -141,30 +141,31 @@ ElizaService.method.say.idempotency; // MethodOptions_IdempotencyLevel.NoSideEf ``` Note that: -* instead of the plural form `methods`, it now uses the singular `method`. -* the property `kind` for `MethodKind` has been renamed to `methodKind` and it now + +- instead of the plural form `methods`, it now uses the singular `method`. +- the property `kind` for `MethodKind` has been renamed to `methodKind` and it now returns a string. -* the enum type of the `idempotency` property has slightly changed. +- the enum type of the `idempotency` property has slightly changed. -Other than the above, the generated properties remain largely unchanged and -request and responses do not need any updates when initializers are used or +Other than the above, the generated properties remain largely unchanged and +request and responses do not need any updates when initializers are used or dot notation is used. However, a few noteworthy changes to be aware of: -* The `toPlainMessage` function and the `PlainMessage` type are no longer - necessary. If you create a `proto3` message with `create(UserSchema)`, the - returned object is already a plain object. You can replace the - `PlainMessage` types with `User`. The only difference is that `User` has +- The `toPlainMessage` function and the `PlainMessage` type are no longer + necessary. If you create a `proto3` message with `create(UserSchema)`, the + returned object is already a plain object. You can replace the + `PlainMessage` types with `User`. The only difference is that `User` has a property `$typeName`, which is a simple string with the full name of - the message like `"example.User"`. This property makes sure you don't pass the + the message like `"example.User"`. This property makes sure you don't pass the wrong message to a function by accident. -* The `PartialMessage` type has been mostly replaced by `MessageInitShape`, which +- The `PartialMessage` type has been mostly replaced by `MessageInitShape`, which extracts the init type from a message descriptor. -* A message field using [`google.protobuf.Struct`](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md#googleprotobufstruct) +- A message field using [`google.protobuf.Struct`](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md#googleprotobufstruct) is now generated as a more-convenient `JsonObject`. -* `proto2` fields with default values are no longer generated as optional +- `proto2` fields with default values are no longer generated as optional properties. -* All well-known types and helpers have been moved to an export under - `@bufbuild/protobuf/wkt`. For example, when converting a `google.protobuf.Timestamp` +- All well-known types and helpers have been moved to an export under + `@bufbuild/protobuf/wkt`. For example, when converting a `google.protobuf.Timestamp` to an ECMAScript `Date` object, it is no longer possible via a method and must be done via helpers imported from this path. @@ -173,13 +174,13 @@ as well as the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/prot ### Other breaking changes -* Promise clients are now the default and the previously-deprecated +- Promise clients are now the default and the previously-deprecated `createPromiseClient` has been removed. Any callsites using `createPromiseClient` should be updated to use `createClient`. :white_check_mark: The `connect-migrate` tool will handle this. -* The gRPC Transport now requires HTTP/2. If you are using `createGrpcTransport` +- The gRPC Transport now requires HTTP/2. If you are using `createGrpcTransport` and specifying an `httpVersion`, it will fail compilation. Remove the `httpVersion` property to use the default of HTTP/2. @@ -187,7 +188,7 @@ as well as the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/prot may require bigger architectural changes, but the hope is that this is not a common problem. -* Previously, Connect allowed request objects with matching shapes to be passed +- Previously, Connect allowed request objects with matching shapes to be passed to API calls interchangeably as long as the passed object was a superset of the target type. For example, given the following proto definitions: @@ -217,7 +218,7 @@ as well as the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/prot This was an unintended bug and not a feature. In Connect v2, only the specified target type will pass compilation. -- We have removed the `credentials` option from transports as well as the `init` +* We have removed the `credentials` option from transports as well as the `init` option in interceptors. These two options were used to customize `fetch` routines. Users should now rely on the `fetch` option in transports as a way to perform these customizations. For example: @@ -233,7 +234,7 @@ as well as the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/prot Connect GET request in server-side interceptors, the property `requestMethod: string` has been added to intercepted requests. This property is symmetrical to `HandlerContext.requestMethod`. -* Errors details are now a pair of desc and init values. In Connect v1, error +- Errors details are now a pair of desc and init values. In Connect v1, error details were specified as message instances. In v2, error details are now an object that specifies both a schema and initialization object which are both passed to the `create` function of Protobuf-ES. For example: @@ -268,7 +269,7 @@ as well as the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/prot ); ``` -* `MethodDescriptor` is now self-sufficient. In v1, method descriptors always +- `MethodDescriptor` is now self-sufficient. In v1, method descriptors always had to be used alongside the service descriptors. In v2, they can now be used standalone. This means that all callsites that previously passed a service and method descriptor should now just pass the method descriptor. @@ -282,12 +283,12 @@ as well as the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/prot } ``` -* Interceptors for streaming RPCs now use appropriate stream types. In +- Interceptors for streaming RPCs now use appropriate stream types. In v1, the server used more exact types in interceptors, for example `UnaryRequest` for server-streaming rpcs while the client always used streaming variants. This was unintended behavior and has been fixed in v2. Now all streaming RPCs use the `StreamRequest` and `StreamResponse` types on the server as well. -* Node 16 is no longer supported. Connect v2 now supports Node versions **18.14.1** and up. +- Node 16 is no longer supported. Connect v2 now supports Node versions **18.14.1** and up. -* Connect v2 now requires at least TypeScript **v4.9.5**. +- Connect v2 now requires at least TypeScript **v4.9.5**. From 1c9994bf5287196d02a714ff65886ce0651b1cdd Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Wed, 13 Nov 2024 11:45:05 -0500 Subject: [PATCH 11/13] Feedback Signed-off-by: Steve Ayers --- MIGRATING.md | 332 +++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 257 insertions(+), 75 deletions(-) diff --git a/MIGRATING.md b/MIGRATING.md index c4141693..61b68ddd 100644 --- a/MIGRATING.md +++ b/MIGRATING.md @@ -1,22 +1,24 @@ # Connect v1 to v2 migration guide Connect v2 provides new features and simplifies some common APIs. In addition, -it makes use of all the enhancements of Protobuf-ES v2. If you're currently +it makes use of all the enhancements of Protobuf-ES v2. If you're currently using Connect v1, this document walks you through all you need to know to migrate and start using it right away. ## Running the migration tool -To help with the process of migrating, we provide a tool called -[@connectrpc/connect-migrate](https://www.npmjs.com/package/@connectrpc/connect-migrate) +To help with the process of migrating, we provide a tool called +[@connectrpc/connect-migrate](https://www.npmjs.com/package/@connectrpc/connect-migrate) which will take care of dependency and plugin updates as well as a few minor code -changes. So, as a first step, you will need to run the migration tool. To do so, +changes. So, as a first step, you will need to run the migration tool. To do so, execute the following command: -`npx @connectrpc/connect-migrate@v2.0.0` +```shellsession +npx @connectrpc/connect-migrate@latest +``` While the tool will do a lot of the dependency legwork for you, there are many -use cases in code that it does not cover. Below, we list everything that has changed in +use cases in code that it does not cover. Below, we list everything that has changed in v2. Changes that are handled by the migration tool are noted as such. In summary, the suggested way to migrate to Connect v2 is to first run the migration tool @@ -26,24 +28,124 @@ and then go through the list below, fixing any other instances that weren't hand ### Dependency updates -Perhaps the biggest thing to be aware of is that the plugin `protoc-gen-connect-es` has been -removed in v2. Connect now relies on service descriptors generated by the Protobuf-ES v2 -`protoc-gen-es` and no longer generates code itself. +The first step in migration is updating all dependencies. This includes the +Protobuf-ES and Connect packages in `package.json`, plugin references in +`buf.gen.yaml`, and any usage of generated SDKs in your `package.json`. + +#### Protobuf-ES and Connect packages + +One important dependency change to be aware of is that the plugin +`protoc-gen-connect-es` has been removed in v2. Connect now relies on service +descriptors generated by the Protobuf-ES v2 `protoc-gen-es` and no longer +generates code itself. Therefore, that dependency should be removed entirely +from `package.json`. Your mileage may vary according to what `@bufbuild` +and `@connectrpc` packages you depend on, but a list of relevant, compatible +dependencies should look similar to the following: + +```diff + "dependencies": { + "@bufbuild/protobuf": "^2.2.0", + "@bufbuild/protoc-gen-es": "^2.2.0", + "@connectrpc/connect": "^2.0.0", +- "@connectrpc/protoc-gen-connect-es": "^1.0.0", + "@connectrpc/connect-web": "^2.0.0", + "@connectrpc/connect-node": "^2.0.0", + "@connectrpc/connect-next": "^2.0.0", + "@connectrpc/connect-fastify": "^2.0.0", + "@connectrpc/connect-express": "^2.0.0", + "@connectrpc/connect-query": "^2.0.0", + "@connectrpc/connect-playwright": "^0.6.0" + } +``` + +:white_check_mark: The `connect-migrate` tool will handle this. + +#### buf.gen.yaml -So the first major part of the migration path is removing any usage of the old -`protoc-gen-connect-es` plugin. This could include: +Next, any usage of the `protoc-gen-connect-es` plugin should be removed from +`buf.gen.yaml`. -- Removing `protoc-gen-connect-es` from `package.json`. -- Removing any usage of the plugin in `buf.gen.yaml`. -- Removing any dependency on Generated SDKs for Connect from `package.json`. +If you are using local plugins: -The next step is to upgrade any dependencies on [Protobuf-ES packages](https://github.com/bufbuild/protobuf-es?tab=readme-ov-file#packages) to use v2. +```diff +# buf.gen.yaml +version: v2 +plugins: + - local: protoc-gen-es + out: src/gen + opt: target=ts +- - local: protoc-gen-connect-es +- out: src/gen +- opt: target=ts +``` + +If you are using remote plugins: -The final step is to update the [Connect packages](#packages) in your project -to use v2. +```diff +# buf.gen.yaml +version: v2 +plugins: +- - remote: buf.build/bufbuild/es:v1.10.0 ++ - remote: buf.build/bufbuild/es:v2.2.0 + out: src/gen + opt: target=ts +- - remote: buf.build/connectrpc/es +- out: src/gen +- opt: target=ts +``` :white_check_mark: The `connect-migrate` tool will handle all of these changes. +#### Generated SDKs + +The migration tool does not handle updating generated SDKs so these types of +dependencies will need updated manually. + +Since the Connect plugin no longer exists in v2, any generated +SDK dependencies in your `package.json` that rely on this plugin (i.e. have +`connectrpc_es` as part of their path) should be updated to use the Protobuf-ES +v2 plugin instead. + +The path for a generated SDK dependency is structured as follows: + +``` +@buf/{module_owner}/{module_name}/{plugin_owner}/{plugin_name@{version} +``` + +We will walk through updating a generated SDK on the `googleapis/googleapis` +module. To do this for other modules, simply replace `googleapis/googleapis` +with your module owner and name (also note that your versions will differ per +module). + +A dependency on `googleapis/googleapis` that uses the Connect v1 plugin will +look something like the following (note the `connectrpc_es` in the path): + +``` +@buf/googleapis_googleapis.connectrpc_es@1.4.0-20241107203341-553fd4b4b3a6.3 +``` + +Remove this line from your `package.json`. + +To use the corresponding generated SDK with Protobuf-ES v2, use the +command: + +```shellsession +npm install @buf/googleapis_googleapis.bufbuild_es@latest +``` + +Your `package.json` should now resemble the following: + +```diff + "dependencies": { + ... +- "@buf/googleapis_googleapis.connectrpc_es": "^1.6.1-20241107203341-553fd4b4b3a6.1", ++ "@buf/googleapis_googleapis.bufbuild_es": "^2.2.2-20241107203341-553fd4b4b3a6.1", + "@connectrpc/connect-web": "^2.0.0", + "@bufbuild/protobuf": "^2.2.0", + ... + } +``` + ### Generated code The next step is to re-generate code. The migration tool does not handle code @@ -53,44 +155,82 @@ example, `npx buf generate`, etc. There are a few things to be aware with the switch to the Protobuf-ES v2 plugin as some of the options for that plugin have changed with its v2 release: -- The `import_extension` option is now set to `none` by default, which means it doesn't - add a `.js` extension to import paths. If you use the plugin option - `import_extension=none`, you can delete it. If you require imports to have the - `.js` extension, use `import_extension=js`. -- The `ts_nocheck` option is now off by default. If you require a `// @ts-nocheck` - annotation at the top of generated code, use the option `ts_nocheck=true`. +#### Editing `buf.gen.yaml` -For more information, see the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md#migrating-from-version-1). +If you are using [Node16 module resolution](https://www.typescriptlang.org/tsconfig/#moduleResolution) +and need the `.js` extension on all import paths, add the plugin option `import_extension=js`: + +```diff +# buf.gen.yaml +version: v2 +plugins: + - local: protoc-gen-es + out: src/gen + opt: + - target=ts ++ - import_extension=js +``` -When generating code, you should make sure to delete any now-unused `*_connect.ts` -files. An easy way to do this is to use the `clean` option provided by the Buf CLI, -introduced in v1.36.0. See the docs [here](https://buf.build/docs/configuration/v2/buf-gen-yaml/?h=clean) -for specifying `clean` in your config files. Note that you can also specify `--clean` -to any `buf generate` CLI commands you are running. +If you don't want the `.js` extension added to import paths, you can remove the +plugin option `import_extension=none` - it's the default behavior now: + +```diff +# buf.gen.yaml +version: v2 +plugins: + - local: protoc-gen-es + out: src/gen + opt: + - target=ts +- - import_extension=none +``` + +If you have been using the plugin option `ts_nocheck=false`, you can remove it +as well - it's the default behavior now. + +#### Removing unused files + +When generating code, you should make sure to delete any now-unused `*_connect.ts` +files. An easy way to do this is to use the `clean` option provided by the +Buf CLI introduced in v1.36.0: + +```diff +# buf.gen.yaml +version: v2 ++ clean: true +plugins: + - local: protoc-gen-es + out: src/gen +``` + +With this option, `buf generate` will delete the contents of `src/gen` before +generating code. ### Import paths -Once your code is generated and the vestigial `*_connect` files are removed, +Once your code is generated and the vestigial `*_connect` files are removed, import paths will need to be updated. This is usually an update from `*_connect` to `*_pb`: ```diff - import { ElizaService } from "./gen/eliza_connect.js"; -+ import { ElizaService } from "./gen/eliza_pb.js"; ++ import { ElizaService } from "./gen/eliza_pb.js"; ``` :white_check_mark: The `connect-migrate` tool will handle this. +For more information, see the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md#migrating-from-version-1). + ### Working with generated code As mentioned, Connect now solely uses the `protoc-gen-es` plugin provided by Protobuf-ES v2. As a result, working with generated code has changed in some areas. -#### Creating messages +#### Creating messages -To create a new instance, you now call the function `create()` provided by -Protobuf-ES and pass the generated schema: +To create a new instance, you now call the function `create()` from +`@bufbuild/protobuf` and pass the generated schema: ```diff - import { SayRequest } from "./gen/eliza_connect.js"; @@ -103,18 +243,72 @@ Protobuf-ES and pass the generated schema: }); ``` +#### Working with messages + +When working with messages, there a few noteworthy changes to be aware of: + +* The `toPlainMessage` function and the `PlainMessage` type are no longer + necessary. If you create a `proto3` message with `create(UserSchema)`, the + returned object is already a plain object. You can replace the + `PlainMessage` types with `User`. The only difference is that `User` has + a property `$typeName`, which is a simple string with the full name of + the message like `"example.User"`. This property makes sure you don't pass the + wrong message to a function by accident. +* The `PartialMessage` type has been mostly replaced by `MessageInitShape`, which + extracts the init type from a message descriptor. +* A message field using [`google.protobuf.Struct`](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md#googleprotobufstruct) + is now generated as a more-convenient `JsonObject`. +* `proto2` fields with default values are no longer generated as optional + properties. +* All well-known types and helpers have been moved to an export under + `@bufbuild/protobuf/wkt`. For example, when converting a `google.protobuf.Timestamp` + to an ECMAScript `Date` object, it is no longer possible via a method and must + be done via helpers imported from this path. +* JSON serialization options (passed to a transport or a server plugin) have + been updated in Protobuf-ES v2. The option `emitDefaultValues` is renamed: + + ```diff + const transport = createConnectTransport({ + baseUrl: "https://demo.connectrpc.com", + jsonOptions: { + - emitDefaultValues: true, + + alwaysEmitImplicit: true, + }, + }); + ``` +* The JSON serialization option `typeRegistry` has also been renamed: + + ```diff + import { createRegistry } from "@bufbuild/protobuf"; + import { createConnectTransport } from "@connectrpc/connect-web"; + - import { SayRequest } from "./gen/eliza_pb"; + + import { SayRequestSchema } from "./gen/eliza_pb"; + + const transport = createConnectTransport({ + baseUrl: "https://demo.connectrpc.com", + jsonOptions: { + - typeRegistry: createRegistry(SayRequest), + - registry: createRegistry(SayRequestSchema), + }, + }); + ``` + + Note that registries have received a major update in Protobuf-ES v2 and are + much more capable and flexible now. For more information, see the + [Protobuf registry documentation](https://github.com/bufbuild/protobuf-es/blob/v2.2.2/MANUAL.md#registries). + #### Service descriptors Connect now relies on service descriptors generated by the Protobuf-ES v2 -`protoc-gen-es` plugin. The v2 service descriptors aren't much different from -the service descriptors previously generated by Connect v1. The same basic -information i.e. (typed metadata) is still generated, but it's generated by -`protoc-gen-es` now, along with additional information available. For example, -v2 now provides the full descriptor, with features such as [custom options](https://github.com/bufbuild/protobuf-es/blob/v2.2.2/MANUAL.md#custom-options), +`protoc-gen-es` plugin. The v2 service descriptors aren't much different from +the service descriptors previously generated by Connect v1. The same basic +information i.e. (typed metadata) is still generated, but it's generated by +`protoc-gen-es` now, along with additional information available. For example, +v2 now provides the full descriptor, with features such as [custom options](https://github.com/bufbuild/protobuf-es/blob/v2.2.2/MANUAL.md#custom-options), which can be very useful in interceptors. -One breaking change to be aware with service descriptors is that the access -pattern of the `MethodDescriptor` has slightly changed. +One breaking change to be aware with service descriptors is that the access +pattern of the `MethodDescriptor` has slightly changed. Service descriptors in Connect v1 looked like this: @@ -141,46 +335,24 @@ ElizaService.method.say.idempotency; // MethodOptions_IdempotencyLevel.NoSideEf ``` Note that: - -- instead of the plural form `methods`, it now uses the singular `method`. -- the property `kind` for `MethodKind` has been renamed to `methodKind` and it now +* instead of the plural form `methods`, it now uses the singular `method`. +* the property `kind` for `MethodKind` has been renamed to `methodKind` and it now returns a string. -- the enum type of the `idempotency` property has slightly changed. +* the `idempotency` property is now a string union instead of an enum. -Other than the above, the generated properties remain largely unchanged and -request and responses do not need any updates when initializers are used or -dot notation is used. However, a few noteworthy changes to be aware of: - -- The `toPlainMessage` function and the `PlainMessage` type are no longer - necessary. If you create a `proto3` message with `create(UserSchema)`, the - returned object is already a plain object. You can replace the - `PlainMessage` types with `User`. The only difference is that `User` has - a property `$typeName`, which is a simple string with the full name of - the message like `"example.User"`. This property makes sure you don't pass the - wrong message to a function by accident. -- The `PartialMessage` type has been mostly replaced by `MessageInitShape`, which - extracts the init type from a message descriptor. -- A message field using [`google.protobuf.Struct`](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md#googleprotobufstruct) - is now generated as a more-convenient `JsonObject`. -- `proto2` fields with default values are no longer generated as optional - properties. -- All well-known types and helpers have been moved to an export under - `@bufbuild/protobuf/wkt`. For example, when converting a `google.protobuf.Timestamp` - to an ECMAScript `Date` object, it is no longer possible via a method and must - be done via helpers imported from this path. For more information on all Protobuf-ES v2 changes, see the Protobuf-ES [manual](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md) as well as the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md#update-your-code). ### Other breaking changes -- Promise clients are now the default and the previously-deprecated +* Promise clients are now the default and the previously-deprecated `createPromiseClient` has been removed. Any callsites using `createPromiseClient` should be updated to use `createClient`. :white_check_mark: The `connect-migrate` tool will handle this. -- The gRPC Transport now requires HTTP/2. If you are using `createGrpcTransport` +* The gRPC Transport now requires HTTP/2. If you are using `createGrpcTransport` and specifying an `httpVersion`, it will fail compilation. Remove the `httpVersion` property to use the default of HTTP/2. @@ -188,7 +360,7 @@ as well as the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/prot may require bigger architectural changes, but the hope is that this is not a common problem. -- Previously, Connect allowed request objects with matching shapes to be passed +* Previously, Connect allowed request objects with matching shapes to be passed to API calls interchangeably as long as the passed object was a superset of the target type. For example, given the following proto definitions: @@ -218,7 +390,17 @@ as well as the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/prot This was an unintended bug and not a feature. In Connect v2, only the specified target type will pass compilation. -* We have removed the `credentials` option from transports as well as the `init` + If you intend to pass a message as a different message with the same fields, + you can use object destructuring to drop the `$typeName`, and copy the rest + of the properties: + + ```ts + const messageA: MessageA; + const { $typeName: _, ...properties } = messageA; + const messageB = create(MessageBSchema, properties); + ``` + +- We have removed the `credentials` option from transports as well as the `init` option in interceptors. These two options were used to customize `fetch` routines. Users should now rely on the `fetch` option in transports as a way to perform these customizations. For example: @@ -234,7 +416,7 @@ as well as the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/prot Connect GET request in server-side interceptors, the property `requestMethod: string` has been added to intercepted requests. This property is symmetrical to `HandlerContext.requestMethod`. -- Errors details are now a pair of desc and init values. In Connect v1, error +* Errors details are now a pair of desc and init values. In Connect v1, error details were specified as message instances. In v2, error details are now an object that specifies both a schema and initialization object which are both passed to the `create` function of Protobuf-ES. For example: @@ -269,7 +451,7 @@ as well as the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/prot ); ``` -- `MethodDescriptor` is now self-sufficient. In v1, method descriptors always +* `MethodDescriptor` is now self-sufficient. In v1, method descriptors always had to be used alongside the service descriptors. In v2, they can now be used standalone. This means that all callsites that previously passed a service and method descriptor should now just pass the method descriptor. @@ -283,12 +465,12 @@ as well as the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/prot } ``` -- Interceptors for streaming RPCs now use appropriate stream types. In +* Interceptors for streaming RPCs now use appropriate stream types. In v1, the server used more exact types in interceptors, for example `UnaryRequest` for server-streaming rpcs while the client always used streaming variants. This was unintended behavior and has been fixed in v2. Now all streaming RPCs use the `StreamRequest` and `StreamResponse` types on the server as well. -- Node 16 is no longer supported. Connect v2 now supports Node versions **18.14.1** and up. +* Node 16 is no longer supported. Connect v2 now supports Node versions **18.14.1** and up. -- Connect v2 now requires at least TypeScript **v4.9.5**. +* Connect v2 now requires at least TypeScript **v4.9.5**. From cbe0af384f68114a25d77b9214bca86385b53838 Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Wed, 13 Nov 2024 11:58:32 -0500 Subject: [PATCH 12/13] Cleanup Signed-off-by: Steve Ayers --- MIGRATING.md | 194 +++++++++++++++++++++++++-------------------------- 1 file changed, 96 insertions(+), 98 deletions(-) diff --git a/MIGRATING.md b/MIGRATING.md index 61b68ddd..8bbfdaa8 100644 --- a/MIGRATING.md +++ b/MIGRATING.md @@ -1,16 +1,16 @@ # Connect v1 to v2 migration guide Connect v2 provides new features and simplifies some common APIs. In addition, -it makes use of all the enhancements of Protobuf-ES v2. If you're currently +it makes use of all the enhancements of Protobuf-ES v2. If you're currently using Connect v1, this document walks you through all you need to know to migrate and start using it right away. ## Running the migration tool -To help with the process of migrating, we provide a tool called -[@connectrpc/connect-migrate](https://www.npmjs.com/package/@connectrpc/connect-migrate) +To help with the process of migrating, we provide a tool called +[@connectrpc/connect-migrate](https://www.npmjs.com/package/@connectrpc/connect-migrate) which will take care of dependency and plugin updates as well as a few minor code -changes. So, as a first step, you will need to run the migration tool. To do so, +changes. So, as a first step, you will need to run the migration tool. To do so, execute the following command: ```shellsession @@ -18,7 +18,7 @@ npx @connectrpc/connect-migrate@latest ``` While the tool will do a lot of the dependency legwork for you, there are many -use cases in code that it does not cover. Below, we list everything that has changed in +use cases in code that it does not cover. Below, we list everything that has changed in v2. Changes that are handled by the migration tool are noted as such. In summary, the suggested way to migrate to Connect v2 is to first run the migration tool @@ -28,15 +28,15 @@ and then go through the list below, fixing any other instances that weren't hand ### Dependency updates -The first step in migration is updating all dependencies. This includes the -Protobuf-ES and Connect packages in `package.json`, plugin references in +The next step in migrating is updating all dependencies. This includes the +Protobuf-ES and Connect packages in `package.json`, plugin references in `buf.gen.yaml`, and any usage of generated SDKs in your `package.json`. #### Protobuf-ES and Connect packages -One important dependency change to be aware of is that the plugin -`protoc-gen-connect-es` has been removed in v2. Connect now relies on service -descriptors generated by the Protobuf-ES v2 `protoc-gen-es` and no longer +One important dependency change to be aware of is that the plugin +`protoc-gen-connect-es` has been removed in v2. Connect now relies on service +descriptors generated by the Protobuf-ES v2 plugin `protoc-gen-es` and no longer generates code itself. Therefore, that dependency should be removed entirely from `package.json`. Your mileage may vary according to what `@bufbuild` and `@connectrpc` packages you depend on, but a list of relevant, compatible @@ -62,7 +62,7 @@ dependencies should look similar to the following: #### buf.gen.yaml -Next, any usage of the `protoc-gen-connect-es` plugin should be removed from +Next, any usage of the `protoc-gen-connect-es` plugin should be removed from `buf.gen.yaml`. If you are using local plugins: @@ -94,39 +94,38 @@ plugins: - opt: target=ts ``` -:white_check_mark: The `connect-migrate` tool will handle all of these changes. +:white_check_mark: The `connect-migrate` tool will handle this. #### Generated SDKs The migration tool does not handle updating generated SDKs so these types of dependencies will need updated manually. -Since the Connect plugin no longer exists in v2, any generated -SDK dependencies in your `package.json` that rely on this plugin (i.e. have -`connectrpc_es` as part of their path) should be updated to use the Protobuf-ES +Since the Connect plugin no longer exists in v2, any generated +SDK dependencies in your `package.json` that rely on this plugin (i.e. have +`connectrpc_es` as part of their path) should be updated to use the Protobuf-ES v2 plugin instead. The path for a generated SDK dependency is structured as follows: ``` -@buf/{module_owner}/{module_name}/{plugin_owner}/{plugin_name@{version} +@buf/{module_owner}_{module_name}.{plugin_owner}_{plugin_name}@{version} ``` -We will walk through updating a generated SDK on the `googleapis/googleapis` +We will walk through updating a generated SDK on the `googleapis/googleapis` module. To do this for other modules, simply replace `googleapis/googleapis` with your module owner and name (also note that your versions will differ per module). A dependency on `googleapis/googleapis` that uses the Connect v1 plugin will -look something like the following (note the `connectrpc_es` in the path): +look something like the following (note the `connectrpc_es` in the path). Remove +this line from your `package.json`. ``` @buf/googleapis_googleapis.connectrpc_es@1.4.0-20241107203341-553fd4b4b3a6.3 ``` -Remove this line from your `package.json`. - -To use the corresponding generated SDK with Protobuf-ES v2, use the +To use the corresponding generated SDK with Protobuf-ES v2, use the command: ```shellsession @@ -148,9 +147,9 @@ Your `package.json` should now resemble the following: ### Generated code -The next step is to re-generate code. The migration tool does not handle code -generation, so be sure to do so in whatever way your project is configured. For -example, `npx buf generate`, etc. +Now that dependencies are updated, the next step is to re-generate code. The +migration tool does not handle code generation, so be sure to do so in whatever +way your project is configured. For example, `npx buf generate`, etc. There are a few things to be aware with the switch to the Protobuf-ES v2 plugin as some of the options for that plugin have changed with its v2 release: @@ -166,12 +165,12 @@ version: v2 plugins: - local: protoc-gen-es out: src/gen - opt: + opt: - target=ts + - import_extension=js ``` -If you don't want the `.js` extension added to import paths, you can remove the +If you don't want the `.js` extension added to import paths, you can remove the plugin option `import_extension=none` - it's the default behavior now: ```diff @@ -180,56 +179,54 @@ version: v2 plugins: - local: protoc-gen-es out: src/gen - opt: + opt: - target=ts - - import_extension=none ``` -If you have been using the plugin option `ts_nocheck=false`, you can remove it +If you have been using the plugin option `ts_nocheck=false`, you can remove it as well - it's the default behavior now. #### Removing unused files -When generating code, you should make sure to delete any now-unused `*_connect.ts` -files. An easy way to do this is to use the `clean` option provided by the +When generating code, you should make sure to delete any now-unused `*_connect.ts` +files. An easy way to do this is to use the `clean` option provided by the Buf CLI introduced in v1.36.0: ```diff -# buf.gen.yaml -version: v2 + # buf.gen.yaml + version: v2 + clean: true -plugins: - - local: protoc-gen-es - out: src/gen + plugins: + - local: protoc-gen-es + out: src/gen ``` -With this option, `buf generate` will delete the contents of `src/gen` before +With this option, `buf generate` will delete the contents of `src/gen` before generating code. ### Import paths -Once your code is generated and the vestigial `*_connect` files are removed, +Once your code is generated and the vestigial `*_connect` files are removed, import paths will need to be updated. This is usually an update from `*_connect` to `*_pb`: ```diff - import { ElizaService } from "./gen/eliza_connect.js"; -+ import { ElizaService } from "./gen/eliza_pb.js"; ++ import { ElizaService } from "./gen/eliza_pb.js"; ``` :white_check_mark: The `connect-migrate` tool will handle this. -For more information, see the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md#migrating-from-version-1). - ### Working with generated code -As mentioned, Connect now solely uses the `protoc-gen-es` plugin provided by +As mentioned, Connect now solely uses the `protoc-gen-es` plugin provided by Protobuf-ES v2. As a result, working with generated code has changed in some areas. -#### Creating messages +#### Creating messages -To create a new instance, you now call the function `create()` from +To create a new instance, you now call the function `create()` from `@bufbuild/protobuf` and pass the generated schema: ```diff @@ -245,27 +242,27 @@ To create a new instance, you now call the function `create()` from #### Working with messages -When working with messages, there a few noteworthy changes to be aware of: +When working with messages, there are a few noteworthy changes to be aware of: -* The `toPlainMessage` function and the `PlainMessage` type are no longer - necessary. If you create a `proto3` message with `create(UserSchema)`, the - returned object is already a plain object. You can replace the - `PlainMessage` types with `User`. The only difference is that `User` has +- The `toPlainMessage` function and the `PlainMessage` type are no longer + necessary. If you create a `proto3` message with `create(UserSchema)`, the + returned object is already a plain object. You can replace the + `PlainMessage` types with `User`. The only difference is that `User` has a property `$typeName`, which is a simple string with the full name of - the message like `"example.User"`. This property makes sure you don't pass the + the message like `"example.User"`. This property makes sure you don't pass the wrong message to a function by accident. -* The `PartialMessage` type has been mostly replaced by `MessageInitShape`, which +- The `PartialMessage` type has been mostly replaced by `MessageInitShape`, which extracts the init type from a message descriptor. -* A message field using [`google.protobuf.Struct`](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md#googleprotobufstruct) +- A message field using [`google.protobuf.Struct`](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md#googleprotobufstruct) is now generated as a more-convenient `JsonObject`. -* `proto2` fields with default values are no longer generated as optional +- `proto2` fields with default values are no longer generated as optional properties. -* All well-known types and helpers have been moved to an export under - `@bufbuild/protobuf/wkt`. For example, when converting a `google.protobuf.Timestamp` +- All well-known types and helpers have been moved to an export under + `@bufbuild/protobuf/wkt`. For example, when converting a `google.protobuf.Timestamp` to an ECMAScript `Date` object, it is no longer possible via a method and must be done via helpers imported from this path. -* JSON serialization options (passed to a transport or a server plugin) have - been updated in Protobuf-ES v2. The option `emitDefaultValues` is renamed: +- JSON serialization options (passed to a transport or a server plugin) have + been updated in Protobuf-ES v2. The option `emitDefaultValues` has been renamed: ```diff const transport = createConnectTransport({ @@ -276,39 +273,40 @@ When working with messages, there a few noteworthy changes to be aware of: }, }); ``` -* The JSON serialization option `typeRegistry` has also been renamed: - - ```diff - import { createRegistry } from "@bufbuild/protobuf"; - import { createConnectTransport } from "@connectrpc/connect-web"; - - import { SayRequest } from "./gen/eliza_pb"; - + import { SayRequestSchema } from "./gen/eliza_pb"; - - const transport = createConnectTransport({ - baseUrl: "https://demo.connectrpc.com", - jsonOptions: { - - typeRegistry: createRegistry(SayRequest), - - registry: createRegistry(SayRequestSchema), - }, - }); - ``` - - Note that registries have received a major update in Protobuf-ES v2 and are - much more capable and flexible now. For more information, see the - [Protobuf registry documentation](https://github.com/bufbuild/protobuf-es/blob/v2.2.2/MANUAL.md#registries). + +- The JSON serialization option `typeRegistry` has also been renamed: + + ```diff + import { createRegistry } from "@bufbuild/protobuf"; + import { createConnectTransport } from "@connectrpc/connect-web"; + - import { SayRequest } from "./gen/eliza_pb"; + + import { SayRequestSchema } from "./gen/eliza_pb"; + + const transport = createConnectTransport({ + baseUrl: "https://demo.connectrpc.com", + jsonOptions: { + - typeRegistry: createRegistry(SayRequest), + - registry: createRegistry(SayRequestSchema), + }, + }); + ``` + +Note that registries have received a major update in Protobuf-ES v2 and are +much more capable and flexible now. For more information, see the +[Protobuf registry documentation](https://github.com/bufbuild/protobuf-es/blob/v2.2.2/MANUAL.md#registries). #### Service descriptors Connect now relies on service descriptors generated by the Protobuf-ES v2 -`protoc-gen-es` plugin. The v2 service descriptors aren't much different from -the service descriptors previously generated by Connect v1. The same basic -information i.e. (typed metadata) is still generated, but it's generated by -`protoc-gen-es` now, along with additional information available. For example, -v2 now provides the full descriptor, with features such as [custom options](https://github.com/bufbuild/protobuf-es/blob/v2.2.2/MANUAL.md#custom-options), +`protoc-gen-es` plugin. The v2 service descriptors aren't much different from +the service descriptors previously generated by Connect v1. The same basic +information (i.e. typed metadata) is still generated, but it's generated by +`protoc-gen-es` now along with some additional information. For example, +v2 now provides the full descriptor with features such as [custom options](https://github.com/bufbuild/protobuf-es/blob/v2.2.2/MANUAL.md#custom-options), which can be very useful in interceptors. -One breaking change to be aware with service descriptors is that the access -pattern of the `MethodDescriptor` has slightly changed. +One breaking change to be aware with service descriptors is that the access +pattern of the `MethodDescriptor` has slightly changed. Service descriptors in Connect v1 looked like this: @@ -335,24 +333,24 @@ ElizaService.method.say.idempotency; // MethodOptions_IdempotencyLevel.NoSideEf ``` Note that: -* instead of the plural form `methods`, it now uses the singular `method`. -* the property `kind` for `MethodKind` has been renamed to `methodKind` and it now - returns a string. -* the `idempotency` property is now a string union instead of an enum. +- instead of the plural form `methods`, it now uses the singular `method`. +- the property `kind` for `MethodKind` has been renamed to `methodKind` and it now + returns a string. +- the `idempotency` property is now a string union instead of an enum. For more information on all Protobuf-ES v2 changes, see the Protobuf-ES [manual](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md) as well as the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/protobuf-es/blob/main/MANUAL.md#update-your-code). ### Other breaking changes -* Promise clients are now the default and the previously-deprecated +- Promise clients are now the default and the previously-deprecated `createPromiseClient` has been removed. Any callsites using `createPromiseClient` should be updated to use `createClient`. :white_check_mark: The `connect-migrate` tool will handle this. -* The gRPC Transport now requires HTTP/2. If you are using `createGrpcTransport` +- The gRPC Transport now requires HTTP/2. If you are using `createGrpcTransport` and specifying an `httpVersion`, it will fail compilation. Remove the `httpVersion` property to use the default of HTTP/2. @@ -360,7 +358,7 @@ as well as the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/prot may require bigger architectural changes, but the hope is that this is not a common problem. -* Previously, Connect allowed request objects with matching shapes to be passed +- Previously, Connect allowed request objects with matching shapes to be passed to API calls interchangeably as long as the passed object was a superset of the target type. For example, given the following proto definitions: @@ -390,8 +388,8 @@ as well as the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/prot This was an unintended bug and not a feature. In Connect v2, only the specified target type will pass compilation. - If you intend to pass a message as a different message with the same fields, - you can use object destructuring to drop the `$typeName`, and copy the rest + If you intend to pass a message as a different message with the same fields, + you can use object destructuring to drop the `$typeName`, and copy the rest of the properties: ```ts @@ -400,7 +398,7 @@ as well as the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/prot const messageB = create(MessageBSchema, properties); ``` -- We have removed the `credentials` option from transports as well as the `init` +* We have removed the `credentials` option from transports as well as the `init` option in interceptors. These two options were used to customize `fetch` routines. Users should now rely on the `fetch` option in transports as a way to perform these customizations. For example: @@ -416,7 +414,7 @@ as well as the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/prot Connect GET request in server-side interceptors, the property `requestMethod: string` has been added to intercepted requests. This property is symmetrical to `HandlerContext.requestMethod`. -* Errors details are now a pair of desc and init values. In Connect v1, error +- Errors details are now a pair of desc and init values. In Connect v1, error details were specified as message instances. In v2, error details are now an object that specifies both a schema and initialization object which are both passed to the `create` function of Protobuf-ES. For example: @@ -451,7 +449,7 @@ as well as the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/prot ); ``` -* `MethodDescriptor` is now self-sufficient. In v1, method descriptors always +- `MethodDescriptor` is now self-sufficient. In v1, method descriptors always had to be used alongside the service descriptors. In v2, they can now be used standalone. This means that all callsites that previously passed a service and method descriptor should now just pass the method descriptor. @@ -465,12 +463,12 @@ as well as the [Protobuf-ES v2 migration guide](https://github.com/bufbuild/prot } ``` -* Interceptors for streaming RPCs now use appropriate stream types. In +- Interceptors for streaming RPCs now use appropriate stream types. In v1, the server used more exact types in interceptors, for example `UnaryRequest` for server-streaming rpcs while the client always used streaming variants. This was unintended behavior and has been fixed in v2. Now all streaming RPCs use the `StreamRequest` and `StreamResponse` types on the server as well. -* Node 16 is no longer supported. Connect v2 now supports Node versions **18.14.1** and up. +- Node 16 is no longer supported. Connect v2 now supports Node versions **18.14.1** and up. -* Connect v2 now requires at least TypeScript **v4.9.5**. +- Connect v2 now requires at least TypeScript **v4.9.5**. From d2acd0740caba8d5cb9292b2e7dc1e51e5ff1c23 Mon Sep 17 00:00:00 2001 From: Steve Ayers Date: Wed, 13 Nov 2024 12:03:03 -0500 Subject: [PATCH 13/13] Update MIGRATING.md Signed-off-by: Steve Ayers --- MIGRATING.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/MIGRATING.md b/MIGRATING.md index 8bbfdaa8..a73d224a 100644 --- a/MIGRATING.md +++ b/MIGRATING.md @@ -194,12 +194,12 @@ files. An easy way to do this is to use the `clean` option provided by the Buf CLI introduced in v1.36.0: ```diff - # buf.gen.yaml - version: v2 +# buf.gen.yaml +version: v2 + clean: true - plugins: - - local: protoc-gen-es - out: src/gen +plugins: + - local: protoc-gen-es + out: src/gen ``` With this option, `buf generate` will delete the contents of `src/gen` before