Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Schema Registry Avro] Publish samples #21814

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion sdk/schemaregistry/schema-registry-avro/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@
"devDependencies": {
"@azure/dev-tool": "^1.0.0",
"@azure/eslint-plugin-azure-sdk": "^3.0.0",
"@azure/event-hubs": "^5.8.0-beta.3",
"@azure/event-hubs": "^5.8.0",
"@azure/identity": "^2.0.1",
"@azure/test-utils": "^1.0.0",
"@azure-tools/test-recorder": "^1.0.0",
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Azure Schema Registry client library samples for JavaScript

These sample programs show how to use the JavaScript client libraries for Azure Schema Registry in some common scenarios.

| **File Name** | **Description** |
| ----------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [schemaRegistryAvroSample.js][schemaregistryavrosample] | Demonstrates the use of AvroSerializer to create messages with avro-serialized payload using schema from Schema Registry. |
| [withEventHubsBufferedProducerClient.js][witheventhubsbufferedproducerclient] | Demonstrates the use of AvroSerializer to create messages with avro-serialized payload using schema from Schema Registry and send them to an Event Hub using the EventHub Buffered Producer Client. |
| [withEventHubsConsumerClient.js][witheventhubsconsumerclient] | Demonstrates the use of AvroSerializer to deserialize messages with avro-serialized payload received from the Event Hub Consumer Client. |
| [withEventHubsProducerClient.js][witheventhubsproducerclient] | Demonstrates the use of AvroSerializer to create messages with avro-serialized payload using schema from Schema Registry and send them to an Event Hub using the EventHub Producer Client. |

## Prerequisites

The sample programs are compatible with [LTS versions of Node.js](https://nodejs.org/about/releases/).

You need [an Azure subscription][freesub] and the following Azure resources to run these sample programs:

- [Azure Schema Registry resource][createinstance_azureschemaregistryresource]

Samples retrieve credentials to access the service endpoint from environment variables. Alternatively, edit the source code to include the appropriate credentials. See each individual sample for details on which environment variables/credentials it requires to function.

Adapting the samples to run in the browser may require some additional consideration. For details, please see the [package README][package].

## Setup

To run the samples using the published version of the package:

1. Install the dependencies using `npm`:

```bash
npm install
```

2. Edit the file `sample.env`, adding the correct credentials to access the Azure service and run the samples. Then rename the file from `sample.env` to just `.env`. The sample programs will read this file automatically.

3. Run whichever samples you like (note that some samples may require additional setup, see the table above):

```bash
node schemaRegistryAvroSample.js
```

Alternatively, run a single sample with the correct environment variables set (setting up the `.env` file is not required if you do this), for example (cross-platform):

```bash
npx cross-env SCHEMA_REGISTRY_ENDPOINT="<schema registry endpoint>" SCHEMA_REGISTRY_GROUP="<schema registry group>" node schemaRegistryAvroSample.js
```

## Next Steps

Take a look at our [API Documentation][apiref] for more information about the APIs that are available in the clients.

[schemaregistryavrosample]: https://github.com/Azure/azure-sdk-for-js/blob/main/sdk/schemaregistry/schema-registry-avro/samples/v1/javascript/schemaRegistryAvroSample.js
[witheventhubsbufferedproducerclient]: https://github.com/Azure/azure-sdk-for-js/blob/main/sdk/schemaregistry/schema-registry-avro/samples/v1/javascript/withEventHubsBufferedProducerClient.js
[witheventhubsconsumerclient]: https://github.com/Azure/azure-sdk-for-js/blob/main/sdk/schemaregistry/schema-registry-avro/samples/v1/javascript/withEventHubsConsumerClient.js
[witheventhubsproducerclient]: https://github.com/Azure/azure-sdk-for-js/blob/main/sdk/schemaregistry/schema-registry-avro/samples/v1/javascript/withEventHubsProducerClient.js
[apiref]: https://docs.microsoft.com/javascript/api/@azure/schema-registry-avro
[freesub]: https://azure.microsoft.com/free/
[createinstance_azureschemaregistryresource]: https://aka.ms/schemaregistry
[package]: https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/schemaregistry/schema-registry-avro/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
{
"name": "@azure-samples/schema-registry-avro-js",
"private": true,
"version": "1.0.0",
"description": "Azure Schema Registry client library samples for JavaScript",
"engines": {
"node": ">=12.0.0"
},
"repository": {
"type": "git",
"url": "git+https://github.com/Azure/azure-sdk-for-js.git",
"directory": "sdk/schemaregistry/schema-registry-avro"
},
"keywords": [
"azure",
"cloud",
"typescript"
],
"author": "Microsoft Corporation",
"license": "MIT",
"bugs": {
"url": "https://github.com/Azure/azure-sdk-for-js/issues"
},
"homepage": "https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/schemaregistry/schema-registry-avro",
"dependencies": {
"@azure/schema-registry-avro": "latest",
"dotenv": "latest",
"@azure/identity": "^2.0.1",
"@azure/schema-registry": "^1.0.1",
"@azure/event-hubs": "^5.8.0"
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
SCHEMA_REGISTRY_ENDPOINT=<Endpoint URL>
SCHEMA_REGISTRY_GROUP=<Group name for schemas in registry>

# Used to authenticate using Azure AD as a service principal for role-based authentication
# in the tokenAuth sample.
#
# See the documentation for `EnvironmentCredential` at the following link:
# https://docs.microsoft.com/javascript/api/@azure/identity/environmentcredential
AZURE_TENANT_ID=<AD tenant id or name>
AZURE_CLIENT_ID=<ID of the user/service principal to authenticate as>
AZURE_CLIENT_SECRET=<client secret used to authenticate to Azure AD>

# Used in samples that use Event Hubs. Retrieve these values from an Event Hub in the Azure portal.
EVENTHUB_CONNECTION_STRING=<Event Hub connection string>
EVENTHUB_NAME=<Event Hub name>
CONSUMER_GROUP_NAME=<Event Hub Group name>

# Used in samples that use Event Grid. Retrieve these values from an Event Grid topic in the Azure portal
EVENT_GRID_TOPIC_ENDPOINT=<Event Grid topic endpoint>
EVENT_GRID_TOPIC_API_KEY=<Event Grid topic API key>
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

/**
* @summary Demonstrates the use of AvroSerializer to create messages with avro-serialized payload using schema from Schema Registry.
*/

const { DefaultAzureCredential } = require("@azure/identity");
const { SchemaRegistryClient } = require("@azure/schema-registry");
const { AvroSerializer } = require("@azure/schema-registry-avro");

// Load the .env file if it exists
require("dotenv").config();

// The fully qualified namespace for schema registry
const schemaRegistryFullyQualifiedNamespace =
process.env["SCHEMA_REGISTRY_ENDPOINT"] || "<endpoint>";

// The schema group to use for schema registeration or lookup
const groupName = process.env["SCHEMA_REGISTRY_GROUP"] || "AzureSdkSampleGroup";

// Sample Avro Schema for user with first and last names
const schemaObject = {
type: "record",
name: "User",
namespace: "com.azure.schemaregistry.samples",
fields: [
{
name: "firstName",
type: "string",
},
{
name: "lastName",
type: "string",
},
],
};

const schema = JSON.stringify(schemaObject);

// Description of the schema for registration
const schemaDescription = {
name: `${schemaObject.namespace}.${schemaObject.name}`,
groupName,
format: "Avro",
definition: schema,
};

async function main() {
// Create a new client
const client = new SchemaRegistryClient(
schemaRegistryFullyQualifiedNamespace,
new DefaultAzureCredential()
);

// Register the schema. This would generally have been done somewhere else.
// You can also skip this step and let `serialize` automatically register
// schemas using autoRegisterSchemas=true, but that is NOT recommended in production.
await client.registerSchema(schemaDescription);

// Create a new serializer backed by the client
const serializer = new AvroSerializer(client, { groupName });

// serialize an object that matches the schema and put it in a message
const value = { firstName: "Jane", lastName: "Doe" };
const message = await serializer.serialize(value, schema);
console.log("Created message:");
console.log(JSON.stringify(message));

// deserialize the message back to an object
const deserializedObject = await serializer.deserialize(message);
console.log("Deserialized object:");
console.log(JSON.stringify(deserializedObject));
}

main().catch((err) => {
console.error("The sample encountered an error:", err);
});

module.exports = { main };
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
// Copyright (c) Microsoft Corporation.
// Licensed under the MIT License.

/**
* @summary Demonstrates the use of AvroSerializer to create messages with avro-serialized payload using schema from Schema Registry and send them to an Event Hub using the EventHub Buffered Producer Client.
*/

const { DefaultAzureCredential } = require("@azure/identity");
const { SchemaRegistryClient } = require("@azure/schema-registry");
const { AvroSerializer } = require("@azure/schema-registry-avro");
const { EventHubBufferedProducerClient, createEventDataAdapter } = require("@azure/event-hubs");

// Load the .env file if it exists
require("dotenv").config();

// The fully qualified namespace for schema registry
const schemaRegistryFullyQualifiedNamespace =
process.env["SCHEMA_REGISTRY_ENDPOINT"] || "<endpoint>";

// The schema group to use for schema registeration or lookup
const groupName = process.env["SCHEMA_REGISTRY_GROUP"] || "AzureSdkSampleGroup";

// The connection string for Event Hubs
const eventHubsConnectionString = process.env["EVENTHUB_CONNECTION_STRING"] || "";

// The name of Event Hub the client will connect to
const eventHubName = process.env["EVENTHUB_NAME"] || "";

// Sample Avro Schema for user with first and last names
const schemaObject = {
type: "record",
name: "User",
namespace: "com.azure.schemaregistry.samples",
fields: [
{
name: "firstName",
type: "string",
},
{
name: "lastName",
type: "string",
},
],
};

const schema = JSON.stringify(schemaObject);

// Description of the schema for registration
const schemaDescription = {
name: `${schemaObject.namespace}.${schemaObject.name}`,
groupName,
format: "Avro",
definition: schema,
};

async function handleError() {
console.log("An error occured when sending a message");
}

async function main() {
// Create a new client
const schemaRegistryClient = new SchemaRegistryClient(
schemaRegistryFullyQualifiedNamespace,
new DefaultAzureCredential()
);

// Register the schema. This would generally have been done somewhere else.
// You can also skip this step and let `serialize` automatically register
// schemas using autoRegisterSchemas=true, but that is NOT recommended in production.
await schemaRegistryClient.registerSchema(schemaDescription);

// Create a new serializer backed by the client
const serializer = new AvroSerializer(schemaRegistryClient, {
groupName,
messageAdapter: createEventDataAdapter(),
});

const eventHubsBufferedProducerClient = new EventHubBufferedProducerClient(
eventHubsConnectionString,
eventHubName,
{
onSendEventsErrorHandler: handleError,
}
);

// serialize an object that matches the schema
const value = { firstName: "Jane", lastName: "Doe" };
const message = await serializer.serialize(value, schema);
console.log("Created message:");
console.log(message);

await eventHubsBufferedProducerClient.enqueueEvent(message);
console.log(`Message was added to the queue and is about to be sent`);

// Wait for a bit before cleaning up the sample
setTimeout(async () => {
await eventHubsBufferedProducerClient.close({ flush: true });
console.log(`Exiting sample`);
}, 30 * 1000);
}

main().catch((err) => {
console.error("The sample encountered an error:", err);
});

module.exports = { main };
Loading