Skip to content

Cadl Java Quickstart

Jump to bottom
Weidong Xu edited this page Feb 9, 2023 · 41 revisions

Before you Start

Java Azure SDK Design Guidelines is the overall design guideline of the data-plane SDK. Make sure you are familiar with concepts such as "Service Client" and "Packaging".

Make sure you are familiar with Git and Maven, especially the Build Lifecycle.

Please also take a look at Protocol Methods on the overview of the typical protocol methods from minimal data-plane client SDK.

⚠️ If you uses OpenAPI 2.0, please read Quickstart with AutoRest-Java instead.

Generate SDK

Module folder in SDK repository

For service, it is usually your service name in REST API specifications. For instance, if the JSON is under specification/storage, the service would normally be storage.

For module, please refer to Namespace in Java Guideline. For example, if the namespace is com.azure.storage.blob, the module should be azure-storage-blob.

☑️ Namespace is decided by SDK arch board.

After service and module, the output-dir would be <your-local-repository-dir>/<service>/<module>. This is where the SDK code should be generated when you call npx cadl compile.

Use SDK Automation from REST API specifications

☑️ It is recommended to configure Cadl Specs on REST API specifications. Please refer to the Guidelines.

After configuration is completed, making a draft pull request on azure-rest-api-specs repository would automatically trigger SDK generation for all configured SDKs. The automation will create new pull request on azure-sdk-for-java repository. One can fork it for further development.

Use Cadl locally

See Cadl-Java readme or latest Cadl-Java NPM.

Generally, namespace is required in emitter options in "cadl-project.yaml".

When generating SDK code at your local, use command line npx cadl compile <cadl-source> --emit=@azure-tools/cadl-java --options='@azure-tools/cadl-java.emitter-output-dir=<output-dir>'.

It only generates for Java, and it generates code to the <output-dir> folder.

The <cadl-source> usually be either client.cadl or main.cadl.

☑️ If there is a client.cadl, always use client.cadl, as it is the source for SDK code generator.

client.cadl

@convenientAPI decorator enables generating convenience API for .NET and Java. They are by default enabled, hence usually one does not need to specify the decorator.

@client decorator configures the "Client"s of the SDK. If no @client is provided, SDK uses a single Client with name from service-name, or your Cadl namespace. You may use the decorator like this example.

Both decorators are from cadl-dpg. See augment decorators.

Follow-up code generation

If you modify the Cadl source, you may need to re-generate the SDK. This could happen frequently when you do SDK archboard review, and reviewer might ask you to rename some Cadl models.

If you have not customized anything in the SDK, it is advised to delete <output-dir>/src/main/java folder, before re-generate with the same npx cadl compile ... command line.

The reason is to avoid previous model/class get remained in Java source files, after Cadl source already renamed them to new name.

Build

See Building and Adding a Module.

See the contributing guide.

Here is the list of files that you are required to modify/confirm for your new module.

  • eng/versioning/version_client.txt for versioning
  • sdk/<service>/pom.xml for project
  • sdk/<service>/ci.yml for CI

Install for Local Test

See Building. Install SDK to local maven repository: mvn install -f sdk/<service>/<module>/pom.xml -Dgpg.skip -Drevapi.skip -DskipTests

Tips: Javadoc might fail if other than Java 11. If you see error message like "Error while generating Javadoc", you can add option -Dmaven.javadoc.skip to skip Javadoc.

See Maven Getting Start Guide, if you would like to create a client that uses your SDK. See sdk/<service>/<module>/README.md for adding the SDK to dependency.

Improve SDK Documentation

There is many information about the SDK that Cadl-Java will never know. These can only be provided by a contributor.

README

A minimal README.md is generated by Cadl-Java. Your first task will be improving it.

☑️ You would want to modify [product_documentation] to point to your service documentation.

Samples

A blank Java sample file is generated as ReadmeSamples.java, which is synchronized with your README (see Codesnippet Maven Plugin).

Add your sample code between // BEGIN: ... and // END: ... in ReadmeSamples.java, and build the project, you will find your Examples section in README updated with the code.

A collection of generated Java sample files (based on x-ms-examples in OpenAPI specs) is generated under <module>.generated package. They might not be directly usable as runnable sample codes, however you should get the basic idea on how code works, and may even modify some to be runnable.

An example of adding samples is demonstrated in this commit.

Tests

A <client>TestBase file and a collection of generated Java test files (based on x-ms-examples in OpenAPI specs) is generated under <module>.generated package.

See Adding Tests.

Feature Checklist

Authentication and credential

Cadl-Java supports OAuth2 and ApiKey authentication.

However, if service requires advanced solution (for example, JWT claim or HMAC-SHA256 authentication scheme), you are responsible to provide the implementation (usually as a subclass of HttpPipelinePolicy), and customize the ClientBuilder.

Long-running operation

Cadl-Java supports a default chained polling strategy for long-running operation.

It supports status monitor LRO and simple Location-header-based LRO. It does not support the outdated resource based LRO.

If service uses non-supported or non-standard LRO, you can contact Java SDK for guidance on the implementation of a specialized polling strategy, and customization.

File upload

Cadl-Java supports application/octet-stream, but it does not support multipart/form-data. An issue in azure-core tracks the status of the feature.

Review API Documentation

Generate Javadoc: mvn javadoc:javadoc -f sdk/<service>/<module>/pom.xml

Review the generated API documentation sdk/<service>/<module>/target/site/apidocs/index.html

Improve SDK Code

ClientBuilder, Client, AsyncClient is the user-facing API surface that you might be interested to improve. See Partial Update and Customization.

Code Review

Code review happens on 2 places.

One is the usual GitHub pull request, for developers.

Another is the apiview, for architects. GitHub pull request should automatically trigger the apiview. If this does not happen, upload one manually:

  1. Build the SDK.
  2. Login apiview with GitHub account.
  3. "Create review" at top-left corner.
  4. "Browse" and upload sdk/<service>/<module>/target/<module>-<version>-sources.jar. If you did not find the source jar in step 4, run mvn source:jar.

Package Release

See Release Checklist.

Prepare Release Script

In particular, you need to follow "Prepare Release Script" and make another "prepare release" PR.

Before running the script, make sure your CHANGELOG.md is correct. Except for first release, it should contain following text on dependency updates.

### Other Changes

#### Dependency Updates

- Upgraded `azure-core` from `1.32.0` to version `1.33.0`.
- Upgraded `azure-core-http-netty` from `1.12.5` to version `1.12.6`.

After running the script, check the "release date" and make sure it is your scheduled release date.

Appendix

Java Development Environment

Windows

Download Java, install.

Download Maven, unpack, set to PATH.

Download Node.js, install.

Optional for Git:

  • Download Git, install.

Linux with APT

sudo apt -y install openjdk-17-jdk

sudo apt -y install maven

curl -sL https://deb.nodesource.com/setup_16.x | sudo bash -
sudo apt -y install nodejs

Adding Tests

Adding runnable tests requires both a good understanding of the service, and the knowledge of the client and test framework. Feel free to contact SDK developers, if you encountered issues on client or test framework.

Test environment

Before testing the client, it is recommended that you prepare a Bicep or ARM template (or code, or CLI script, equivalent to the ARM template) that creates all required Azure resources and assigns necessary RBAC for your client. This ARM template will be very helpful for you to re-run the tests after version update, or for other developers to create their test environment.

Client authentication

If your service does not support connection string or API key authentication, it would likely use AzureAD OAuth2 token for authentication. A common solution is to provide an application and its service principal and to provide RBAC to the service principal for the access to the Azure resource of your service.

Client requires following 3 environment variables for the service principal using client ID/secret for authentication:

AZURE_TENANT_ID
AZURE_CLIENT_ID
AZURE_CLIENT_SECRET

On Linux, you can call export:

export AZURE_TENANT_ID=<tenant_id>
export AZURE_CLIENT_ID=<client_id>
export AZURE_CLIENT_SECRET=<client_secret>

On Windows, it is set.

Live tests

Live tests requires one more environment variable:

export AZURE_TEST_MODE=LIVE

Cadl-Java already provided you the generated Java test files at sdk/<service>/<module>/src/test/java/<namespace>/generated folder, and a <client>TestBase class. You should double check the beforeTest() method in <client>TestBase class, and modify it if necessary.

A sample <client>TestBase class can be found here. The "Client" is configured different for different test mode. For "PLAYBACK", it uses a mock TokenCredential and a playback HttpClient; for "RECORD", it uses a record HttpPolicy; for "LIVE", it uses what customer would use.

It is recommended to copy the test files you would like to modify and use to sdk/<service>/<module>/src/test/java/<namespace> folder.

It is common for you to put configuration from your test environment to the code, as long as they are not a secret. Alternatively, you can keep rely on Configuration.getGlobalConfiguration().get("<env_variable>") and define a few additional environment variables.

After you settle down the builder, you can decide which REST APIs you want to test, and write the code and verification in the testClient() method. The generated samples under <module>.generated package can be used as reference for your test code.

After you finish the test code, call mvn test under sdk/<service>/<module> to execute the tests. If everything works fine, you will see following log.

[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.776 s - in com.azure.messaging.webpubsub.ClientTests
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

An example of adding live tests is demonstrated in this commit.

Record/playback tests

The live tests only runs when environment variable AZURE_TEST_MODE is set to LIVE, and hence it is usually run manually.

If you are making lots of customization to the SDK, and wish these tests to run every time during continuous integration when you modify code, you can consider changing them to record/playback tests.

⚠️ Use TestBase.setPlaybackSyncPollerPollInterval on the SyncPoller instance, if your test involves LRO API.

Record

Firstly, the tests need to be recorded.

Remove @DoNotRecord(skipInPlayback = true) annotation in ClientTests.java. And set environment variable AZURE_TEST_MODE to RECORD:

export AZURE_TEST_MODE=RECORD

Then call mvn test as if this is live tests. Different from live tests, a folder sdk/<service>/<module>/target/test-classes/session-records/ is created with the recorded request/response of the REST APIs. Copy the folder to sdk/<service>/<module>/src/test/resources/session-records/ and commit it to the SDK repository.

Playback

After session-records folder added to SDK repository, these recorded tests can now be run in playback mode automatically (it can be run locally via the same mvn test call, by setting environment variable AZURE_TEST_MODE to PLAYBACK).

Be aware that in playback mode during continuous integration, there is no test environment nor environment variables available. Hence if you rely on Configuration.getGlobalConfiguration().get("<env_variable>"), make sure you provide a second parameter as the default value, for the case when <env_variable> is not available.

An example of adding record/playback tests is demonstrated in this commit.

Partial Update

You can in-place modify the generated code, and Cadl-Java will keep the hand-written code intact when it re-generates the SDK.

The user-facing API is of pattern:

FooClient client = new FooClientBuilder()
    .endpoint(...)
    .credential(new DefaultAzureCredentialBuilder().build())
    .buildClient();

FooAsyncClient asyncClient = new FooClientBuilder()
    .endpoint(...)
    .credential(new DefaultAzureCredentialBuilder().build())
    .buildAsyncClient();

And the classes involved is FooClient, FooAsyncClient, and FooClientBuilder.

All of these classes can be customized in-place, to enhance the usability of the SDK, or to add new feature.

You can add new class variables and new class methods to these classes, and with partial-update option in emitter options, new code generated via Cadl-Java will not override your change.

For modifying the signature or the implementation of a class method, please remove the @Generated annotation, so Cadl-Java can recognize that you have modified the method.

If you would like to remove a class method, simply make the method package private, so it is not exposed to customer. Also you need to remove the @Generated annotation.

It is advised that you keep the modifications simple. If certain feature requires complicated code, it is better to put the implementation code in some other classes within implementation package, and only add the public class method and invocation in these Client or ClientBuilder classes.

Troubleshoot CI Failure

Check "Build" and "Analyze" first, the "Test"s.

It helps to run mvn clean verify locally, before checking CI failure.

Verify versions in POM files

See the contributing guide on versioning.

TLDR: run python eng/versioning/update_versions.py --ut library --bt client --sr

Check spelling

Fix spelling in code or in markdown.

Suppression at .vscode/cspell.json.

Verify Readmes / Verify sample metadata / Verify ChangeLogEntry

Fix corresponding files.

Run SpotBugs, Checkstyle, RevApi, and Javadoc

Fix code for SpotBugs, Checkstyle, and Javadoc.

Avoid breaking changes for RevApi.

Suppression at eng/code-quality-reports/src/main/resources.

Client Library

User Guides

Extension Points

Contributor Guides

Management Library

User Guides

Contributor Guides

Spring Cloud Azure

Clone this wiki locally