Update directory-structure.md: clarify service, service group, uniform versioning.

cSpell.json

@@ -3,6 +3,7 @@
     "language": "en",
     "words": [
         "authnotrequired",
+        "confidentialledger",
         "Creds",
         "fixdate",
         "frontends",
@@ -291,38 +292,38 @@
                 "freemium"
             ]
         },
-		{
+        {
             "filename": "**/specification/sql/resource-manager/Microsoft.Sql/preview/2023-02-01-preview/ManagedInstances.json",
             "words": [
                 "freemium"
             ]
         },
-		{
+        {
             "filename": "**/specification/sql/resource-manager/Microsoft.Sql/preview/2023-05-01-preview/ManagedInstances.json",
             "words": [
                 "freemium"
             ]
         },
-		{
+        {
             "filename": "**/specification/sql/resource-manager/Microsoft.Sql/preview/2023-08-01-preview/ManagedInstances.json",
             "words": [
                 "freemium"
             ]
         },
-		{
+        {
             "filename": "**/specification/sql/resource-manager/Microsoft.Sql/preview/2023-08-01-preview/ServerTrustCertificates.json",
             "words": [
                 "freemium"
             ]
         },
-		{
+        {
             "filename": "**/specification/sql/resource-manager/Microsoft.Sql/preview/2023-08-01-preview/ManagedInstanceDtcs.json",
             "words": [
                 "msdtcdns",
                 "msdtc's"
             ]
         },
-		{
+        {
             "filename": "**/specification/sql/resource-manager/Microsoft.Sql/preview/2023-08-01-preview/TimeZones.json",
             "words": [
                 "freemium"
@@ -505,7 +506,7 @@
                 "Idms",
                 "Cluter",
                 "Mfamily",
-				"Datacentre"
+                "Datacentre"
             ]
         },
         {
@@ -865,15 +866,13 @@
                 "psqldb"
             ]
         },
-
         {
             "filename": "**/specification/maps/data-plane/Weather/stable/1.1/weather.json",
             "words": [
                 "locationally",
                 "unittype"
             ]
         },
-
         {
             "filename": "**/specification/maps/data-plane/Microsoft.Maps/Render/preview/1.0/render.json",
             "words": [
@@ -1183,10 +1182,9 @@
         {
             "filename": "**/specification/containerservice/resource-manager/Microsoft.ContainerService/**/*.json",
             "words": [
-                 "containerd"
+                "containerd"
             ]
         },
-
         {
             "filename": "**/specification/batch/data-plane/Microsoft.Batch/**/*.json",
             "words": [
@@ -1225,8 +1223,8 @@
             ]
         },
         {
-           "filename": "**/specification/confidentialledger/data-plane/Microsoft.CodeTransparency/preview/**/*.json",
-           "words": [
+            "filename": "**/specification/confidentialledger/data-plane/Microsoft.CodeTransparency/preview/**/*.json",
+            "words": [
                 "cbor",
                 "txids",
                 "operationid",
@@ -1254,8 +1252,8 @@
         {
             "filename": "**/specification/sqlvirtualmachine/resource-manager/Microsoft.SqlVirtualMachine/**/*.json",
             "words": [
-                 "WUMU",
-                 "WSUS"
+                "WUMU",
+                "WSUS"
             ]
         },
         {

documentation/directory-structure.md

@@ -7,6 +7,8 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one
 -->
 
 - [`specification` directory structure](#specification-directory-structure)
+  - [Key concepts](#key-concepts)
+    - [Uniform versioning](#uniform-versioning)
   - [`specification` folder](#specification-folder)
   - [`resource-manager` and `data-plane` folders](#resource-manager-and-data-plane-folders)
   - [AutoRest configuration `README.md` files](#autorest-configuration-readmemd-files)
@@ -17,7 +19,9 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one
   - [Advanced scenario: service group](#advanced-scenario-service-group)
     - [Service group `common-types`](#service-group-common-types)
     - [Versioning services in a service group](#versioning-services-in-a-service-group)
+    - [Migrating from singular service to service group](#migrating-from-singular-service-to-service-group)
   - [Deprecated structure and hand-written OpenAPI specs](#deprecated-structure-and-hand-written-openapi-specs)
+
 # `specification` directory structure
 
 
@@ -30,13 +34,72 @@ You may be also interested in following:
 > The structure described in this article is strictly enforced. There exist some exceptions for historical reasons.
 > These exceptions are not allowed going forward.
 
+## Key concepts
+
+The directory structure is a reflection of few key concepts.
+
+A `service` is a set of operation endpoints (typically HTTP REST API endpoints) that **version uniformly**.
+
+A `service group` is a set of services that share common (ARM) **Resource Provider (RP) Namespace**, `RPNS`.
+
+In case of ARM, a `service` consists of multiple `resource types`.
+A `resource type` can be derived from the URL of HTTP API operations that pertain to it.
+
+### Uniform versioning
+
+A `service` **must version uniformly**. This means the following:
+
+1. Any deployed service operation endpoint must belong to an API version. An API version once deployed is immutable.
+   Its behavior or constituent operations cannot be changed.
+1. Any given service API version can be composed only of HTTP API operations and ARM resource types (if applicable)
+   that have the same API version. 
+1. Any documentation pertaining to the service and any SDKs generated from the service must pertain to only
+   one service version.
+1. The service version must be always represented in its entirety; any SDK or documentation referring to the service
+   must encompass all of it.
+1. The `common-types` OpenAPI definition shared across multiple services can version independently of the service.
+   However, the rule that API version is immutable still must be observed.
+
+The **uniform versioning** has several implications and implementation decisions supporting it:
+
+- The service is effectively defined as a series of consecutive API versions.
+- Each API version is represented by a pair of folders representing its lifecycle stage and release date, 
+  like `stable/2024-03-05` or `preview/2024-05-15-preview`.
+- The entirety of given API service specification must be placed inside its folder, e.g. `stable/2024-03-05`.
+  The service consists of operation endpoints defined in OpenAPI spec `.json` files placed in its API version folder.
+- The service, its documentation, and any SDK referencing it, all must version in lockstep. If a new service API
+  version is released, also a new entry of documentation must be released to cover it. Similarly, a new version
+  of given SDK package must be released to refer to the new service version.
+- Nowhere within a service, documentation for it, or SDK referencing it,
+  can multiple service API versions be mixed. As such:
+  - `preview` API versions cannot be mixed with `stable` API versions.
+  - No REST API endpoint for given API version can have any kind of dependency on service endpoint from any other API version.
+- Any AutoRest config README.md file definition for the service must have corresponding tags to the API versions
+  present in the directory structure. Each of these tags must include all OpenAPI spec `.json` files for given API version,
+  and only for that API version.
+- All the shared OpenAPI definitions (i.e. `common-types`) the service depends on must have the same version, 
+  [e.g. `v6`][common-types v6], but it can be different from the version of the service.
+- Updating `common-types` version requires updating the API version.
+  For example, if `common-types` published an updated version of `v7`,
+  then if the service wants to take dependency on it, it can only do it in
+  a new version. For example, `2024-04-17`.
+  Because the service version is now `2024-04-17`, all the OpenAPI specification `.json` files the service is composed
+  of must have the same version of `2024-04-17` and the `info.version` property must say `2024-04-17`.
+  In addition, a new SDK must be generated from the service, and new documentation published, both tagged with
+  service version `2024-04-17`.
+- All of these rules apply both for OpenAPI specs emitted from TypeSpec as well as for hand-written.
+
 ## `specification` folder
 
 The `specification` folder is the root folder for all service specifications.
 
-Each child of the `specification` folder corresponds to a `service` specification for given Azure team. Here we denote such folder as `<azureTeam>`.
-In advanced cases for big teams the `<azureTeam>` folder can host multiple services, known as `service group`.
-Read [the relevant section](#advanced-scenario-service-group) for details.
+Each child of the `specification` folder corresponds to a `service` or `service group` specification
+for given Azure team, depending on if given Azure team owns one or multiple services.
+We denote such folder as `<azureTeam>`.
+
+This article explains the singular `service` scenario. The details specific to `service group` and how they differ
+from the `service` scenario are explained in the
+[advanced scenario: service group](#advanced-scenario-service-group) section.
 
 Given `<azureTeam>` has following structure:
 
@@ -50,7 +113,8 @@ You can learn more about TypeSpec at [aka.ms/azsdk/typespec] and [aka.ms/typespe
 
 ## `resource-manager` and `data-plane` folders
 
-The `<azureTeam>/resource-manager` contains the ARM OpenAPI specifications emitted from TypeSpec in `<azureTeam>/<typeSpecFolder>`.
+The `<azureTeam>/resource-manager` contains the ARM OpenAPI specifications emitted from TypeSpec in `<azureTeam>/<typeSpecFolder>`
+folders.
 
 The `resource-manager` folder has exactly one child folder whose name matches the **Resource Provider (RP) Namespace** (`<RPNS>`), 
 such as `Microsoft.Automation` (full list of namespaces is [here][Resource Provider list]).
@@ -184,6 +248,17 @@ In case of big Azure teams, their `specification/<azureTeam>` hosts multiple ser
 The main difference between one service and a service group is how they are presented to Azure customers:
 One service has one SDK package and one documentation portal, while a service group has separate SDK package for each service and separate documentation.
 
+In case of a `service group`, the structure is like for `service`, but the contents of each
+`resource-manager/<RPNS>` and `data-plane/<dataPlaneSubfolder>` is instead additionally nested in a `<service>` parent, for each service in the service group.
+
+The Azure team that has a service group has following directory structure:
+
+- `<azureTeam>/<typeSpecFolder>` (multiple folders)
+- `<azureTeam>/resource-manager/<RPNS>/<service>` (multiple folders, one for each value of `<service>`)
+- `<azureTeam>/data-plane/<dataPlaneSubfolder>/<service>` (multiple folders, one for each value of `<service>`)
+
+Furthermore the `README.md` files are no longer located in the  `resource-manager` or `data-plane` folders, but instead in `<service>` folders.
+
 For example, [`specification/containerservice`] is a `service group` for both `aks` and `fleet` services.
 
 The doc for `aks` is [Azure Kubernetes Service]. It points to aks REST reference e.g. for [API version `2024-01-01`][aks REST reference 2024-01-01],
@@ -211,6 +286,18 @@ This means that each service in the service group must obey the same versioning
 However, multiple separate services can have different versioning cycles, including different SDK packages. Refer to the aforementioned `aks` and `fleet`
 services for examples of different versioning cycles in a service group.
 
+### Migrating from singular service to service group
+
+In case your azure SDK team owned a singular service and is now expanding to a service group, do the following:
+
+- In one pull request, move the singular service to become a service in a service group. This means it will be moved
+  inside a directory denoting the service name (i.e. `/<service>/`), and its README.md file will also be moved there. 
+  As this PR is moving the service, it means the PR diff will include deletions of the singular service README.md and
+  API version folders, and it will include addition of the new README.md and API version folder,
+  both inside the service folder.
+- In a follow-up PRs add other services to belong to the service group. One service per PR. This means each such PR will
+  add new `/<service>/` folder with its own README.md and API version folders.
+
 ## Deprecated structure and hand-written OpenAPI specs
 
 As mentioned at the beginning of this article, for historical reasons, some `specification/<azureTeam>` folders may
@@ -241,6 +328,7 @@ All of the aforementioned cases are considered legacy and are not allowed going
 [aks REST reference 2024-01-01]: https://github.com/Azure/azure-rest-api-specs/tree/main/specification/containerservice/resource-manager/Microsoft.ContainerService/aks/stable/2024-01-01
 [Azure Kubernetes Fleet Manager]: https://learn.microsoft.com/en-us/azure/kubernetes-fleet/
 [Azure Kubernetes Service]: https://learn.microsoft.com/en-us/azure/aks/
+[common-types v6]: https://github.com/Azure/azure-rest-api-specs/tree/main/specification/common-types/resource-management/
 [fleet REST reference 2023-10-15]:  https://learn.microsoft.com/en-us/rest/api/fleet/operation-groups?view=rest-fleet-2023-10-15
 [Resource Provider list]: https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/azure-services-resource-providers#match-resource-provider-to-service
 [Specification index]: https://azure.github.io/azure-sdk/releases/latest/all/specs.html