More decomposition of matter.js

Moves all non-deprecated packages to @matter.js org.

Adds new packages:

  - packages/types contains the bulk of data definitions from matter.js.  This includes datatypes, TLV, clusters, etc.
  - packages/protocol contains the bulk of the lower-level Matter logic from matter.js
  - packages/node contains the higher-level node API including behaviors, endpoints and nodes
  - packages/main is a new entry point that reexports all of above and automatically loads specialization for the detected platform via export conditions
  - packages/nodejs-ble is rename of packages/matter-node-ble.js
  - packages/examples is rename of packages/matter-node.js-examples

The new package structures conform more closely to the export layout as defined by "exports" in package.json.  Previously we had a mix of snake-case and CamelCase for sub-package exports.  I moved to all "snake" for public exports.

The new structures encourages direct import of types (clusters, behaviors, endpoint types) that are expensive either at build time or runtime.  There are index exports for all of these components but they are not in the default package export.  So e.g. you can import OnOff from @project-chip/matter.js-types/clusters or (better) @project-chip/matter.js-types/clusters/on-off.

The original "matter.js" package is mostly for backwards compatibility now except for some of the controller stuff that still needs to move into the "protocol" package.  Ideally the new "main" package would be our "main" package but this will need to wait on either a new package name or a deprecation period sufficient for a (very) major breaking change.

"matter.js" contains a large number of re-exports to maintain backward compatibility.  The "legacy" API we've been threatening to deprecate still lives there as well.  Should make it easier to make a clean break when the time comes.

The sub-package exports of generated code required a huge number of export files both for compatibility and for re-export in the "main" package.  I added custom build logic to "main" and "matter.js" to create these at build time.

Given the large number of import changes I figured it was an opportune time to add package aliasing.  We now alias via package.json "imports" field for most intra- (and some inter-) package imports.  This makes the imports considerably more concise and aids in portability.  A bit fiddly to set up but they work well once in place.

I did various refactoring along the way.  In particular, I removed generic parameters from a number of the components in "protocol" that no longer need to be generic for the "node" API.  This makes the code less verbose and was easy to compensate for in the legacy API where this specialization is necessary.

I also removed the "ContextT" template parameter from many of the APIs in "protocol".  Makes things less wordy and hopefully not controversial.  The controller vs. device branches of the code both only ever deal with a single context type.  I moved a bit of shared functionality into the base SessionContext type and added assertion functions for the few places where specialization is still required.

The default implementation of "time" is now fully functional.  Platform specialization is still available if necessary but I was able to fully implement the API using standard JS so I don't expect it'll ever be necessary.

Includes numerous codegen updates to match the new package layout and make the codegen logic available at build time.

.eslintrc.json

@@ -117,7 +117,11 @@
 
         // Always of dubious value, this rule is buggy as of typescript-eslint 7.2.0 and TS 5.4.2 (works w/ TS 5.3.3
         // though)
-        "@typescript-eslint/no-redundant-type-constituents": "off"
+        "@typescript-eslint/no-redundant-type-constituents": "off",
+
+        // This rule is fine...  Except it's redundant with tsc and seems to be buggy with wildcard package.json
+        // import aliases
+        "import/no-unresolved": "off"
     },
     "settings": {
         "import/extensions": [".ts"],

.gitattributes

@@ -0,0 +1,9 @@
+packages/model/src/standard/elements/*.ts linguist-generated=true
+packages/types/src/clusters/* linguist-generated=true
+packages/types/src/globals/* linguist-generated=true
+packages/node/src/tags/* linguist-generated=true
+packages/node/src/endpoints/* linguist-generated=true
+packages/node/src/devices/* linguist-generated=true
+packages/node/src/behaviors/*/index.ts linguist-generated=true
+packages/node/src/behaviors/*/*Behavior.ts linguist-generated=true
+packages/node/src/behaviors/*/*Interface.ts linguist-generated=true

.prettierignore

@@ -3,9 +3,11 @@
 models/src/chip.ts
 models/src/spec.ts
 packages/model/src/standard/elements/*
-packages/matter.js/src/cluster/definitions/*
-packages/matter.js/src/cluster/globals/*
-packages/matter.js/src/endpoint/definitions/*
-packages/matter.js/src/behavior/definitions/*/export.ts
-packages/matter.js/src/behavior/definitions/*/*Behavior.ts
-packages/matter.js/src/behavior/definitions/*/*Interface.ts
+packages/types/src/clusters/*
+packages/types/src/globals/*
+packages/node/src/tags/*
+packages/node/src/endpoints/*
+packages/node/src/devices/*
+packages/node/src/behaviors/*/index.ts
+packages/node/src/behaviors/*/*Behavior.ts
+packages/node/src/behaviors/*/*Interface.ts

CHANGELOG.md

@@ -11,12 +11,12 @@ The main work (all changes without a GitHub username in brackets in the below li
 
 ## __WORK IN PROGRESS__
 
--   IMPORTANT: As of 0.10.0 the @project-chip/matter.js module has grown quite large.  This release includes major refactoring that moves functional areas into independent NPM packages.  We have added exports to maintain backwards compatibility but these are not exhaustive.  In some cases you may need to update imports to reference new code locations.
+-   IMPORTANT: As of 0.10.0 the @project-chip/matter.js module has grown quite large.  This release includes major refactoring that moves functional areas into independent NPM packages under the "@matter.js" org.  We have added exports to maintain backwards compatibility but these are not exhaustive.  In some cases you may need to update imports to reference new code locations.
 
 -   matter.js-general:
-    -   General functionality that is not Matter specific previously resided in @project-chip/matter.js.  It now lives in @project-chip/matter.js-general
+    -   General functionality that is not Matter specific previously resided in @project-chip/matter.js.  It now lives in @matter.js/general
     -   BREAKING: The "ByteArray" type is removed, replaced with native-JS Uint8Array and a small collection of utility functions in the "Bytes" namespace
-    -   The Matter object model previously exported as @project-chip/matter.js/model now resides in @project-chip/matter.js-model
+    -   The Matter object model previously exported as @project-chip/matter.js/model now resides in @matter.js/model
 
 -   matter.js-nodejs:
     -   Node specialization is moved to matter.js-nodejs.  matter-node.js remains as a compatibility import.

README.md

@@ -137,25 +137,25 @@ This will install all dependencies and create symlinks between the packages, so
 On Windows in order to successfully build all the packages (tested on Windows 11 Pro) make sure that to have installed Node.js 18+, the windows-build-tools and node-gyp version 10.
 On Non-Windows platforms and having Python 3.12+ installed please also make sure to use npm 10.2.3+.
 
-### To use matter(-node).js in own projects
+### To use matter.js in your own project
 
-To use matter.js in own projects you simply use matter-node.js as dependency in your project. This package also re-exports all interfaces from matter.js, so it is not needed to use matter.js directly. If you need BLE support you can use matter-node-ble.js as additional dependency.
+To use matter.js you import `@matter.js/main` as dependency in your project. This package re-exports functionality from other packages, so it is not necessary to have multiple dependencies. If you need BLE support you can use `@matter.js/nodejs-ble` as additional dependency.
 
 ```bash
-npm install @project-chip/matter-node.js --save
+npm install @matter.js/main --save
 ```
 
 If your project is not based on Node.js you need to implement the platform specific parts. See [How to use matter.js in own projects](#how-to-use-matterjs-in-own-projects) for more details.
 
 ### To try out the Node.js based Examples
 
-if you just want to try out the Node.js based examples you can use the following commands:
+if you just want to try out the examples on Node.js you can use the following commands:
 
 ```bash
-npm install @project-chip/matter-node.js-examples
+npm install @matter.js/examples
 ```
 
-Please refer to the Examples Readme on how to use the examples: [matter-node.js-examples](packages/matter-node.js-examples/README.md)
+Please refer to the Examples readme on how to use the examples: [@matter.js/examples](packages/examples/README.md)
 
 ## Code style
 

chip-testing/package.json

@@ -1,5 +1,5 @@
 {
-    "name": "@project-chip/matter.js-chip-testing",
+    "name": "@matter.js/chip-testing",
     "version": "0.0.0-git",
     "description": "Testing of matter.js with CHIP tool",
     "private": true,
@@ -20,9 +20,10 @@
         "build-clean": "matter-build --clean"
     },
     "dependencies": {
-        "@project-chip/matter.js-general": "*",
-        "@project-chip/matter.js-nodejs": "*",
-        "@project-chip/matter.js": "*",
-        "@project-chip/matter.js-tools": "*"
+        "@matter.js/main": "*",
+        "@project-chip/matter.js": "*"
+    },
+    "devDependencies": {
+        "@matter-node/tools": "*"
     }
 }

chip-testing/src/AllClustersTestApp.ts

@@ -4,7 +4,7 @@
  * Copyright 2022-2024 Matter.js Authors
  * SPDX-License-Identifier: Apache-2.0
  */
-import "@project-chip/matter.js-nodejs";
+import "@matter.js/nodejs";
 
 import { AllClustersTestInstance } from "./AllClustersTestInstance.js";
 import { startTestApp } from "./GenericTestApp.js";

chip-testing/src/AllClustersTestInstance.ts

@@ -4,40 +4,42 @@
  * SPDX-License-Identifier: Apache-2.0
  */
 
-import { Bytes, Storage } from "@project-chip/matter.js-general";
-import { AdministratorCommissioningServer } from "@project-chip/matter.js/behavior/definitions/administrator-commissioning";
-import { BooleanStateServer } from "@project-chip/matter.js/behavior/definitions/boolean-state";
-import { CarbonDioxideConcentrationMeasurementServer } from "@project-chip/matter.js/behavior/definitions/carbon-dioxide-concentration-measurement";
-import { CarbonMonoxideConcentrationMeasurementServer } from "@project-chip/matter.js/behavior/definitions/carbon-monoxide-concentration-measurement";
-import { ColorControlServer } from "@project-chip/matter.js/behavior/definitions/color-control";
-import { FixedLabelServer } from "@project-chip/matter.js/behavior/definitions/fixed-label";
-import { FlowMeasurementServer } from "@project-chip/matter.js/behavior/definitions/flow-measurement";
-import { FormaldehydeConcentrationMeasurementServer } from "@project-chip/matter.js/behavior/definitions/formaldehyde-concentration-measurement";
-import { IlluminanceMeasurementServer } from "@project-chip/matter.js/behavior/definitions/illuminance-measurement";
-import { LocalizationConfigurationServer } from "@project-chip/matter.js/behavior/definitions/localization-configuration";
-import { ModeSelectServer } from "@project-chip/matter.js/behavior/definitions/mode-select";
-import { NetworkCommissioningServer } from "@project-chip/matter.js/behavior/definitions/network-commissioning";
-import { NitrogenDioxideConcentrationMeasurementServer } from "@project-chip/matter.js/behavior/definitions/nitrogen-dioxide-concentration-measurement";
-import { OccupancySensingServer } from "@project-chip/matter.js/behavior/definitions/occupancy-sensing";
-import { OzoneConcentrationMeasurementServer } from "@project-chip/matter.js/behavior/definitions/ozone-concentration-measurement";
-import { Pm1ConcentrationMeasurementServer } from "@project-chip/matter.js/behavior/definitions/pm1-concentration-measurement";
-import { Pm10ConcentrationMeasurementServer } from "@project-chip/matter.js/behavior/definitions/pm10-concentration-measurement";
-import { Pm25ConcentrationMeasurementServer } from "@project-chip/matter.js/behavior/definitions/pm25-concentration-measurement";
-import { PowerSourceServer } from "@project-chip/matter.js/behavior/definitions/power-source";
-import { PowerTopologyServer } from "@project-chip/matter.js/behavior/definitions/power-topology";
-import { PressureMeasurementServer } from "@project-chip/matter.js/behavior/definitions/pressure-measurement";
-import { PumpConfigurationAndControlServer } from "@project-chip/matter.js/behavior/definitions/pump-configuration-and-control";
-import { RadonConcentrationMeasurementServer } from "@project-chip/matter.js/behavior/definitions/radon-concentration-measurement";
-import { RelativeHumidityMeasurementServer } from "@project-chip/matter.js/behavior/definitions/relative-humidity-measurement";
-import { SwitchServer } from "@project-chip/matter.js/behavior/definitions/switch";
-import { TemperatureMeasurementServer } from "@project-chip/matter.js/behavior/definitions/temperature-measurement";
-import { ThermostatUserInterfaceConfigurationServer } from "@project-chip/matter.js/behavior/definitions/thermostat-user-interface-configuration";
-import { TimeFormatLocalizationServer } from "@project-chip/matter.js/behavior/definitions/time-format-localization";
-import { TotalVolatileOrganicCompoundsConcentrationMeasurementServer } from "@project-chip/matter.js/behavior/definitions/total-volatile-organic-compounds-concentration-measurement";
-import { UnitLocalizationServer } from "@project-chip/matter.js/behavior/definitions/unit-localization";
-import { UserLabelServer } from "@project-chip/matter.js/behavior/definitions/user-label";
-import { AirQualityServer } from "@project-chip/matter.js/behaviors/air-quality";
-import { DescriptorServer } from "@project-chip/matter.js/behaviors/descriptor";
+import { Bytes, Storage } from "@matter.js/general";
+import {
+    AdministratorCommissioningServer,
+    AirQualityServer,
+    BooleanStateServer,
+    CarbonDioxideConcentrationMeasurementServer,
+    CarbonMonoxideConcentrationMeasurementServer,
+    ColorControlServer,
+    DescriptorServer,
+    FixedLabelServer,
+    FlowMeasurementServer,
+    FormaldehydeConcentrationMeasurementServer,
+    IlluminanceMeasurementServer,
+    LocalizationConfigurationServer,
+    ModeSelectServer,
+    NetworkCommissioningServer,
+    NitrogenDioxideConcentrationMeasurementServer,
+    OccupancySensingServer,
+    OzoneConcentrationMeasurementServer,
+    Pm10ConcentrationMeasurementServer,
+    Pm1ConcentrationMeasurementServer,
+    Pm25ConcentrationMeasurementServer,
+    PowerSourceServer,
+    PowerTopologyServer,
+    PressureMeasurementServer,
+    PumpConfigurationAndControlServer,
+    RadonConcentrationMeasurementServer,
+    RelativeHumidityMeasurementServer,
+    SwitchServer,
+    TemperatureMeasurementServer,
+    ThermostatUserInterfaceConfigurationServer,
+    TimeFormatLocalizationServer,
+    TotalVolatileOrganicCompoundsConcentrationMeasurementServer,
+    UnitLocalizationServer,
+    UserLabelServer,
+} from "@matter.js/main/behaviors";
 import {
     AdministratorCommissioning,
     AirQuality,
@@ -57,13 +59,11 @@ import {
     ThermostatUserInterfaceConfiguration,
     TimeFormatLocalization,
     WindowCovering,
-} from "@project-chip/matter.js/cluster";
-import { DeviceTypeId, EndpointNumber, VendorId } from "@project-chip/matter.js/datatype";
-import { OnOffLightDevice } from "@project-chip/matter.js/devices/OnOffLightDevice";
-import { Endpoint } from "@project-chip/matter.js/endpoint";
-import { Environment, StorageService } from "@project-chip/matter.js/environment";
-import { ServerNode } from "@project-chip/matter.js/node";
-import { NumberTag } from "@project-chip/matter.js/tags/NumberTag";
+} from "@matter.js/main/clusters";
+import { OnOffLightDevice } from "@matter.js/main/devices";
+import { Environment, StorageService } from "@matter.js/main/general";
+import { Endpoint, NumberTag, ServerNode } from "@matter.js/main/node";
+import { DeviceTypeId, EndpointNumber, VendorId } from "@matter.js/main/types";
 import { TestActivatedCarbonFilterMonitoringServer } from "./cluster/TestActivatedCarbonFilterMonitoringServer.js";
 import { TestGeneralDiagnosticsServer } from "./cluster/TestGeneralDiagnosticsServer.js";
 import { TestHepaFilterMonitoringServer } from "./cluster/TestHEPAFilterMonitoringServer.js";

chip-testing/src/AllClustersTestInstanceLegacy.ts

@@ -6,7 +6,7 @@
 
 import { CommissioningServer, MatterServer } from "@project-chip/matter.js";
 
-import { Storage, StorageManager } from "@project-chip/matter.js-general";
+import { Storage, StorageManager } from "@matter.js/general";
 
 import {
     AdministratorCommissioning,

chip-testing/src/BridgeTestApp.ts

@@ -4,7 +4,7 @@
  * Copyright 2022-2024 Matter.js Authors
  * SPDX-License-Identifier: Apache-2.0
  */
-import "@project-chip/matter.js-nodejs";
+import "@matter.js/nodejs";
 import { BridgeTestInstance } from "./BridgeTestInstance.js";
 import { startTestApp } from "./GenericTestApp.js";
 import { StorageBackendAsyncJsonFile } from "./storage/StorageBackendAsyncJsonFile.js";

chip-testing/src/BridgeTestInstance.ts

@@ -4,16 +4,16 @@
  * SPDX-License-Identifier: Apache-2.0
  */
 
-import { Storage } from "@project-chip/matter.js-general";
-import { AdministratorCommissioningServer } from "@project-chip/matter.js/behavior/definitions/administrator-commissioning";
-import { BridgedDeviceBasicInformationServer } from "@project-chip/matter.js/behavior/definitions/bridged-device-basic-information";
-import { NetworkCommissioningServer } from "@project-chip/matter.js/behavior/definitions/network-commissioning";
-import { AdministratorCommissioning, BasicInformation, NetworkCommissioning } from "@project-chip/matter.js/cluster";
-import { DeviceTypeId, VendorId } from "@project-chip/matter.js/datatype";
-import { Endpoint } from "@project-chip/matter.js/endpoint";
-import { AggregatorEndpoint, DimmableLightDevice } from "@project-chip/matter.js/endpoint/definitions";
-import { Environment, StorageService } from "@project-chip/matter.js/environment";
-import { ServerNode } from "@project-chip/matter.js/node";
+import { Storage } from "@matter.js/general";
+import { AdministratorCommissioningServer } from "@matter.js/main/behaviors/administrator-commissioning";
+import { BridgedDeviceBasicInformationServer } from "@matter.js/main/behaviors/bridged-device-basic-information";
+import { NetworkCommissioningServer } from "@matter.js/main/behaviors/network-commissioning";
+import { AdministratorCommissioning, BasicInformation, NetworkCommissioning } from "@matter.js/main/clusters";
+import { DimmableLightDevice } from "@matter.js/main/devices/dimmable-light";
+import { AggregatorEndpoint } from "@matter.js/main/endpoints/aggregator";
+import { Environment, StorageService } from "@matter.js/main/general";
+import { Endpoint, ServerNode } from "@matter.js/main/node";
+import { DeviceTypeId, VendorId } from "@matter.js/main/types";
 import { TestInstance } from "./GenericTestApp.js";
 
 export class BridgeTestInstance implements TestInstance {

chip-testing/src/GenericTestApp.ts

@@ -5,9 +5,7 @@
  * SPDX-License-Identifier: Apache-2.0
  */
 
-import { ClassExtends } from "@project-chip/matter.js-general";
-import { getIntParameter, getParameter, hasParameter } from "@project-chip/matter.js-nodejs";
-import { Environment } from "@project-chip/matter.js/environment";
+import { ClassExtends, Environment } from "@matter.js/main/general";
 import { StorageBackendAsyncJsonFile } from "./storage/StorageBackendAsyncJsonFile.js";
 import { StorageBackendSyncJsonFile } from "./storage/StorageBackendSyncJsonFile.js";
 
@@ -22,17 +20,19 @@ export async function startTestApp(
     testInstanceClass: ClassExtends<TestInstance>,
     storageType: typeof StorageBackendSyncJsonFile | typeof StorageBackendAsyncJsonFile = StorageBackendSyncJsonFile,
 ) {
-    const storageName = `/tmp/chip_${getParameter("KVS") ?? "kvs"}`;
+    const vars = Environment.default.vars;
+
+    const storageName = `/tmp/chip_${vars.string("KVS") ?? "kvs"}`;
 
     const storage = new storageType(storageName);
-    if (hasParameter("factoryreset")) {
+    if (vars.boolean("factoryreset")) {
         await storage.clear();
     }
 
     const testInstance = new testInstanceClass(storage, {
         appName,
-        discriminator: getIntParameter("discriminator"),
-        passcode: getIntParameter("passcode"),
+        discriminator: vars.number("discriminator"),
+        passcode: vars.string("passcode"),
     });
 
     await testInstance.setup();

chip-testing/src/NamedPipeCommandHandler.ts

@@ -4,10 +4,9 @@
  * SPDX-License-Identifier: Apache-2.0
  */
 
-import { isObject } from "@project-chip/matter.js-general";
-import { SwitchServer } from "@project-chip/matter.js/behaviors/switch";
-import { Endpoint } from "@project-chip/matter.js/endpoint";
-import { ServerNode } from "@project-chip/matter.js/node";
+import { isObject } from "@matter.js/general";
+import { SwitchServer } from "@matter.js/main/behaviors";
+import { Endpoint, ServerNode } from "@matter.js/node";
 import { execSync } from "child_process";
 import { constants } from "node:fs";
 import { FileHandle, open, unlink } from "node:fs/promises";

chip-testing/src/TvTestApp.ts

@@ -4,7 +4,7 @@
  * Copyright 2022-2024 Matter.js Authors
  * SPDX-License-Identifier: Apache-2.0
  */
-import "@project-chip/matter.js-nodejs";
+import "@matter.js/nodejs";
 import { startTestApp } from "./GenericTestApp.js";
 import { TvTestInstance } from "./TvTestInstance.js";
 

chip-testing/src/TvTestInstance.ts

@@ -4,16 +4,15 @@
  * SPDX-License-Identifier: Apache-2.0
  */
 
-import { Bytes, Storage } from "@project-chip/matter.js-general";
-import { AdministratorCommissioningServer } from "@project-chip/matter.js/behavior/definitions/administrator-commissioning";
-import { ApplicationBasicServer } from "@project-chip/matter.js/behavior/definitions/application-basic";
-import { WakeOnLanServer } from "@project-chip/matter.js/behavior/definitions/wake-on-lan";
-import { AdministratorCommissioning, ApplicationBasic, BasicInformation } from "@project-chip/matter.js/cluster";
-import { DeviceTypeId, EndpointNumber, VendorId } from "@project-chip/matter.js/datatype";
-import { DimmableLightDevice } from "@project-chip/matter.js/devices/DimmableLightDevice";
-import { Endpoint } from "@project-chip/matter.js/endpoint";
-import { Environment, StorageService } from "@project-chip/matter.js/environment";
-import { ServerNode } from "@project-chip/matter.js/node";
+import { Bytes, Storage } from "@matter.js/general";
+import { AdministratorCommissioningServer } from "@matter.js/main/behaviors/administrator-commissioning";
+import { ApplicationBasicServer } from "@matter.js/main/behaviors/application-basic";
+import { WakeOnLanServer } from "@matter.js/main/behaviors/wake-on-lan";
+import { AdministratorCommissioning, ApplicationBasic, BasicInformation } from "@matter.js/main/clusters";
+import { DimmableLightDevice } from "@matter.js/main/devices/dimmable-light";
+import { Environment, StorageService } from "@matter.js/main/general";
+import { Endpoint, ServerNode } from "@matter.js/main/node";
+import { DeviceTypeId, EndpointNumber, VendorId } from "@matter.js/main/types";
 import { TestInstance } from "./GenericTestApp.js";
 import { TestLowPowerServer } from "./cluster/TestLowPowerServer.js";
 

chip-testing/src/cluster/TestActivatedCarbonFilterMonitoringServer.ts

@@ -3,8 +3,8 @@
  * Copyright 2022-2024 Matter.js Authors
  * SPDX-License-Identifier: Apache-2.0
  */
-import { ActivatedCarbonFilterMonitoringServer } from "@project-chip/matter.js/behavior/definitions/activated-carbon-filter-monitoring";
-import { ResourceMonitoring } from "@project-chip/matter.js/cluster";
+import { ActivatedCarbonFilterMonitoringServer } from "@matter.js/main/behaviors/activated-carbon-filter-monitoring";
+import { ResourceMonitoring } from "@matter.js/main/clusters/resource-monitoring";
 
 const TestActivatedCarbonFilterMonitoringServerBase = ActivatedCarbonFilterMonitoringServer.with(
     ResourceMonitoring.Feature.Condition,