Skip to content

Commit

Permalink
fix: core extension tutorial
Browse files Browse the repository at this point in the history
  • Loading branch information
@doctrino
doctrino committed Dec 15, 2024
1 parent 2a0603e commit c375138
Show file tree
Hide file tree
Showing 3 changed files with 143 additions and 55 deletions.
Binary file added docs/artifacts/figs/core_extension_no_connection.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
172 changes: 123 additions & 49 deletions docs/tutorials/data-modeling/core_extension.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,8 @@
"name": "stdout",
"output_type": "stream",
"text": [
"CDF <class 'tuple'> ('cdf_cdm', 'CogniteCore', 'v1') read successfully\n"
"Neat Engine 2.0.3 loaded.\n",
"Data model read passed\n"
]
},
{
Expand Down Expand Up @@ -124,7 +125,7 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": 5,
"id": "4ca48575-6876-410a-814a-ed8757ad0a9a",
"metadata": {},
"outputs": [],
Expand All @@ -137,12 +138,13 @@
"id": "5349941f-efbb-4bde-92c5-3ea4328518f3",
"metadata": {},
"source": [
"Next we will turn the `CogniteCore` model into the basis for our new model `ExtensionCore`"
"Next we will turn the `CogniteCore` model into the basis for our new model `Enterprise` model. This means \n",
"we are creating an extension from the `CogniteCore`."
]
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": 6,
"id": "0c533aa7-9b69-4a6f-98e7-a96cdfc1f65c",
"metadata": {
"editable": true,
Expand All @@ -153,15 +155,67 @@
},
"outputs": [],
"source": [
"neat.prepare.data_model.to_extension(new_data_model_id=(\"sp_doctrino_space\", \"ExtensionCore\", \"v1\"), prefix=\"MyOrg\")"
"neat.prepare.data_model.to_enterprise(\n",
" data_model_id=(\"sp_doctrino_space\", \"ExtensionCore\", \"v1\"), org_name=\"MyOrg\", move_connections=True\n",
")"
]
},
{
"cell_type": "markdown",
"id": "d24f6f4a-c5c0-462d-bc8e-06b9eb95fee7",
"metadata": {},
"source": [
"Note that we set `move_connections=True`. This repeats all connections in the `CogniteCore` model and updates the value type to match our new types.\n",
"\n",
"We can show the results by using the `show.data_model()` method."
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "309ecda9-bbcf-482c-bdd9-3e66d96ee161",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"dms_data_model.html\n"
]
},
{
"data": {
"text/html": [
"\n",
" <iframe\n",
" width=\"100%\"\n",
" height=\"750px\"\n",
" src=\"dms_data_model.html\"\n",
" frameborder=\"0\"\n",
" allowfullscreen\n",
" \n",
" ></iframe>\n",
" "
],
"text/plain": [
"<IPython.lib.display.IFrame at 0x1d568611070>"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"neat.show.data_model()"
]
},
{
"cell_type": "markdown",
"id": "e8baac0a-104b-40d7-be8e-feba27d12454",
"metadata": {},
"source": [
"The new base model is now ready. We can now directly publish it to CDF "
"The new enterprise model is now ready. We can now publish it to CDF "
]
},
{
Expand All @@ -170,61 +224,67 @@
"id": "f27e25fe-9c94-4514-b235-d27c5e713e04",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"You can inspect the details with the .inspect.outcome.data_model(...) method.\n"
]
},
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"<style type=\"text/css\">\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
"<table id=\"T_040fc\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>name</th>\n",
" <th>created</th>\n",
" <tr>\n",
" <th class=\"blank level0\" >&nbsp;</th>\n",
" <th id=\"T_040fc_level0_col0\" class=\"col_heading level0 col0\" >name</th>\n",
" <th id=\"T_040fc_level0_col1\" class=\"col_heading level0 col1\" >created</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>spaces</td>\n",
" <td>1.0</td>\n",
" <th id=\"T_040fc_level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
" <td id=\"T_040fc_row0_col0\" class=\"data row0 col0\" >schema</td>\n",
" <td id=\"T_040fc_row0_col1\" class=\"data row0 col1\" >0</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>containers</td>\n",
" <td>NaN</td>\n",
" <th id=\"T_040fc_level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
" <td id=\"T_040fc_row1_col0\" class=\"data row1 col0\" >spaces</td>\n",
" <td id=\"T_040fc_row1_col1\" class=\"data row1 col1\" >1</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>views</td>\n",
" <td>11.0</td>\n",
" <th id=\"T_040fc_level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
" <td id=\"T_040fc_row2_col0\" class=\"data row2 col0\" >containers</td>\n",
" <td id=\"T_040fc_row2_col1\" class=\"data row2 col1\" >11</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>data_models</td>\n",
" <td>1.0</td>\n",
" <th id=\"T_040fc_level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
" <td id=\"T_040fc_row3_col0\" class=\"data row3 col0\" >views</td>\n",
" <td id=\"T_040fc_row3_col1\" class=\"data row3 col1\" >11</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_040fc_level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
" <td id=\"T_040fc_row4_col0\" class=\"data row4 col0\" >data_models</td>\n",
" <td id=\"T_040fc_row4_col1\" class=\"data row4 col1\" >1</td>\n",
" </tr>\n",
" <tr>\n",
" <th id=\"T_040fc_level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
" <td id=\"T_040fc_row5_col0\" class=\"data row5 col0\" >nodes</td>\n",
" <td id=\"T_040fc_row5_col1\" class=\"data row5 col1\" >0</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
"</table>\n"
],
"text/plain": [
"[UploadResult(name='spaces', error_messages=[], issues=[], created={'sp_doctrino_space'}, upserted=set(), deleted=set(), changed=set(), unchanged=set(), skipped=set(), failed_created=set(), failed_upserted=set(), failed_changed=set(), failed_deleted=set()),\n",
" UploadResult(name='containers', error_messages=[], issues=[], created=set(), upserted=set(), deleted=set(), changed=set(), unchanged=set(), skipped=set(), failed_created=set(), failed_upserted=set(), failed_changed=set(), failed_deleted=set()),\n",
" UploadResult(name='views', error_messages=[], issues=[], created={ViewId(space='sp_doctrino_space', external_id='MyOrgUnit', version='v1'), ViewId(space='sp_doctrino_space', external_id='MyOrgAsset', version='v1'), ViewId(space='sp_doctrino_space', external_id='MyOrgAssetType', version='v1'), ViewId(space='sp_doctrino_space', external_id='MyOrgActivity', version='v1'), ViewId(space='sp_doctrino_space', external_id='MyOrgEquipment', version='v1'), ViewId(space='sp_doctrino_space', external_id='MyOrgEquipmentType', version='v1'), ViewId(space='sp_doctrino_space', external_id='MyOrgSourceSystem', version='v1'), ViewId(space='sp_doctrino_space', external_id='MyOrgFileCategory', version='v1'), ViewId(space='sp_doctrino_space', external_id='MyOrgTimeSeries', version='v1'), ViewId(space='sp_doctrino_space', external_id='MyOrgFile', version='v1'), ViewId(space='sp_doctrino_space', external_id='MyOrgAssetClass', version='v1')}, upserted=set(), deleted=set(), changed=set(), unchanged=set(), skipped=set(), failed_created=set(), failed_upserted=set(), failed_changed=set(), failed_deleted=set()),\n",
" UploadResult(name='data_models', error_messages=[], issues=[], created={DataModelId(space='sp_doctrino_space', external_id='ExtensionCore', version='v1')}, upserted=set(), deleted=set(), changed=set(), unchanged=set(), skipped=set(), failed_created=set(), failed_upserted=set(), failed_changed=set(), failed_deleted=set())]"
"[UploadResultCore(name='schema', error_messages=[], issues=[]),\n",
" UploadResult(name='spaces', error_messages=[], issues=[], created={'sp_doctrino_space'}, upserted=set(), deleted=set(), changed=set(), unchanged=set(), skipped=set(), failed_created=set(), failed_upserted=set(), failed_changed=set(), failed_deleted=set()),\n",
" UploadResult(name='containers', error_messages=[], issues=[], created={ContainerId(space='sp_doctrino_space', external_id='MyOrgAsset'), ContainerId(space='sp_doctrino_space', external_id='MyOrgActivity'), ContainerId(space='sp_doctrino_space', external_id='MyOrgAssetClass'), ContainerId(space='sp_doctrino_space', external_id='MyOrgFileCategory'), ContainerId(space='sp_doctrino_space', external_id='MyOrgEquipmentType'), ContainerId(space='sp_doctrino_space', external_id='MyOrgFile'), ContainerId(space='sp_doctrino_space', external_id='MyOrgAssetType'), ContainerId(space='sp_doctrino_space', external_id='MyOrgUnit'), ContainerId(space='sp_doctrino_space', external_id='MyOrgEquipment'), ContainerId(space='sp_doctrino_space', external_id='MyOrgTimeSeries'), ContainerId(space='sp_doctrino_space', external_id='MyOrgSourceSystem')}, upserted=set(), deleted=set(), changed=set(), unchanged=set(), skipped=set(), failed_created=set(), failed_upserted=set(), failed_changed=set(), failed_deleted=set()),\n",
" UploadResult(name='views', error_messages=[], issues=[], created={ViewId(space='sp_doctrino_space', external_id='MyOrgEquipment', version='v1'), ViewId(space='sp_doctrino_space', external_id='MyOrgFileCategory', version='v1'), ViewId(space='sp_doctrino_space', external_id='MyOrgEquipmentType', version='v1'), ViewId(space='sp_doctrino_space', external_id='MyOrgTimeSeries', version='v1'), ViewId(space='sp_doctrino_space', external_id='MyOrgSourceSystem', version='v1'), ViewId(space='sp_doctrino_space', external_id='MyOrgAssetType', version='v1'), ViewId(space='sp_doctrino_space', external_id='MyOrgFile', version='v1'), ViewId(space='sp_doctrino_space', external_id='MyOrgActivity', version='v1'), ViewId(space='sp_doctrino_space', external_id='MyOrgAsset', version='v1'), ViewId(space='sp_doctrino_space', external_id='MyOrgAssetClass', version='v1'), ViewId(space='sp_doctrino_space', external_id='MyOrgUnit', version='v1')}, upserted=set(), deleted=set(), changed=set(), unchanged=set(), skipped=set(), failed_created=set(), failed_upserted=set(), failed_changed=set(), failed_deleted=set()),\n",
" UploadResult(name='data_models', error_messages=[], issues=[], created={DataModelId(space='sp_doctrino_space', external_id='ExtensionCore', version='v1')}, upserted=set(), deleted=set(), changed=set(), unchanged=set(), skipped=set(), failed_created=set(), failed_upserted=set(), failed_changed=set(), failed_deleted=set()),\n",
" UploadResult(name='nodes', error_messages=[], issues=[], created=set(), upserted=set(), deleted=set(), changed=set(), unchanged=set(), skipped=set(), failed_created=set(), failed_upserted=set(), failed_changed=set(), failed_deleted=set())]"
]
},
"execution_count": 9,
Expand Down Expand Up @@ -266,7 +326,7 @@
},
{
"cell_type": "code",
"execution_count": 11,
"execution_count": 10,
"id": "06df8a88-2afa-49dc-8df3-d29987e7a988",
"metadata": {
"editable": true,
Expand All @@ -277,7 +337,7 @@
},
"outputs": [],
"source": [
"neat.to.excel(\"my_extension.xlsx\")"
"neat.to.excel(\"my_extension.xlsx\", model=\"dms\")"
]
},
{
Expand Down Expand Up @@ -541,7 +601,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 20,
"id": "888713e2-a68e-4b83-ae8e-36a1563c6e61",
"metadata": {
"editable": true,
Expand All @@ -552,13 +612,27 @@
"hide"
]
},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Deleted space []\n"
]
}
],
"source": [
"# This cell is hidden so it does not show in the docs.\n",
"from cognite.neat._utils.cdf.loaders import SpaceLoader\n",
"\n",
"SpaceLoader(client).clean(\"sp_doctrino_space\");"
"neat._client.loaders.spaces.clean(\"sp_doctrino_space\");"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b61e33fe-a45a-44af-9f17-7d164daf9ea1",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
Expand Down
26 changes: 20 additions & 6 deletions docs/tutorials/data-modeling/dms_data_model.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -261,13 +261,27 @@ <h1></h1>
>
<option selected>Select a Node by ID</option>

<option value="Documentation">Documentation</option>
<option value="MyOrgActivity">MyOrgActivity</option>

<option value="Pump">Pump</option>
<option value="MyOrgAsset">MyOrgAsset</option>

<option value="Facility">Facility</option>
<option value="MyOrgEquipment">MyOrgEquipment</option>

<option value="CogniteTimeSeries">CogniteTimeSeries</option>
<option value="MyOrgSourceSystem">MyOrgSourceSystem</option>

<option value="MyOrgTimeSeries">MyOrgTimeSeries</option>

<option value="MyOrgAssetClass">MyOrgAssetClass</option>

<option value="MyOrgFile">MyOrgFile</option>

<option value="MyOrgAssetType">MyOrgAssetType</option>

<option value="MyOrgEquipmentType">MyOrgEquipmentType</option>

<option value="MyOrgFileCategory">MyOrgFileCategory</option>

<option value="MyOrgUnit">MyOrgUnit</option>

</select>
</div>
Expand Down Expand Up @@ -321,8 +335,8 @@ <h1></h1>


// parsing and collecting nodes and edges from the python
nodes = new vis.DataSet([{"color": "#97c2fc", "id": "Documentation", "label": "Documentation", "shape": "dot", "size": 10}, {"color": "#97c2fc", "id": "Pump", "label": "Pump", "shape": "dot", "size": 10}, {"color": "#97c2fc", "id": "Facility", "label": "Facility", "shape": "dot", "size": 10}, {"color": "#97c2fc", "id": "CogniteTimeSeries", "label": "CogniteTimeSeries", "shape": "dot", "size": 10}]);
edges = new vis.DataSet([{"arrows": "to", "from": "Documentation", "label": "relatedPumps", "to": "Pump", "width": 1}, {"arrows": "to", "from": "Facility", "label": "hasPumps", "to": "Pump", "width": 1}, {"arrows": "to", "from": "Pump", "label": "documentation", "to": "Documentation", "width": 1}, {"arrows": "to", "from": "Pump", "label": "livesIn", "to": "Facility", "width": 1}, {"arrows": "to", "from": "Pump", "label": "temperature", "to": "CogniteTimeSeries", "width": 1}]);
nodes = new vis.DataSet([{"color": "#97c2fc", "id": "MyOrgActivity", "label": "MyOrgActivity", "shape": "dot", "size": 10}, {"color": "#97c2fc", "id": "MyOrgAsset", "label": "MyOrgAsset", "shape": "dot", "size": 10}, {"color": "#97c2fc", "id": "MyOrgEquipment", "label": "MyOrgEquipment", "shape": "dot", "size": 10}, {"color": "#97c2fc", "id": "MyOrgSourceSystem", "label": "MyOrgSourceSystem", "shape": "dot", "size": 10}, {"color": "#97c2fc", "id": "MyOrgTimeSeries", "label": "MyOrgTimeSeries", "shape": "dot", "size": 10}, {"color": "#97c2fc", "id": "MyOrgAssetClass", "label": "MyOrgAssetClass", "shape": "dot", "size": 10}, {"color": "#97c2fc", "id": "MyOrgFile", "label": "MyOrgFile", "shape": "dot", "size": 10}, {"color": "#97c2fc", "id": "MyOrgAssetType", "label": "MyOrgAssetType", "shape": "dot", "size": 10}, {"color": "#97c2fc", "id": "MyOrgEquipmentType", "label": "MyOrgEquipmentType", "shape": "dot", "size": 10}, {"color": "#97c2fc", "id": "MyOrgFileCategory", "label": "MyOrgFileCategory", "shape": "dot", "size": 10}, {"color": "#97c2fc", "id": "MyOrgUnit", "label": "MyOrgUnit", "shape": "dot", "size": 10}]);
edges = new vis.DataSet([{"arrows": "to", "from": "MyOrgActivity", "label": "Assets", "to": "MyOrgAsset", "width": 1}, {"arrows": "to", "from": "MyOrgActivity", "label": "Equipment", "to": "MyOrgEquipment", "width": 1}, {"arrows": "to", "from": "MyOrgActivity", "label": "source", "to": "MyOrgSourceSystem", "width": 1}, {"arrows": "to", "from": "MyOrgActivity", "label": "Time series", "to": "MyOrgTimeSeries", "width": 1}, {"arrows": "to", "from": "MyOrgAsset", "label": "Activities", "to": "MyOrgActivity", "width": 1}, {"arrows": "to", "from": "MyOrgAsset", "label": "Asset class", "to": "MyOrgAssetClass", "width": 1}, {"arrows": "to", "from": "MyOrgAsset", "label": "Root", "to": "MyOrgAsset", "width": 1}, {"arrows": "to", "from": "MyOrgAsset", "label": "Equipment", "to": "MyOrgEquipment", "width": 1}, {"arrows": "to", "from": "MyOrgAsset", "label": "Files", "to": "MyOrgFile", "width": 1}, {"arrows": "to", "from": "MyOrgAsset", "label": "source", "to": "MyOrgSourceSystem", "width": 1}, {"arrows": "to", "from": "MyOrgAsset", "label": "Time series", "to": "MyOrgTimeSeries", "width": 1}, {"arrows": "to", "from": "MyOrgAsset", "label": "Asset type", "to": "MyOrgAssetType", "width": 1}, {"arrows": "to", "from": "MyOrgAssetType", "label": "Asset class", "to": "MyOrgAssetClass", "width": 1}, {"arrows": "to", "from": "MyOrgEquipment", "label": "Activities", "to": "MyOrgActivity", "width": 1}, {"arrows": "to", "from": "MyOrgEquipment", "label": "Asset", "to": "MyOrgAsset", "width": 1}, {"arrows": "to", "from": "MyOrgEquipment", "label": "Equipment type", "to": "MyOrgEquipmentType", "width": 1}, {"arrows": "to", "from": "MyOrgEquipment", "label": "Files", "to": "MyOrgFile", "width": 1}, {"arrows": "to", "from": "MyOrgEquipment", "label": "source", "to": "MyOrgSourceSystem", "width": 1}, {"arrows": "to", "from": "MyOrgEquipment", "label": "Time series", "to": "MyOrgTimeSeries", "width": 1}, {"arrows": "to", "from": "MyOrgFile", "label": "Assets", "to": "MyOrgAsset", "width": 1}, {"arrows": "to", "from": "MyOrgFile", "label": "Category", "to": "MyOrgFileCategory", "width": 1}, {"arrows": "to", "from": "MyOrgFile", "label": "Equipment", "to": "MyOrgEquipment", "width": 1}, {"arrows": "to", "from": "MyOrgFile", "label": "source", "to": "MyOrgSourceSystem", "width": 1}, {"arrows": "to", "from": "MyOrgTimeSeries", "label": "Activities", "to": "MyOrgActivity", "width": 1}, {"arrows": "to", "from": "MyOrgTimeSeries", "label": "Assets", "to": "MyOrgAsset", "width": 1}, {"arrows": "to", "from": "MyOrgTimeSeries", "label": "Equipment", "to": "MyOrgEquipment", "width": 1}, {"arrows": "to", "from": "MyOrgTimeSeries", "label": "source", "to": "MyOrgSourceSystem", "width": 1}, {"arrows": "to", "from": "MyOrgTimeSeries", "label": "Unit", "to": "MyOrgUnit", "width": 1}]);

nodeColors = {};
allNodes = nodes.get({ returnType: "Object" });
Expand Down

0 comments on commit c375138

Please sign in to comment.