Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

README files improvments: Analytical store, limitatios, workarounds. #13188

Merged
merged 4 commits into from
Aug 19, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions sdk/cosmos/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Use the Azure Cosmos DB SQL API SDKs for application development and database ma

## Contents of this folder

### azure-cosmos
### [azure-cosmos](./azure-cosmos)

This is the Azure Cosmos DB SQL API SDK for Python to manage databases and the JSON documents they contain in this NoSQL database service. High level capabilities are:

Expand All @@ -17,7 +17,7 @@ This is the Azure Cosmos DB SQL API SDK for Python to manage databases and the J

Use this package if you are creating an application or exploring data.

### azure-mgmt-cosmosdb
### [azure-mgmt-cosmosdb](./azure-mgmt-cosmosdb)

This is the Microsoft Azure Cosmos DB Management Client Library. High level capabilities are:

Expand All @@ -34,7 +34,7 @@ This is the Microsoft Azure Cosmos DB Management Client Library. High level capa

Use this package if you are creating an account level management application.

### azure-mgmt-documentdb
### [azure-mgmt-documentdb](./azure-mgmt-documentdb)

Legacy DocumentDB SDK.

Expand Down
42 changes: 41 additions & 1 deletion sdk/cosmos/azure-cosmos/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,16 @@ Use the Azure Cosmos DB SQL API SDK for Python to manage databases and the JSON

[SDK source code][source_code] | [Package (PyPI)][cosmos_pypi] | [API reference documentation][ref_cosmos_sdk] | [Product documentation][cosmos_docs] | [Samples][cosmos_samples]

> This SDK is used for the [SQL API](https://docs.microsoft.com/en-us/azure/cosmos-db/sql-query-getting-started). For all other APIs, please check the [Azure Cosmos DB documentation](https://docs.microsoft.com/en-us/azure/cosmos-db/introduction) to evaluate the best SDK for your project.

## Getting started

### Prerequisites

* Azure subscription - [Create a free account][azure_sub]
* Azure [Cosmos DB account][cosmos_account] - SQL API
* [Python 2.7 or 3.5.3+][python]


If you need a Cosmos DB SQL API account, you can create one with this [Azure CLI][azure_cli] command:

```Bash
Expand All @@ -39,6 +41,7 @@ Although not required, you can keep your base system and Azure SDK environments
python3 -m venv azure-cosmosdb-sdk-environment
source azure-cosmosdb-sdk-environment/bin/activate
```

### Authenticate the client

Interaction with Cosmos DB starts with an instance of the [CosmosClient][ref_cosmosclient] class. You need an **account**, its **URI**, and one of its **account keys** to instantiate the client object.
Expand All @@ -52,6 +55,7 @@ ACCT_NAME=<cosmos-db-account-name>
export ACCOUNT_URI=$(az cosmosdb show --resource-group $RES_GROUP --name $ACCT_NAME --query documentEndpoint --output tsv)
export ACCOUNT_KEY=$(az cosmosdb list-keys --resource-group $RES_GROUP --name $ACCT_NAME --query primaryMasterKey --output tsv)
```

### Create the client

Once you've populated the `ACCOUNT_URI` and `ACCOUNT_KEY` environment variables, you can create the [CosmosClient][ref_cosmosclient].
Expand All @@ -77,12 +81,26 @@ Once you've initialized a [CosmosClient][ref_cosmosclient], you can interact wit

For more information about these resources, see [Working with Azure Cosmos databases, containers and items][cosmos_resources].

## Limitations

As of August 2020 the features below are not yet supported.

* Bulk/Batch processing
* Group By queries
* Direct TCP Mode access
* Language Native async i/o

## Limitations Workaround

If you want to use Python SDK to perform bulk inserts to Cosmos DB, the best alternative is to use [stored procedures](https://docs.microsoft.com/en-us/azure/cosmos-db/how-to-write-stored-procedures-triggers-udfs) to write multiple items with the same partition key.

## Examples

The following sections provide several code snippets covering some of the most common Cosmos DB tasks, including:

* [Create a database](#create-a-database "Create a database")
* [Create a container](#create-a-container "Create a container")
* [Create an Analytical Store Enabled container](#create-an-analytical-store-enabled-container "Create a container")
* [Get an existing container](#get-an-existing-container "Get an existing container")
* [Insert data](#insert-data "Insert data")
* [Delete data](#delete-data "Delete data")
Expand Down Expand Up @@ -131,6 +149,28 @@ except exceptions.CosmosHttpResponseError:
raise
```

### Create an Analytical Store enabled container

This example creates a container with [Analytical Store](https://docs.microsoft.com/en-us/azure/cosmos-db/analytical-store-introduction) enabled, for reporting, BI, AI, and Advanced Analytics with [Azure Synapse Link](https://docs.microsoft.com/en-us/azure/cosmos-db/synapse-link).

Options:

+ 0 or Null = Not enabled.
+ -1 = The data will be stored infinitely.
+ Any other number is the actual ttl, in seconds.

```Python
container_name = 'products'
try:
container = database.create_container(id=container_name, partition_key=PartitionKey(path="/productName"),analytical_storage_ttl=-1)
except exceptions.CosmosResourceExistsError:
container = database.get_container_client(container_name)
except exceptions.CosmosHttpResponseError:
raise
```

The preceding snippet also handles the [CosmosHttpResponseError][ref_httpfailure] exception if the container creation failed. For more information on error handling and troubleshooting, see the [Troubleshooting](#troubleshooting "Troubleshooting") section.

The preceding snippet also handles the [CosmosHttpResponseError][ref_httpfailure] exception if the container creation failed. For more information on error handling and troubleshooting, see the [Troubleshooting](#troubleshooting "Troubleshooting") section.

### Get an existing container
Expand Down
32 changes: 23 additions & 9 deletions sdk/cosmos/azure-cosmos/samples/container_management.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
# 2.4 - Create container with unique key
# 2.5 - Create Container with partition key V2
# 2.6 - Create Container with partition key V1
# 2.7 - Create Container with analytical store enabled
#
# 3. Manage Container Provisioned Throughput
# 3.1 - Get Container provisioned throughput (RU/s)
Expand Down Expand Up @@ -67,8 +68,8 @@ def find_container(db, id):


def create_container(db, id):
""" Execute the most basic Create of container.
This will create a container with 400 RUs throughput and default indexing policy """
""" Execute basic container creation.
This will create containers with 400 RUs with different indexing, partitioning, and storage options """

partition_key = PartitionKey(path='/id', kind='Hash')
print("\n2.1 Create Container - Basic")
Expand All @@ -84,9 +85,8 @@ def create_container(db, id):

try:
coll = {
"id": "container_custom_index_policy",
"id": id+"_container_custom_index_policy",
"indexingPolicy": {
"indexingMode": "lazy",
"automatic": False
}
}
Expand All @@ -107,9 +107,8 @@ def create_container(db, id):
print("\n2.3 Create Container - With custom provisioned throughput")

try:
coll = {"id": "container_custom_throughput"}
container = db.create_container(
id=coll['id'],
id=id+"_container_custom_throughput",
partition_key=partition_key,
offer_throughput=400
)
Expand All @@ -122,7 +121,7 @@ def create_container(db, id):

try:
container = db.create_container(
id="container_unique_keys",
id= id+"_container_unique_keys",
partition_key=partition_key,
unique_key_policy={'uniqueKeys': [{'paths': ['/field1/field2', '/field3']}]}
)
Expand All @@ -138,7 +137,7 @@ def create_container(db, id):

try:
container = db.create_container(
id="container_partition_key_v2",
id=id+"_container_partition_key_v2",
partition_key=PartitionKey(path='/id', kind='Hash')
)
properties = container.read()
Expand All @@ -152,7 +151,7 @@ def create_container(db, id):

try:
container = db.create_container(
id="container_partition_key_v1",
id=id+"_container_partition_key_v1",
partition_key=PartitionKey(path='/id', kind='Hash', version=1)
)
properties = container.read()
Expand All @@ -162,6 +161,21 @@ def create_container(db, id):
except exceptions.CosmosResourceExistsError:
print('A container with id \'container_partition_key_v1\' already exists')

print("\n2.7 Create Container - With analytical store enabled")

try:
container = db.create_container(
id=id+"_container_analytical_store",
partition_key=PartitionKey(path='/id', kind='Hash',analytical_storage_ttl=-1)
)
properties = container.read()
print('Container with id \'{0}\' created'.format(container.id))
print('Partition Key - \'{0}\''.format(properties['partitionKey']))

except exceptions.CosmosResourceExistsError:
print('A container with id \'_container_analytical_store\' already exists')



def manage_provisioned_throughput(db, id):
print("\n3.1 Get Container provisioned throughput (RU/s)")
Expand Down
1 change: 0 additions & 1 deletion sdk/cosmos/azure-cosmos/samples/index_management.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@
# excludedPaths
#
# We can toggle 'automatic' to eiher be True or False depending upon whether we want to have indexing over all columns by default or not.
# indexingMode can be either of consistent, lazy or none
#
# We can provide options while creating documents. indexingDirective is one such,
# by which we can tell whether it should be included or excluded in the index of the parent container.
Expand Down