Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Tables] Track2 tables #15276

Merged
merged 45 commits into from
Sep 2, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
19da635
Initial generation of the aztables module (#14441)
christothes Apr 5, 2021
3e724e2
Update ci.yml
christothes Apr 5, 2021
6ddf807
Introduce internal/testFramework and test aztables with it (#14491)
christothes Apr 21, 2021
1559e92
sanitizer fixes (#14600)
catalinaperalta Apr 28, 2021
fed39eb
Progress on aztables (#14676)
christothes May 21, 2021
b606abd
AsModels
christothes May 28, 2021
8e10930
rename
christothes May 28, 2021
c39e2ca
comments
christothes Jun 1, 2021
c9d0dd3
comment tweaks, options as pointer
christothes Jun 3, 2021
b427598
add EntityMapAsModel
christothes Jun 3, 2021
9747f1d
refactors to AddEntity
christothes Jun 3, 2021
02b61fb
update some code docs
christothes Jun 4, 2021
d21b634
doc fixes
christothes Jun 4, 2021
1b06de5
doc fixes
christothes Jun 4, 2021
09e4fb6
doc improvements
christothes Jun 4, 2021
63e0566
rename testframework to recording
christothes Jun 22, 2021
67fa153
move ci.yml
christothes Jun 28, 2021
e743034
ci path
christothes Jun 28, 2021
f2d1250
ci path
christothes Jun 28, 2021
274d7ac
[Tables] Update from testframework -> recording (#14888)
seankane-msft Jun 30, 2021
ef70309
[Tables] ci fix (#14956)
seankane-msft Jul 1, 2021
31a7267
[Tables] Add post-request validation of entity (#14960)
seankane-msft Jul 2, 2021
6bc983b
[Tables] changing QueryOptions param to be a ptr (#14961)
seankane-msft Jul 8, 2021
3d6b5ce
[Tables] linting Fixes (#15028)
seankane-msft Jul 9, 2021
658d8c5
[Tables] Adding swagger.md (#15016)
seankane-msft Jul 14, 2021
8d1167e
[Tables] Add get/set access policy (#15027)
seankane-msft Jul 20, 2021
88aa7f2
[Tables] Add GetStats, GetProperties, and SetProperties methods (#15112)
seankane-msft Jul 21, 2021
e224444
[Tables] Add support for Entity struct (#15003)
seankane-msft Jul 26, 2021
f905d11
[Tables] change Query to List (#15099)
seankane-msft Jul 26, 2021
6e69790
[Tables] Update transactions method (#15125)
seankane-msft Jul 26, 2021
e165488
Adding Table suffix to Create, Delete, and List methods on service cl…
seankane-msft Jul 28, 2021
4723b6d
[Tables] Architect Feedback (#15196)
seankane-msft Aug 3, 2021
4aa5a96
[EngSys] Test proxy (#15101)
seankane-msft Aug 12, 2021
a8c839f
[Tables] Add authentication via SAS credential (#15256)
seankane-msft Aug 16, 2021
ee892d5
Merge branch 'main' into track2-tables
seankane-msft Aug 18, 2021
ecb0e72
fixing internal (#15315)
seankane-msft Aug 18, 2021
b1fe68c
Fixes2 (#15316)
seankane-msft Aug 18, 2021
088d5b0
[Tables] bump deps (#15350)
seankane-msft Aug 20, 2021
7e6523f
[Tables] connection string parser (#15187)
seankane-msft Aug 24, 2021
c9c467b
[Tables] simplify code space by moving generated to `internal` module…
seankane-msft Sep 1, 2021
e627020
Merge branch 'main' into track2-tables
seankane-msft Sep 1, 2021
ed6e5b6
[Tables] Docs and cleanup (#15428)
seankane-msft Sep 2, 2021
94396e5
[Tables]Rename package (#15448)
seankane-msft Sep 2, 2021
151ce4b
Merge branch 'main' into track2-tables
seankane-msft Sep 2, 2021
71d5aea
[Tables] Merging internal (#15451)
seankane-msft Sep 2, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion eng/config.json
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,11 @@
},
{
"Name": "internal",
"CoverageGoal": 0.90
"CoverageGoal": 0.70
},
{
"Name": "data",
"CoverageGoal": 0.40
}
]
}
4 changes: 3 additions & 1 deletion eng/pipelines/templates/steps/build-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,8 @@ steps:
- pwsh: ../eng/scripts/create_coverage.ps1 ${{parameters.ServiceDirectory}}
displayName: 'Generate Coverage XML'
workingDirectory: '${{parameters.GoWorkspace}}sdk'
env:
GO111MODULE: 'off'

- task: PublishTestResults@2
condition: succeededOrFailed()
Expand All @@ -86,4 +88,4 @@ steps:
codeCoverageTool: Cobertura
summaryFileLocation: '${{parameters.GoWorkspace}}sdk/${{parameters.ServiceDirectory}}/coverage.xml'
additionalCodeCoverageFiles: '${{parameters.GoWorkspace}}sdk/${{parameters.ServiceDirectory}}/coverage.html'
failIfCoverageEmpty: true
failIfCoverageEmpty: true
30 changes: 30 additions & 0 deletions sdk/data/autorest.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
## Go

These settings apply only when `--go` is specified on the command line.

<!-- Original autorest command used by Chris Scott -->
<!-- autorest --use=@autorest/go@4.0.0-preview.20 https://raw.githubusercontent.com/Azure/azure-rest-api-specs/master/specification/cosmos-db/data-plane/readme.md --tag=package-2019-02 --file-prefix="zz_generated_" --modelerfour.lenient-model-deduplication --license-header=MICROSOFT_MIT_NO_VERSION --output-folder=aztables --module=aztables --openapi-type="data-plane" --credential-scope=none -->

``` yaml
go: true
version: "^3.0.0"
input-file: https://github.com/Azure/azure-rest-api-specs/blob/d744b6bcb95ab4034832ded556dbbe58f4287c5b/specification/cosmos-db/data-plane/Microsoft.Tables/preview/2019-02-02/table.json
license-header: MICROSOFT_MIT_NO_VERSION
clear-output-folder: false
output-folder: aztables/internal
tag: package-2019-02
credential-scope: none
use: "@autorest/go@4.0.0-preview.27"
module-version: 0.1.0
security: "AADToken"
security-scopes: "https://storage.azure.com/.default"
modelerfour:
group-parameters: false
```

### Go multi-api

``` yaml $(go) && $(multiapi)
batch:
- tag: package-2019-02
```
4 changes: 4 additions & 0 deletions sdk/data/aztables/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Release History

## v0.1.0 (09-07-2021)
* This is the initial release of the `aztables` library
21 changes: 21 additions & 0 deletions sdk/data/aztables/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) Microsoft Corporation. All rights reserved.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE
151 changes: 151 additions & 0 deletions sdk/data/aztables/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
# Azure Tables client library for Go

Azure Table storage is a service that stores large amounts of structured NoSQL data in the cloud, providing
a key/attribute store with a schema-less design.

Azure Cosmos DB provides a Table API for applications that are written for Azure Table storage that need premium capabilities like:

- Turnkey global distribution.
- Dedicated throughput worldwide.
- Single-digit millisecond latencies at the 99th percentile.
- Guaranteed high availability.
- Automatic secondary indexing.

The Azure Tables client library can seamlessly target either Azure Table storage or Azure Cosmos DB table service endpoints with no code changes.

## Getting started

### Install the package
Install the Azure Tables client library for Go :

### Prerequisites
* An [Azure subscription][azure_sub].
* An existing Azure storage account or Azure Cosmos DB database with Azure Table API specified.

If you need to create either of these, you can use the [Azure CLI][azure_cli].

#### Creating a storage account

Create a storage account `mystorageaccount` in resource group `MyResourceGroup`
in the subscription `MySubscription` in the West US region.


#### Creating a Cosmos DB

Create a Cosmos DB account `MyCosmosDBDatabaseAccount` in resource group `MyResourceGroup`
in the subscription `MySubscription` and a table named `MyTableName` in the account.


### Authenticate the Client

Learn more about options for authentication _(including Connection Strings, Shared Key, and Shared Key Signatures)_ in our samples.

## Key concepts

- `TableServiceClient` - Client that provides methods to interact at the Table Service level such as creating, listing, and deleting tables
- `TableClient` - Client that provides methods to interact at an table entity level such as creating, querying, and deleting entities within a table.
- `Table` - Tables store data as collections of entities.
- `Entity` - Entities are similar to rows. An entity has a primary key and a set of properties. A property is a name value pair, similar to a column.

Common uses of the Table service include:

- Storing TBs of structured data capable of serving web scale applications
- Storing datasets that don't require complex joins, foreign keys, or stored procedures and can be de-normalized for fast access
- Quickly querying data using a clustered index
### Create the Table service client

First, we need to construct a `TableServiceClient`.

### Create an Azure table
Next, we can create a new table.


### Get an Azure table
The set of existing Azure tables can be queries using an OData filter.


### Delete an Azure table

Individual tables can be deleted from the service.


### Create the Table client

To interact with table entities, we must first construct a `TableClient`.


### Add table entities

Let's define a new `TableEntity` so that we can add it to the table.

Using the `TableClient` we can now add our new entity to the table.


### Query table entities

To inspect the set of existing table entities, we can query the table using an OData filter.


If you prefer LINQ style query expressions, we can query the table using that syntax as well.


### Delete table entities

If we no longer need our new table entity, it can be deleted.


## Troubleshooting

When you interact with the Azure table library using the .NET SDK, errors returned by the service correspond to the same HTTP
status codes returned for [REST API][tables_rest] requests.

For example, if you try to create a table that already exists, a `409` error is returned, indicating "Conflict".


### Setting up console logging

The simplest way to see the logs is to enable the console logging.
To create an Azure SDK log listener that outputs messages to console use AzureEventSourceListener.CreateConsoleLogger method.


To learn more about other logging mechanisms see [here][logging].

## Next steps

Get started with our [Table samples][table_client_samples].

## Known Issues

A list of currently known issues relating to Cosmos DB table endpoints can be found [here](https://aka.ms/tablesknownissues).

## Contributing

This project welcomes contributions and suggestions. Most contributions require
you to agree to a Contributor License Agreement (CLA) declaring that you have
the right to, and actually do, grant us the rights to use your contribution. For
details, visit [cla.microsoft.com][cla].

This project has adopted the [Microsoft Open Source Code of Conduct][coc].
For more information see the [Code of Conduct FAQ][coc_faq] or contact
[opencode@microsoft.com][coc_contact] with any additional questions or comments.

## Generating the client

From the tables dir:

autorest --use=@autorest/go@4.0.0-preview.20 https://raw.githubusercontent.com/Azure/azure-rest-api-specs/master/specification/cosmos-db/data-plane/readme.md --tag=package-2019-02 --file-prefix="zz_generated_" --modelerfour.lenient-model-deduplication --license-header=MICROSOFT_MIT_NO_VERSION --output-folder=aztables --module=aztables --openapi-type="data-plane" --credential-scope=none

<!-- LINKS -->
[tables_rest]: https://docs.microsoft.com/rest/api/storageservices/table-service-rest-api
[azure_cli]: https://docs.microsoft.com/cli/azure
[azure_sub]: https://azure.microsoft.com/free/
[table_client_nuget_package]: https://www.nuget.org/packages?q=Azure.Data.Tables
[table_client_samples]: https://github\.com/Azure/azure-sdk-for-go
[table_client_src]: https://github\.com/Azure/azure-sdk-for-go
[logging]: https://github\.com/Azure/azure-sdk-for-go
[cla]: https://cla.microsoft.com
[coc]: https://opensource.microsoft.com/codeofconduct/
[coc_faq]: https://opensource.microsoft.com/codeofconduct/faq/
[coc_contact]: mailto:opencode@microsoft.com

![Impressions](https://azure-sdk-impressions.azurewebsites.net/api/impressions/azure-sdk-for-net%2Fsdk%2Ftables%2FAzure.Data.Tables%2FREADME.png)
151 changes: 151 additions & 0 deletions sdk/data/aztables/access_policy_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

package aztables

import (
"strconv"
"testing"
"time"

"github.com/stretchr/testify/require"
)

func TestSetEmptyAccessPolicy(t *testing.T) {
client, delete := initClientTest(t, "storage", true)
defer delete()

_, err := client.SetAccessPolicy(ctx, &SetAccessPolicyOptions{})
require.NoError(t, err)
}

func TestSetAccessPolicy(t *testing.T) {
client, delete := initClientTest(t, "storage", true)
defer delete()

start := time.Date(2020, 1, 1, 0, 0, 0, 0, time.UTC)
expiration := time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)
permission := "r"
id := "1"

signedIdentifiers := make([]*SignedIdentifier, 0)

signedIdentifiers = append(signedIdentifiers, &SignedIdentifier{
AccessPolicy: &AccessPolicy{
Expiry: &expiration,
Start: &start,
Permission: &permission,
},
ID: &id,
})

param := SetAccessPolicyOptions{
TableACL: signedIdentifiers,
}

_, err := client.SetAccessPolicy(ctx, &param)
require.NoError(t, err)
}

func TestSetMultipleAccessPolicies(t *testing.T) {
client, delete := initClientTest(t, "storage", true)
defer delete()

id := "empty"

signedIdentifiers := make([]*SignedIdentifier, 0)
signedIdentifiers = append(signedIdentifiers, &SignedIdentifier{
ID: &id,
})

permission2 := "r"
id2 := "partial"

signedIdentifiers = append(signedIdentifiers, &SignedIdentifier{
ID: &id2,
AccessPolicy: &AccessPolicy{
Permission: &permission2,
},
})

id3 := "full"
permission3 := "r"
start := time.Date(2021, 6, 8, 2, 10, 9, 0, time.UTC)
expiry := time.Date(2021, 6, 8, 2, 10, 9, 0, time.UTC)

signedIdentifiers = append(signedIdentifiers, &SignedIdentifier{
ID: &id3,
AccessPolicy: &AccessPolicy{
Start: &start,
Expiry: &expiry,
Permission: &permission3,
},
})

param := SetAccessPolicyOptions{
TableACL: signedIdentifiers,
}

_, err := client.SetAccessPolicy(ctx, &param)
require.NoError(t, err)

// Make a Get to assert two access policies
resp, err := client.GetAccessPolicy(ctx, nil)
require.NoError(t, err)
require.Equal(t, len(resp.SignedIdentifiers), 3)
}

func TestSetTooManyAccessPolicies(t *testing.T) {
client, delete := initClientTest(t, "storage", true)
defer delete()

start := time.Date(2020, 1, 1, 0, 0, 0, 0, time.UTC)
expiration := time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)
permission := "r"
id := "1"
signedIdentifiers := make([]*SignedIdentifier, 0)

for i := 0; i < 6; i++ {
expiration = time.Date(2024+i, 1, 1, 0, 0, 0, 0, time.UTC)
id = strconv.Itoa(i)

signedIdentifiers = append(signedIdentifiers, &SignedIdentifier{
AccessPolicy: &AccessPolicy{
Expiry: &expiration,
Start: &start,
Permission: &permission,
},
ID: &id,
})

}

param := SetAccessPolicyOptions{TableACL: signedIdentifiers}

_, err := client.SetAccessPolicy(ctx, &param)
require.NotNil(t, err, "Set access policy succeeded but should have failed")
require.Contains(t, err.Error(), errTooManyAccessPoliciesError.Error())
}

func TestSetNullAccessPolicy(t *testing.T) {
client, delete := initClientTest(t, "storage", true)
defer delete()

id := "null"

signedIdentifiers := make([]*SignedIdentifier, 0)
signedIdentifiers = append(signedIdentifiers, &SignedIdentifier{
ID: &id,
})

param := SetAccessPolicyOptions{
TableACL: signedIdentifiers,
}

_, err := client.SetAccessPolicy(ctx, &param)
require.NoError(t, err)

resp, err := client.GetAccessPolicy(ctx, nil)
require.NoError(t, err)
require.Equal(t, len(resp.SignedIdentifiers), 1)
}
Loading