Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ChangeFeed]Unify cursor and add live mode #13243

Merged
merged 9 commits into from
Sep 9, 2020

Conversation

xiafu-msft
Copy link
Contributor

No description provided.

@ghost ghost added the Storage Storage Service (Queues, Blobs, Files) label Aug 20, 2020
@@ -88,6 +88,7 @@ def list_changes(self, **kwargs):

:keyword datetime start_time:
Filters the results to return only events which happened after this time.
If start_time and continuation_token are both set, start_time will be ignored.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we throw some validation error if both are provided? Silent fallback might be harder to debug.

Copy link
Contributor Author

@xiafu-msft xiafu-msft Sep 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking if users want to use live mode and list events every one minute starting from a specific time, validation error would make it complicated to implement. When I added a top level while loop in samples/change_feed_samples.py, I feel remove the validation error could be more user friendly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

User should be able to initialize CF with start date and then use continuation in subsequent iterations. but that should be two separate calls. Accepting two arguments that mean same (both start time and continuation point to starting point) and ignoring one of them isn't great. We should avoid this kind of implicit rules.
Take a look at .NET sample for reference https://github.com/Azure/azure-sdk-for-net/blob/5dbcc86da8b60ddec313455ddb6cb9d72b5175a3/sdk/storage/Azure.Storage.Blobs.ChangeFeed/samples/Sample01b_HelloWorldAsync.cs#L132-L155

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I agree with Kamil here, in Java at least we have 3 methods - no start time or end time, one with start and end time (optional) and one with cursor

Copy link
Contributor Author

@xiafu-msft xiafu-msft Sep 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I put the reason here #13243 (comment)

end_time_in_cursor_obj = \
datetime.strptime(end_time_in_cursor, '%Y-%m-%dT%H:%M:%S+00:00') if end_time_in_cursor else None
# self.end_time is in datetime format
self.end_time = end_time or end_time_in_cursor_obj
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In C# and JAVA we either accept token OR start and end time. Should we make these mutually exclusive here as well ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don’t think we want to make it mutual exclusive.
It’s possible that the user doesn’t want to specify an end time when listing events at first, then the user started using the continuation token that doesn’t have an end time, while he wants to add an end time. It’s also possible he wants to overwrite the end time.

Copy link
Contributor

@kasobol-msft kasobol-msft Sep 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this would only complicate usage of this feature and number of corner cases grows if we allow that.
I think user should:

  • use start_time and end_time (optional) to define boundaries of iteration
  • use continuation_token to progress/resume already defined listing of events in frozen context.

@gapra-msft what do you think ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, yeah I think it makes more sense to reason if they are mutually exclusive. Think about if a user specifies an end time randomly before the continuation token start time or something like that? There are a lot of random cases we need to look out for if we make them all configurable.

Copy link
Contributor Author

@xiafu-msft xiafu-msft Sep 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ahhh while I think it's just to give customers more choice. We never stop adding features because that will cause more random cases to look out right?

Customers can always run into weird problems because they love to use the SDK in a weird way, so I think we want to support as much as we can.

Copy link
Member

@gapra-msft gapra-msft Sep 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think adding a feature when its asked for is a little different than this case. I think to start off, we need to make using CF as simple as possible. Should a customer request this ability, we can always have a discussion about it then. I'm worried we're over-engineering this ability

Copy link
Contributor Author

@xiafu-msft xiafu-msft Sep 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is the python changefeed interface, list_changes(self, **kwargs):
end_time is in kwargs, it will not add any complexity.
user need to specify end_time=XXXX then it will be applied.

Comment on lines 97 to 111
def list_events_in_live_mode(self):
# Instantiate a ChangeFeedClient
cf_client = ChangeFeedClient("https://{}.blob.core.windows.net".format(self.ACCOUNT_NAME),
credential=self.ACCOUNT_KEY)

start_time = datetime(2020, 9, 1, 1)
token = None

while True:
# start_time will be ignored if start_time and continuation_token are both non-empty
change_feed = cf_client.list_changes(start_time=start_time).by_page(continuation_token=token)

for page in change_feed:
for event in page:
print(event)
token = change_feed.continuation_token

sleep(60)
print("continue printing events")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to enable users to start listing events from a specific time. That’s why I removed the start_time and continuation toke mutual exclusive check. I feel the code is simple in this way. I also added a reminder in the doc that start time will be ignored if it’s set together with continuation token.

I can add revert the change if you strongly feel the doc is not as safe as throw exception.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think about what is least possible to surprise customer and how much time does it take they find out they have a bug.
I would say those are options in general (sorted by how fast feedback is):

  • compiler enforced contract gives fastest feedback (which in python is grey area I know). You can't run program until you fix code.
  • validation (as soon as they run program it blows up, please take a look at first page here https://www.martinfowler.com/ieeeSoftware/failFast.pdf )
  • documentation. this can be ignored initially or misunderstood by user. they'll most probably pay closer attention after long debugging session.

Copy link
Contributor Author

@xiafu-msft xiafu-msft Sep 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I can get you, I can revert this one since the point is just to let the customer listing events with a start time continuously using one while loop. Users can use a workaround by getting the continuation token first then start the while loop.

How about the end_time one? If we make end_time and continuation token mutual exlusive, those user only having a continuation token cannot list range of events.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here, let's not mix them.
Let's use (start time, end time) tuple to define iteration boundary and continuation just to progress through. If user wants to change boundaries they should start new CF.

Copy link
Contributor Author

@xiafu-msft xiafu-msft Sep 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay! I will make start_time/end_time and continuation token mutual exclusive.
Also I will create a github issue in case users want to specify end_time when using continuation token without an end_time. Does it sound good?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think more than likely a user is going to use the continuation token API when they are just trying to infinitely loop over events (so endTime is null or Max). For now at least I can't see a situation where someone wants to stop listing earlier - and even then the customer can do this by filtering for events and stopping the iteration when they hit a certain date

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I can get you. so I'm asking if we want to create an issue in case users want that, currently python hasn't received any users feedback of changefeed, we don't know what they want.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As for making an issue, I think we should let the request come from the customer if they really do want it. I wonder if creating a github issue might invite the idea into their head which isnt ideal

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would rather steer customer away from using api that way. Also YAGNI

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay to not steer the customer away, I made them mutual exclusive and I will not create an issue 😀

gapra-msft
gapra-msft previously approved these changes Sep 8, 2020
2. make the list of shards in cursor to a dict in internal code
3. test list 3-shard events in multiple times generate same results as
   list all events at once
4. Java is using sequential list, so it could give 1 shard cursor even
   there are 3 shards, the test makes sure python is working with 1 shard
   cursor.
@xiafu-msft xiafu-msft merged commit f52dba8 into Azure:master Sep 9, 2020
xiafu-msft added a commit that referenced this pull request Sep 23, 2020
* Sync eng/common directory with azure-sdk-tools repository for Tools PR 916 (#13374)

* Update Language Settings File (#13389)

* [Storage]Revert equal sign in async (#13501)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 930 (#13384)

* fix authorization header on asyncio requests containing url-encoded chars (#13346)

* fix authorization header on asyncio requests containing url-encoded-able characters (=, ! etc)

* edit all authentication files and add a test

Co-authored-by: xiafu <xiafu@microsoft.com>

* If match headers (#13315)

* verifying match conditions are correct, adding tests for other conditions

* changes to verify etag is working, think it's being ignored by service

* another change to test file, passing the correct etag to the service, it returns a 204 so no error is produced

* renaming vars

* testing for if-match conditions to make sure they are properly parsed and sent to service

* reverting header back

* testing update to reflect new response in create_entity

* [text analytics] link opinion mining in the readme (#13493)

* modify bing id docstring to mention Bing Entity Search more (#13509)

* Tableitem metadata (#13445)

* started collecting metadata, issue with asynciterator not being iterable

* simplifying code, improving testing

* issue with one recording, reran and seems good to go

* fixing doc strings

* fixed two tests, skipped another one. AsyncPageIterator is missing something to make it an iterator

* new recording and skipping one test until can figure out AsyncPageIterator issue

* adding asserts

* add test for date and table_name, api_version is not returned unless full_metadata is specified

* [EventGrid] Receive Functions (#13428)

* initial changes

* comments

* str

* static

* from_json

* Update sdk/eventgrid/azure-eventgrid/azure/eventgrid/_helpers.py

* Fix storage file datalake readme and samples issues (#12512)

* Fix storage file datalake readme issues

* Update README.md

* Make doc-owner recommended revisions to eventhub readme and samples. (#13518)

Move auth samples into samples and link from readme rather than line.
Add snippets to the top of samples lacking any verbiage.
Whitespace improvements in client auth sample
Explicit samples for connstr auth to link from new terse readme auth section.

* [ServiceBus] Clean up README prior to P6 with doc-owner recommendations. (#13511)

* Remove long-form samples from readme authentication section and move them into formal samples with URIs. (This by recommendation of docs folks, and supported via UX interview with user stating that the README was getting long in the teeth with this section being less critical)

* Live pipeline issues (#13526)

* changes to the tests that reflects the new serialization of EntityProperties, failing on acl payloads and sas signed identifiers

* re-generated code, fixed a couple test methods, solved the media type issue for AccessPolicies, re-recorded tests because of changes in EntityProperty

* updating a recording that slipped through

* 404 python erroring sanitize_setup. should not be (#13532)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 946 (#13533)

* update release date for Sep (#13371)

* update release date for Sep

* fix typo

* update release date for Sep (#13370)

* update changelog (#13369)

* [text analytics] add --pre suffix to pip install (#13523)

* Change prerelease versioning (#13500)

- Use a instead of dev
- Fix dev to alpha for regression test
- Clean-up some devops feed publishing steps

* add test for opinion in diff sentence (#13524)

* don't want to exclude mgmt. auto-increments are fine here (#13549)

* [keyvault] fix include_pending param and 2016-10-01 compatibility (#13161)

* Add redirect_uri argument to InteractiveBrowserCredential (#13480)

* Sync eng/common directory with azure-sdk-tools for PR 955 (#13553)

* Increment package version after release of azure_storage_file_datalake (#13111)

* Increment package version after release of azure_storage_file_share (#13106)

* Clean up "?" if there is no query in request URL (#13530)

* cleaned up the ? in source urls

* [devtool]turn primary_endpoint and key from unicode into str

Co-authored-by: xiafu <xiafu@microsoft.com>

* Update readme samples (#13486)

* updated each sample to use env variables, more informative print statements

* updating readme with more accurate links, still missing a few

* grammar fixes

* added a readme, and async versions for auth and create/delete operations

* added more files (copies for async), and a README to the samples portion

* basic commit, merging others into one branch

* put async into another folder

* more README updates to align with .NET

* initial draft of README and samples

* initial comments from cala and kate

* caught a text analytics reference, thanks kate

* caught a text analytics reference, thanks kate

* reverting version

* fixing two broken links

* recording for test_list_tables that got left out

* fixing broken links

* another attempt at broken links

* changing parentheses to a bracket per kristas recommendation

* commenting out one test with weird behavior

* found an actual broken link

* lint fixes

* update to readme and samples per cala, issy, and kates recs. added examples to sphinx docs that were not there.

* added a quote around odata filter, thanks Chris for the catch

* updating a few more broken links

* adding an aka.ms link for pypi

* two more broken links

* reverting a change back

* fixing missing END statements, removing link to nowhere, correcting logger

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 926 (#13368)

* [keyvault] add scope enum (#13516)

* delete connection_string in recorded tests (#13557)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 823 (#13394)

* Add sample docstrings to eg samples (#13572)

* docstrings

* Apply suggestions from code review

* few more docstrings

* decode

* add licesnse

* [EventGrid] README.md updates (#13517)

* Fix a link which did not have a target
* Use "Event Grid" instead of "Eventgrid" or "EventGrid" in prose

* Cloud Event Abstraction (#13542)

* initial commit

* oops

* some changes

* lint

* test fix

* sas key

* some more cahgnes

* test fix

* 2.7 compat

* comments

* extensions

* changes

* Update sdk/eventgrid/azure-eventgrid/azure/eventgrid/_models.py

* remove test

* Increment version for storage releases (#13096)

* Increment package version after release of azure_storage_blob

* Fix a docstring problem (#13579)

* SharedTokenCacheCredential uses MSAL when given an AuthenticationRecord (#13490)

* [EventHub] add SAS token auth capabilities to EventHub (#13354)

* Add SAS token support to EventHub for connection string via 'sharedaccesssignature'
* Adds changelog/docs/samples/tests, for now, utilize the old-style of test structure for sync vs async instead of preparer until import issue is resolved.

* [Evengrid] Regenrate code gen  (#13584)

* Regenrate swagger

* fix import

* regen

* Update Language Settings file (#13583)

* Added blob exists method  (#13221)

* added feature and unit tests

* fixed failing test issue

* added async method and more unit tests

* ffixed passed parameters

* fixed python 27 issue with kwargs

* reset commit

* Update _blob_client_async.py

removed unused import, fixed linter

* Update sdk/storage/azure-storage-blob/azure/storage/blob/aio/_blob_client_async.py

Co-authored-by: Xiaoxi Fu <49707495+xiafu-msft@users.noreply.github.com>

* Update sdk/storage/azure-storage-blob/azure/storage/blob/_blob_client.py

Co-authored-by: Xiaoxi Fu <49707495+xiafu-msft@users.noreply.github.com>

* fixed failing tests

* fixed linting/import order

Co-authored-by: Xiaoxi Fu <49707495+xiafu-msft@users.noreply.github.com>

* [keyvault] administration package readme (#13489)

* add release date to changelog (#13590)

* Sync eng/common directory with azure-sdk-tools repository (#13589)

* VisualStudioCodeCredential raises CredentialUnavailableError when configured for ADFS (#13556)

* Implement full vault backup and restore (#13525)

* Identity release notes (#13585)

* [DataLake][Rename]Rename with Sas (#12057)

* [DataLake][Rename]Rename with Sas

* small fix

* recordings

* fix pylint

* fix pylint

* fix pylint

* Update sdk/storage/azure-storage-file-datalake/azure/storage/filedatalake/aio/_data_lake_directory_client_async.py

* [Schema Registry + Avro Serializer] 1.0.0b1 (#13124)

* init commit

* avro serializer structure

* adding avro serializer

* tweak api version and fix a typo

* test template

* avro serializer sync draft

* major azure sr client work done

* add sample docstring for sr

* avro serializer async impl

* close the writer

* update avro se/de impl

* update avro serializer impl

* fix apireview reported error in sr

* srav namespace, setup update

* doc update

* update doc and api

* impl, doc update

* partial update according to laruent's feedback

* be consistent with eh extension structure

* more update code according to feedback

* update credential config

* rename package name to azure-schemaregistry-avroserializer

* fix pylint

* try ci fix

* fix test for py27 as avro only accept unicode

* first round of review feedback

* remove temp ci experiment

* init add conftest.py to pass py2.7 test

* laurent feedback update

* remove dictmixin for b1, update comment in sample

* update api in avroserializer and update test and readme

* update test, docs and links

* add share requirement

* update avro dependency

* pr feedback and livetest update

* Win py27 issue (#13595)

* change to the processor to check for BOM at the beginning

* changes to serialize and deserialize to be consistent with py2/3 strings/unicode types. new recordings for all that, changes to tests for everything, and a bug in sas file

* forgot to add a unicode explicitly, re-recorded some tests because of that

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 969 (#13591)

* [ServiceBus] Expose internal amqp message properties via AMQPMessage wrapper object on Message (#13564)

* Expose internal amqp message properties via AMQPMessage wrapper object on Message.  Add test, changelog notes and docstring.  (Note: Cannot rename old message as uamqp relies on the internal property name.  Should likely be adapted.)

Co-authored-by: Adam Ling (MSFT) <adam_ling@outlook.com>

* Update doc link in README.md (#13618)

* Bump template version (#13580)

* Increment package version after release of azure_appconfiguration (#13620)

* Increment package version after release of azure_search_documents (#13622)

* Increment package version after release of azure_core (#13621)

* Fix data nspkg (#13623)

* Add data nspkg to CI (#13626)

* Update EH and SB code owner (#13633)

* Rename ServiceBusManagementClient to ServiceBusAdministrationClient (#13597)

* Rename ServiceBusManagementClient to ServiceBusAdministrationClient

* Increment package version after release of azure_keyvault_certificates (#13630)

* Add administration package to Key Vault pipeline (#13631)

* fixed the long description, addition to changelog (#13637)

* fixed the long description, addition to changelog

* addresssing izzy's comments

* [EventGrid] Fix lint errors (#13640)

* Fix lint errors

* comments

* * Remove `is_anonymous_accessible` from management entities. (#13628)

* Remove `support_ordering` from `create_queue` and `QueueProperties`
* Remove `enable_subscription_partitioning` from `create_topic` and `TopicProperties`
* update tests/changelog

* Eventgrid Prepare for Release (#13643)

* Release preparation

* shared

* codeowners

* [Schema Registry] Fix docstring and docs (#13625)

* fix docstring and docs

* update codeowner and ci config

* update init in serializer

* update readme

* update sr dependecy in avro serializer

* update module __init__.py

* revert dependcy to see if it helps doc generetion

* [ServiceBus] Replace get_*_deadletter_receiver functions with a sub_queue parameter (#13552)

* Remove get_*_deadletter_receiver functions and add a sub_queue parameter to get_*_receiver functions taking enum SubQueue to specify the target subqueue.
Adjusts tests, docs accordingly.

Co-authored-by: Adam Ling (MSFT) <adam_ling@outlook.com>

* [Storage]Fix a permission bug and add enable test for list blob with metadata (#13260)

* Fix permission bug

* test list blobs with metadata

* add snapshot to get_blob_properties() response

* fix test

* [ChangeFeed]Unify cursor and add live mode (#13243)

* [ChangeFeed]Unify cursor and add live mode

* make the token into a str

* address comments

* 1. add a while True for sample
2. make the list of shards in cursor to a dict in internal code
3. test list 3-shard events in multiple times generate same results as
   list all events at once
4. Java is using sequential list, so it could give 1 shard cursor even
   there are 3 shards, the test makes sure python is working with 1 shard
   cursor.

* make end_time/start_time and continuation_token mutual exclusive

* update dependency version

* make all '' to "" in cursor

* fix pylint and update changelog

* fix blob pylint

* added playback mode only marker (#13636)

* Update changelog (#13641)

* added changelogs

* added prs/issues in the changelogs

* Update CHANGELOG.md (#13657)

* - Rename `entity_availability_status` to `availability_status` (#13629)

- Make it unsettable in create_* mgmt operations. (as it is readonly)
- adjust tests, docs etc.

* init resourcemover ci (#13666)

Co-authored-by: xichen <xichen@microsoft.com>

* Sdk automation/track2 azure mgmt keyvault (#13662)

* Generated from c273efbfeb4c2c2e0729579114947c91ab747daa

add tag

* version and test

Co-authored-by: SDK Automation <sdkautomation@microsoft.com>

* Increment package version after release of azure_keyvault_administration (#13651)

* Increment package version after release of azure_identity (#13652)

* [SchemaRegistry] small fix in setup.py (#13677)

* [SchemaRegistry] Pin avro serializer dependency version (#13649)

* Increment package version after release of azure_data_tables (#13642)

* Increment version for eventhub releases (#13644)

* Increment package version after release of azure_eventhub

* Increment package version after release of azure_eventhub_checkpointstoreblob

* Increment package version after release of azure_eventhub_checkpointstoreblob_aio

* Add parameters to function (#13653)

* [ServiceBus] Consistency review changes as detailed in issue #12415. (#13160)

* Consistency review changes as detailed in issue #12415.
* significant amount of renames, parameter removal, mgmt shim class building, and a few added capabilities in terms of renew_lock retval and receive_deferred param acceptance.
* Update mgmt test recordings

Co-authored-by: Adam Ling (MSFT) <adam_ling@outlook.com>

* [SchemaRegistry] Re-enable links check (#13689)

* Release sdk resourcemover (#13665)

* Generated from b7867a975ec9c797332d735ed8796474322c6621

fix schemas parameter

* init ci and version

Co-authored-by: SDK Automation <sdkautomation@microsoft.com>
Co-authored-by: xichen <xichen@microsoft.com>

* [ServiceBus] Support SAS token-via-connection-string auth, and remove ServiceBusSharedKeyCredential export (#13627)

- Remove public documentation and exports of ServiceBusSharedKeyCredential until we chose to release it across all languages.
- Support for Sas Token connection strings (tests, etc)
- Add safety net for if signature and key are both provided in connstr (inspired by .nets approach)

Co-authored-by: Rakshith Bhyravabhotla <rakshith.bhyravabhotla@gmail.com>

* [text analytics] default to v3.1-preview.2, have it listed under enum V3_1_PREVIEW (#13708)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 982 (#13701)

* Remove locale from docs links (#13672)

* [ServiceBus] Set 7.0.0b6 release date in changelog (#13715)

* [ServiceBus] Sample Fix (#13719)

* fix samples

* revert duration

* Increment package version after release of azure_schemaregistry_avroserializer (#13682)

* Increment package version after release of azure_schemaregistry (#13679)

* Revert the changes of relative links (#13681)

* KeyVaultBackupClient tests (#13709)

* Release sdk automanage (#13693)

* add automanage ci

* auto generated sdk

* add init

Co-authored-by: xichen <xichen@microsoft.com>

* Replace UTC_Now() workaround with MSRest.UTC (#13498)

* use msrest.serialization utc instead of custom implementation

* update reference for utc

Co-authored-by: Andy Gee <angee@microsoft.com>

* Increment package version after release of azure_servicebus (#13720)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 965 (#13578)

* Test preparer region config loader (and DeleteAfter fixes) (#12924)

* Initial implementation of region-loading-from-config, and opting in for SB and EH to support canary region specification.

* Truncate DeleteAfter at milliseconds to hopefully allow it to get picked up by engsys.  Add get_region_override to __init__ exports.

* Provide better validation semantics for the get_region_override function. (empty/null regions is invalid)

* Sync eng/common directory with azure-sdk-tools for PR 973 (#13645)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 973

* Update update-docs-metadata.ps1

* Update update-docs-metadata.ps1

Co-authored-by: Sima Zhu <48036328+sima-zhu@users.noreply.github.com>

* Update Tables Async Samples Refs (#13764)

* remove dependency install from azure-sdk-tools

* update tables samples w/ appropriate relative URLs

* undo accidental commit

* admonition in table service client not indented properly

* Internal code for performing Key Vault crypto operations locally (#12490)

* Abstract auth to the dev feeds. additionally, add pip auth (#13522)

* abstract auth to the dev feeds. additionally, add pip auth
* rename to yml-style filename formatting

* [EventHubs] Make __init__.py compatible with pkgutil-style namespace (#13210)

* Make __init__.py compatible with pkgutil-style namespace

Fixes #13187

* fix pylint and add license info

Co-authored-by: Yunhao Ling <adam_ling@outlook.com>

* Raise msal-extensions dependency to ~=0.3.0 (#13635)

* add redacted_text to samples (#13521)

* adds support for enums by converting to string before sending on the … (#13726)

* adds support for enums by converting to string before sending on the wire

* forgot about considerations for python2 strings/unicode stuff

* Allow skip publish DocMS or Github IO for each artifact (#13754)

* Update codeowners file for Azure Template (#13485)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 999 (#13791)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 1000 (#13792)

* regenerated with new autorest version (#13814)

* removed try/except wrapper on upsert method, added _process_table_error instead of create call (#13815)

fixes #13678

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 974 (#13650)

* [EventHubs] Update extensions.__ini__.py to the correct namespace module format (#13773)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 895 (#13648)

* unskip aad tests (#13818)

* [text analytics] don't take doc # in json pointer to account for now bc of service bug (#13820)

* Increment package version after release of azure_eventgrid (#13646)

* [T2-GA] Appconfiguration (#13784)

* generate appconfiguration track2 ga version

* fix changelog

* fix version

* fix changelog

* generate keyvault track2 ga version (#13786)

* generate monitor track2 ga version (#13804)

* generate eventhub track2 ga version (#13805)

* generate network track2 ga version (#13810)

* generate compute track2 ga version (#13830)

* [T2-GA] Keyvault (#13785)

* generate keyvault track2 ga version

* fix changelog

* Update CHANGELOG.md

Co-authored-by: changlong-liu <59815250+changlong-liu@users.noreply.github.com>

* CryptographyClient can decrypt and sign locally (#13772)

* update release date (#13841)

* add link to versioning story in samples (#13842)

* GeneralNameReplacer correctly handles bytes bodies (#13710)

* [ServiceBus] Update relative paths in readme/migration guide (#13417)

* update readme paths
* update for urls

Co-authored-by: Andy Gee <angee@microsoft.com>

* Replaced relative link with absolute links and remove locale (#13846)

Replaced relative link with absolute links and remove locale

* Enable the link check on aggregate-report (#13859)

* use azure-mgmt-core 1.2.0 (#13860)

* [T2-GA] Resource (#13833)

* ci.yml (#13862)

* remove azure-common import from azure-devtools resource_testcase (#13881)

* move import from azure-common to within the track1 call (#13880)

* Synapse regenerated on 9/1 with autorest 5.2 preview (#13496)

* Synapse regenerated on 9/1 with autorest 5.2 preview

* 0.3.0

* ChangeLog

* Update with latest autorest + latest Swagger 9/16

* use autorest.python 5.3.0 (#13835)

* use autorest.python 5.3.0

* codeowner

* 20200918 streamanalytics (#13861)

* generate

* add init.py

* Small set of non-blocking changes from b6. (#13690)

- adds EOF whitespace
- renames _id to _session_id
- Adjusts docstring type

Closes #13686

* Add placeholder yml file for pipeline generation

* Bump Storage-Blob Requires to Range (#13825)

* bump dependency versions

* update upper bound appropriately

* revert unecessary changes

* updated formatting to eliminate trailing comma. these setup.py don't do that

* bump shared_requirements

* [ServiceBus] mode  (ReceiveMode) parameter needs better exception behavior (#13531)

* mode parameter needs better exception behavior, as if it is mis-typed now it will return an AttributeError further down the stack without useful guidance (and only does so when creating the handler, as well).  Will now return a TypeError at initialization.
* Add note of AttributeError->TypeError behavior for receive_mode misalignment to changelog.

* [SchemaRegistry] Samples for EH integration (#13884)

* add sample for EH integration

* add samples to readme and tweak the code

* add descriptions

* mention SR and serializer in EH

* small tweak

* Add communication service mapping

* Update docs to reflect Track 2 Python SDK status (#13813)

* Update python mgmt libraries message

* Update and rename mgmt_preview_quickstart.rst to mgmt_quickstart.rst

* Update python_mgmt_migration_guide.rst

* Update index.rst

* Update README.md

* Update README.md

* Update README.md

* KeyVaultPreparer passes required SkuFamily argument (#13845)

* Add code owners for Azure Communication Services (#13946)

* Resolve Failing SchemaRegistry Regressions (#13817)

* make the wheel retrieval a little bit more forgiving

* add 1.0.0b1 to the omission

* update version exclusion

* add deprecate note to v1 of form recognizer (#13945)

* add deprecate note to v1 of form recognizer

* update language, add back to ci.yml

* Additional Fixes from GA-ed Management Packages (#13914)

* add adal to dev_reqs for storage package

* add msrestazure to the dev_reqs for azure-common

* azure-loganalytics and azure-applicationinsights are both still track1. have to add msrestazure to the dev_reqs as azure-common requires it

* remove import of azure-common from the tests

* bump the version for azure-mgmt-core.

* crypto (#13950)

* Added partition key param for querying change feed (#13857)

* initia; changes for partitionkey for query changefeed

* Added test

* updated changelog

* moved partition_key to kwargs

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 1022 (#13885)

* Update testing (#13821)

* changes for test_table.py

* fixed up testing, noting which tests do not pass and the reasoning

* small additions to testing

* updated unicode test for storage

* final update

* updates that fix user_agent tests

* test CI returns a longer user agent so flipping the order for this test should solve the issue

* addresses anna's comments

* re-recorded a test with a recording error

* removed list comprehension for python3.5 compatability

* fixing a testing bug

* track2_azure-mgmt-baremetalinfrastructure for CI run normally (#13963)

* ci.yml for track2_azure-mgmt-baremetalinfrastructure to make CI run normally

* Removed unnecessary includes.

Co-authored-by: Mitch Denny <mitchdenny@outlook.com>

Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Co-authored-by: Chidozie Ononiwu <31145988+chidozieononiwu@users.noreply.github.com>
Co-authored-by: Aviram Hassan <41201924+aviramha@users.noreply.github.com>
Co-authored-by: Sean Kane <68240067+seankane-msft@users.noreply.github.com>
Co-authored-by: iscai-msft <43154838+iscai-msft@users.noreply.github.com>
Co-authored-by: Rakshith Bhyravabhotla <sabhyrav@microsoft.com>
Co-authored-by: Tong Xu (MSFT) <57166602+v-xuto@users.noreply.github.com>
Co-authored-by: KieranBrantnerMagee <kibrantn@microsoft.com>
Co-authored-by: Scott Beddall <45376673+scbedd@users.noreply.github.com>
Co-authored-by: Xiang Yan <xiangsjtu@gmail.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Charles Lowell <chlowe@microsoft.com>
Co-authored-by: tasherif-msft <69483382+tasherif-msft@users.noreply.github.com>
Co-authored-by: Matt Ellis <matell@microsoft.com>
Co-authored-by: Yijun Xie <48257664+YijunXieMS@users.noreply.github.com>
Co-authored-by: Adam Ling (MSFT) <adam_ling@outlook.com>
Co-authored-by: Sima Zhu <48036328+sima-zhu@users.noreply.github.com>
Co-authored-by: Laurent Mazuel <laurent.mazuel@gmail.com>
Co-authored-by: xichen <braincx@gmail.com>
Co-authored-by: xichen <xichen@microsoft.com>
Co-authored-by: changlong-liu <59815250+changlong-liu@users.noreply.github.com>
Co-authored-by: SDK Automation <sdkautomation@microsoft.com>
Co-authored-by: Rakshith Bhyravabhotla <rakshith.bhyravabhotla@gmail.com>
Co-authored-by: Andy Gee <andygee@gmail.com>
Co-authored-by: Andy Gee <angee@microsoft.com>
Co-authored-by: Piotr Jachowicz <pjachowi@gmail.com>
Co-authored-by: Kaihui (Kerwin) Sun <sunkaihuisos@gmail.com>
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
Co-authored-by: nickzhums <56864335+nickzhums@users.noreply.github.com>
Co-authored-by: turalf <tural.ferhadov@gmail.com>
Co-authored-by: Krista Pratico <krpratic@microsoft.com>
Co-authored-by: Srinath Narayanan <srnara@microsoft.com>
Co-authored-by: msyyc <70930885+msyyc@users.noreply.github.com>
Co-authored-by: Mitch Denny <mitchdenny@outlook.com>
xiafu-msft added a commit to xiafu-msft/azure-sdk-for-python that referenced this pull request Oct 1, 2020
* Sync eng/common directory with azure-sdk-tools repository for Tools PR 916 (Azure#13374)

* Update Language Settings File (Azure#13389)

* [Storage]Revert equal sign in async (Azure#13501)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 930 (Azure#13384)

* fix authorization header on asyncio requests containing url-encoded chars (Azure#13346)

* fix authorization header on asyncio requests containing url-encoded-able characters (=, ! etc)

* edit all authentication files and add a test

Co-authored-by: xiafu <xiafu@microsoft.com>

* If match headers (Azure#13315)

* verifying match conditions are correct, adding tests for other conditions

* changes to verify etag is working, think it's being ignored by service

* another change to test file, passing the correct etag to the service, it returns a 204 so no error is produced

* renaming vars

* testing for if-match conditions to make sure they are properly parsed and sent to service

* reverting header back

* testing update to reflect new response in create_entity

* [text analytics] link opinion mining in the readme (Azure#13493)

* modify bing id docstring to mention Bing Entity Search more (Azure#13509)

* Tableitem metadata (Azure#13445)

* started collecting metadata, issue with asynciterator not being iterable

* simplifying code, improving testing

* issue with one recording, reran and seems good to go

* fixing doc strings

* fixed two tests, skipped another one. AsyncPageIterator is missing something to make it an iterator

* new recording and skipping one test until can figure out AsyncPageIterator issue

* adding asserts

* add test for date and table_name, api_version is not returned unless full_metadata is specified

* [EventGrid] Receive Functions (Azure#13428)

* initial changes

* comments

* str

* static

* from_json

* Update sdk/eventgrid/azure-eventgrid/azure/eventgrid/_helpers.py

* Fix storage file datalake readme and samples issues (Azure#12512)

* Fix storage file datalake readme issues

* Update README.md

* Make doc-owner recommended revisions to eventhub readme and samples. (Azure#13518)

Move auth samples into samples and link from readme rather than line.
Add snippets to the top of samples lacking any verbiage.
Whitespace improvements in client auth sample
Explicit samples for connstr auth to link from new terse readme auth section.

* [ServiceBus] Clean up README prior to P6 with doc-owner recommendations. (Azure#13511)

* Remove long-form samples from readme authentication section and move them into formal samples with URIs. (This by recommendation of docs folks, and supported via UX interview with user stating that the README was getting long in the teeth with this section being less critical)

* Live pipeline issues (Azure#13526)

* changes to the tests that reflects the new serialization of EntityProperties, failing on acl payloads and sas signed identifiers

* re-generated code, fixed a couple test methods, solved the media type issue for AccessPolicies, re-recorded tests because of changes in EntityProperty

* updating a recording that slipped through

* 404 python erroring sanitize_setup. should not be (Azure#13532)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 946 (Azure#13533)

* update release date for Sep (Azure#13371)

* update release date for Sep

* fix typo

* update release date for Sep (Azure#13370)

* update changelog (Azure#13369)

* [text analytics] add --pre suffix to pip install (Azure#13523)

* Change prerelease versioning (Azure#13500)

- Use a instead of dev
- Fix dev to alpha for regression test
- Clean-up some devops feed publishing steps

* add test for opinion in diff sentence (Azure#13524)

* don't want to exclude mgmt. auto-increments are fine here (Azure#13549)

* [keyvault] fix include_pending param and 2016-10-01 compatibility (Azure#13161)

* Add redirect_uri argument to InteractiveBrowserCredential (Azure#13480)

* Sync eng/common directory with azure-sdk-tools for PR 955 (Azure#13553)

* Increment package version after release of azure_storage_file_datalake (Azure#13111)

* Increment package version after release of azure_storage_file_share (Azure#13106)

* Clean up "?" if there is no query in request URL (Azure#13530)

* cleaned up the ? in source urls

* [devtool]turn primary_endpoint and key from unicode into str

Co-authored-by: xiafu <xiafu@microsoft.com>

* Update readme samples (Azure#13486)

* updated each sample to use env variables, more informative print statements

* updating readme with more accurate links, still missing a few

* grammar fixes

* added a readme, and async versions for auth and create/delete operations

* added more files (copies for async), and a README to the samples portion

* basic commit, merging others into one branch

* put async into another folder

* more README updates to align with .NET

* initial draft of README and samples

* initial comments from cala and kate

* caught a text analytics reference, thanks kate

* caught a text analytics reference, thanks kate

* reverting version

* fixing two broken links

* recording for test_list_tables that got left out

* fixing broken links

* another attempt at broken links

* changing parentheses to a bracket per kristas recommendation

* commenting out one test with weird behavior

* found an actual broken link

* lint fixes

* update to readme and samples per cala, issy, and kates recs. added examples to sphinx docs that were not there.

* added a quote around odata filter, thanks Chris for the catch

* updating a few more broken links

* adding an aka.ms link for pypi

* two more broken links

* reverting a change back

* fixing missing END statements, removing link to nowhere, correcting logger

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 926 (Azure#13368)

* [keyvault] add scope enum (Azure#13516)

* delete connection_string in recorded tests (Azure#13557)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 823 (Azure#13394)

* Add sample docstrings to eg samples (Azure#13572)

* docstrings

* Apply suggestions from code review

* few more docstrings

* decode

* add licesnse

* [EventGrid] README.md updates (Azure#13517)

* Fix a link which did not have a target
* Use "Event Grid" instead of "Eventgrid" or "EventGrid" in prose

* Cloud Event Abstraction (Azure#13542)

* initial commit

* oops

* some changes

* lint

* test fix

* sas key

* some more cahgnes

* test fix

* 2.7 compat

* comments

* extensions

* changes

* Update sdk/eventgrid/azure-eventgrid/azure/eventgrid/_models.py

* remove test

* Increment version for storage releases (Azure#13096)

* Increment package version after release of azure_storage_blob

* Fix a docstring problem (Azure#13579)

* SharedTokenCacheCredential uses MSAL when given an AuthenticationRecord (Azure#13490)

* [EventHub] add SAS token auth capabilities to EventHub (Azure#13354)

* Add SAS token support to EventHub for connection string via 'sharedaccesssignature'
* Adds changelog/docs/samples/tests, for now, utilize the old-style of test structure for sync vs async instead of preparer until import issue is resolved.

* [Evengrid] Regenrate code gen  (Azure#13584)

* Regenrate swagger

* fix import

* regen

* Update Language Settings file (Azure#13583)

* Added blob exists method  (Azure#13221)

* added feature and unit tests

* fixed failing test issue

* added async method and more unit tests

* ffixed passed parameters

* fixed python 27 issue with kwargs

* reset commit

* Update _blob_client_async.py

removed unused import, fixed linter

* Update sdk/storage/azure-storage-blob/azure/storage/blob/aio/_blob_client_async.py

Co-authored-by: Xiaoxi Fu <49707495+xiafu-msft@users.noreply.github.com>

* Update sdk/storage/azure-storage-blob/azure/storage/blob/_blob_client.py

Co-authored-by: Xiaoxi Fu <49707495+xiafu-msft@users.noreply.github.com>

* fixed failing tests

* fixed linting/import order

Co-authored-by: Xiaoxi Fu <49707495+xiafu-msft@users.noreply.github.com>

* [keyvault] administration package readme (Azure#13489)

* add release date to changelog (Azure#13590)

* Sync eng/common directory with azure-sdk-tools repository (Azure#13589)

* VisualStudioCodeCredential raises CredentialUnavailableError when configured for ADFS (Azure#13556)

* Implement full vault backup and restore (Azure#13525)

* Identity release notes (Azure#13585)

* [DataLake][Rename]Rename with Sas (Azure#12057)

* [DataLake][Rename]Rename with Sas

* small fix

* recordings

* fix pylint

* fix pylint

* fix pylint

* Update sdk/storage/azure-storage-file-datalake/azure/storage/filedatalake/aio/_data_lake_directory_client_async.py

* [Schema Registry + Avro Serializer] 1.0.0b1 (Azure#13124)

* init commit

* avro serializer structure

* adding avro serializer

* tweak api version and fix a typo

* test template

* avro serializer sync draft

* major azure sr client work done

* add sample docstring for sr

* avro serializer async impl

* close the writer

* update avro se/de impl

* update avro serializer impl

* fix apireview reported error in sr

* srav namespace, setup update

* doc update

* update doc and api

* impl, doc update

* partial update according to laruent's feedback

* be consistent with eh extension structure

* more update code according to feedback

* update credential config

* rename package name to azure-schemaregistry-avroserializer

* fix pylint

* try ci fix

* fix test for py27 as avro only accept unicode

* first round of review feedback

* remove temp ci experiment

* init add conftest.py to pass py2.7 test

* laurent feedback update

* remove dictmixin for b1, update comment in sample

* update api in avroserializer and update test and readme

* update test, docs and links

* add share requirement

* update avro dependency

* pr feedback and livetest update

* Win py27 issue (Azure#13595)

* change to the processor to check for BOM at the beginning

* changes to serialize and deserialize to be consistent with py2/3 strings/unicode types. new recordings for all that, changes to tests for everything, and a bug in sas file

* forgot to add a unicode explicitly, re-recorded some tests because of that

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 969 (Azure#13591)

* [ServiceBus] Expose internal amqp message properties via AMQPMessage wrapper object on Message (Azure#13564)

* Expose internal amqp message properties via AMQPMessage wrapper object on Message.  Add test, changelog notes and docstring.  (Note: Cannot rename old message as uamqp relies on the internal property name.  Should likely be adapted.)

Co-authored-by: Adam Ling (MSFT) <adam_ling@outlook.com>

* Update doc link in README.md (Azure#13618)

* Bump template version (Azure#13580)

* Increment package version after release of azure_appconfiguration (Azure#13620)

* Increment package version after release of azure_search_documents (Azure#13622)

* Increment package version after release of azure_core (Azure#13621)

* Fix data nspkg (Azure#13623)

* Add data nspkg to CI (Azure#13626)

* Update EH and SB code owner (Azure#13633)

* Rename ServiceBusManagementClient to ServiceBusAdministrationClient (Azure#13597)

* Rename ServiceBusManagementClient to ServiceBusAdministrationClient

* Increment package version after release of azure_keyvault_certificates (Azure#13630)

* Add administration package to Key Vault pipeline (Azure#13631)

* fixed the long description, addition to changelog (Azure#13637)

* fixed the long description, addition to changelog

* addresssing izzy's comments

* [EventGrid] Fix lint errors (Azure#13640)

* Fix lint errors

* comments

* * Remove `is_anonymous_accessible` from management entities. (Azure#13628)

* Remove `support_ordering` from `create_queue` and `QueueProperties`
* Remove `enable_subscription_partitioning` from `create_topic` and `TopicProperties`
* update tests/changelog

* Eventgrid Prepare for Release (Azure#13643)

* Release preparation

* shared

* codeowners

* [Schema Registry] Fix docstring and docs (Azure#13625)

* fix docstring and docs

* update codeowner and ci config

* update init in serializer

* update readme

* update sr dependecy in avro serializer

* update module __init__.py

* revert dependcy to see if it helps doc generetion

* [ServiceBus] Replace get_*_deadletter_receiver functions with a sub_queue parameter (Azure#13552)

* Remove get_*_deadletter_receiver functions and add a sub_queue parameter to get_*_receiver functions taking enum SubQueue to specify the target subqueue.
Adjusts tests, docs accordingly.

Co-authored-by: Adam Ling (MSFT) <adam_ling@outlook.com>

* [Storage]Fix a permission bug and add enable test for list blob with metadata (Azure#13260)

* Fix permission bug

* test list blobs with metadata

* add snapshot to get_blob_properties() response

* fix test

* [ChangeFeed]Unify cursor and add live mode (Azure#13243)

* [ChangeFeed]Unify cursor and add live mode

* make the token into a str

* address comments

* 1. add a while True for sample
2. make the list of shards in cursor to a dict in internal code
3. test list 3-shard events in multiple times generate same results as
   list all events at once
4. Java is using sequential list, so it could give 1 shard cursor even
   there are 3 shards, the test makes sure python is working with 1 shard
   cursor.

* make end_time/start_time and continuation_token mutual exclusive

* update dependency version

* make all '' to "" in cursor

* fix pylint and update changelog

* fix blob pylint

* added playback mode only marker (Azure#13636)

* Update changelog (Azure#13641)

* added changelogs

* added prs/issues in the changelogs

* Update CHANGELOG.md (Azure#13657)

* - Rename `entity_availability_status` to `availability_status` (Azure#13629)

- Make it unsettable in create_* mgmt operations. (as it is readonly)
- adjust tests, docs etc.

* init resourcemover ci (Azure#13666)

Co-authored-by: xichen <xichen@microsoft.com>

* Sdk automation/track2 azure mgmt keyvault (Azure#13662)

* Generated from c273efbfeb4c2c2e0729579114947c91ab747daa

add tag

* version and test

Co-authored-by: SDK Automation <sdkautomation@microsoft.com>

* Increment package version after release of azure_keyvault_administration (Azure#13651)

* Increment package version after release of azure_identity (Azure#13652)

* [SchemaRegistry] small fix in setup.py (Azure#13677)

* [SchemaRegistry] Pin avro serializer dependency version (Azure#13649)

* Increment package version after release of azure_data_tables (Azure#13642)

* Increment version for eventhub releases (Azure#13644)

* Increment package version after release of azure_eventhub

* Increment package version after release of azure_eventhub_checkpointstoreblob

* Increment package version after release of azure_eventhub_checkpointstoreblob_aio

* Add parameters to function (Azure#13653)

* [ServiceBus] Consistency review changes as detailed in issue Azure#12415. (Azure#13160)

* Consistency review changes as detailed in issue Azure#12415.
* significant amount of renames, parameter removal, mgmt shim class building, and a few added capabilities in terms of renew_lock retval and receive_deferred param acceptance.
* Update mgmt test recordings

Co-authored-by: Adam Ling (MSFT) <adam_ling@outlook.com>

* [SchemaRegistry] Re-enable links check (Azure#13689)

* Release sdk resourcemover (Azure#13665)

* Generated from b7867a975ec9c797332d735ed8796474322c6621

fix schemas parameter

* init ci and version

Co-authored-by: SDK Automation <sdkautomation@microsoft.com>
Co-authored-by: xichen <xichen@microsoft.com>

* [ServiceBus] Support SAS token-via-connection-string auth, and remove ServiceBusSharedKeyCredential export (Azure#13627)

- Remove public documentation and exports of ServiceBusSharedKeyCredential until we chose to release it across all languages.
- Support for Sas Token connection strings (tests, etc)
- Add safety net for if signature and key are both provided in connstr (inspired by .nets approach)

Co-authored-by: Rakshith Bhyravabhotla <rakshith.bhyravabhotla@gmail.com>

* [text analytics] default to v3.1-preview.2, have it listed under enum V3_1_PREVIEW (Azure#13708)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 982 (Azure#13701)

* Remove locale from docs links (Azure#13672)

* [ServiceBus] Set 7.0.0b6 release date in changelog (Azure#13715)

* [ServiceBus] Sample Fix (Azure#13719)

* fix samples

* revert duration

* Increment package version after release of azure_schemaregistry_avroserializer (Azure#13682)

* Increment package version after release of azure_schemaregistry (Azure#13679)

* Revert the changes of relative links (Azure#13681)

* KeyVaultBackupClient tests (Azure#13709)

* Release sdk automanage (Azure#13693)

* add automanage ci

* auto generated sdk

* add init

Co-authored-by: xichen <xichen@microsoft.com>

* Replace UTC_Now() workaround with MSRest.UTC (Azure#13498)

* use msrest.serialization utc instead of custom implementation

* update reference for utc

Co-authored-by: Andy Gee <angee@microsoft.com>

* Increment package version after release of azure_servicebus (Azure#13720)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 965 (Azure#13578)

* Test preparer region config loader (and DeleteAfter fixes) (Azure#12924)

* Initial implementation of region-loading-from-config, and opting in for SB and EH to support canary region specification.

* Truncate DeleteAfter at milliseconds to hopefully allow it to get picked up by engsys.  Add get_region_override to __init__ exports.

* Provide better validation semantics for the get_region_override function. (empty/null regions is invalid)

* Sync eng/common directory with azure-sdk-tools for PR 973 (Azure#13645)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 973

* Update update-docs-metadata.ps1

* Update update-docs-metadata.ps1

Co-authored-by: Sima Zhu <48036328+sima-zhu@users.noreply.github.com>

* Update Tables Async Samples Refs (Azure#13764)

* remove dependency install from azure-sdk-tools

* update tables samples w/ appropriate relative URLs

* undo accidental commit

* admonition in table service client not indented properly

* Internal code for performing Key Vault crypto operations locally (Azure#12490)

* Abstract auth to the dev feeds. additionally, add pip auth (Azure#13522)

* abstract auth to the dev feeds. additionally, add pip auth
* rename to yml-style filename formatting

* [EventHubs] Make __init__.py compatible with pkgutil-style namespace (Azure#13210)

* Make __init__.py compatible with pkgutil-style namespace

Fixes Azure#13187

* fix pylint and add license info

Co-authored-by: Yunhao Ling <adam_ling@outlook.com>

* Raise msal-extensions dependency to ~=0.3.0 (Azure#13635)

* add redacted_text to samples (Azure#13521)

* adds support for enums by converting to string before sending on the … (Azure#13726)

* adds support for enums by converting to string before sending on the wire

* forgot about considerations for python2 strings/unicode stuff

* Allow skip publish DocMS or Github IO for each artifact (Azure#13754)

* Update codeowners file for Azure Template (Azure#13485)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 999 (Azure#13791)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 1000 (Azure#13792)

* regenerated with new autorest version (Azure#13814)

* removed try/except wrapper on upsert method, added _process_table_error instead of create call (Azure#13815)

fixes Azure#13678

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 974 (Azure#13650)

* [EventHubs] Update extensions.__ini__.py to the correct namespace module format (Azure#13773)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 895 (Azure#13648)

* unskip aad tests (Azure#13818)

* [text analytics] don't take doc # in json pointer to account for now bc of service bug (Azure#13820)

* Increment package version after release of azure_eventgrid (Azure#13646)

* [T2-GA] Appconfiguration (Azure#13784)

* generate appconfiguration track2 ga version

* fix changelog

* fix version

* fix changelog

* generate keyvault track2 ga version (Azure#13786)

* generate monitor track2 ga version (Azure#13804)

* generate eventhub track2 ga version (Azure#13805)

* generate network track2 ga version (Azure#13810)

* generate compute track2 ga version (Azure#13830)

* [T2-GA] Keyvault (Azure#13785)

* generate keyvault track2 ga version

* fix changelog

* Update CHANGELOG.md

Co-authored-by: changlong-liu <59815250+changlong-liu@users.noreply.github.com>

* CryptographyClient can decrypt and sign locally (Azure#13772)

* update release date (Azure#13841)

* add link to versioning story in samples (Azure#13842)

* GeneralNameReplacer correctly handles bytes bodies (Azure#13710)

* [ServiceBus] Update relative paths in readme/migration guide (Azure#13417)

* update readme paths
* update for urls

Co-authored-by: Andy Gee <angee@microsoft.com>

* Replaced relative link with absolute links and remove locale (Azure#13846)

Replaced relative link with absolute links and remove locale

* Enable the link check on aggregate-report (Azure#13859)

* use azure-mgmt-core 1.2.0 (Azure#13860)

* [T2-GA] Resource (Azure#13833)

* ci.yml (Azure#13862)

* remove azure-common import from azure-devtools resource_testcase (Azure#13881)

* move import from azure-common to within the track1 call (Azure#13880)

* Synapse regenerated on 9/1 with autorest 5.2 preview (Azure#13496)

* Synapse regenerated on 9/1 with autorest 5.2 preview

* 0.3.0

* ChangeLog

* Update with latest autorest + latest Swagger 9/16

* use autorest.python 5.3.0 (Azure#13835)

* use autorest.python 5.3.0

* codeowner

* 20200918 streamanalytics (Azure#13861)

* generate

* add init.py

* Small set of non-blocking changes from b6. (Azure#13690)

- adds EOF whitespace
- renames _id to _session_id
- Adjusts docstring type

Closes Azure#13686

* Add placeholder yml file for pipeline generation

* Bump Storage-Blob Requires to Range (Azure#13825)

* bump dependency versions

* update upper bound appropriately

* revert unecessary changes

* updated formatting to eliminate trailing comma. these setup.py don't do that

* bump shared_requirements

* [ServiceBus] mode  (ReceiveMode) parameter needs better exception behavior (Azure#13531)

* mode parameter needs better exception behavior, as if it is mis-typed now it will return an AttributeError further down the stack without useful guidance (and only does so when creating the handler, as well).  Will now return a TypeError at initialization.
* Add note of AttributeError->TypeError behavior for receive_mode misalignment to changelog.

* [SchemaRegistry] Samples for EH integration (Azure#13884)

* add sample for EH integration

* add samples to readme and tweak the code

* add descriptions

* mention SR and serializer in EH

* small tweak

* Add communication service mapping

* Update docs to reflect Track 2 Python SDK status (Azure#13813)

* Update python mgmt libraries message

* Update and rename mgmt_preview_quickstart.rst to mgmt_quickstart.rst

* Update python_mgmt_migration_guide.rst

* Update index.rst

* Update README.md

* Update README.md

* Update README.md

* KeyVaultPreparer passes required SkuFamily argument (Azure#13845)

* Add code owners for Azure Communication Services (Azure#13946)

* Resolve Failing SchemaRegistry Regressions (Azure#13817)

* make the wheel retrieval a little bit more forgiving

* add 1.0.0b1 to the omission

* update version exclusion

* add deprecate note to v1 of form recognizer (Azure#13945)

* add deprecate note to v1 of form recognizer

* update language, add back to ci.yml

* Additional Fixes from GA-ed Management Packages (Azure#13914)

* add adal to dev_reqs for storage package

* add msrestazure to the dev_reqs for azure-common

* azure-loganalytics and azure-applicationinsights are both still track1. have to add msrestazure to the dev_reqs as azure-common requires it

* remove import of azure-common from the tests

* bump the version for azure-mgmt-core.

* crypto (Azure#13950)

* Added partition key param for querying change feed (Azure#13857)

* initia; changes for partitionkey for query changefeed

* Added test

* updated changelog

* moved partition_key to kwargs

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 1022 (Azure#13885)

* Update testing (Azure#13821)

* changes for test_table.py

* fixed up testing, noting which tests do not pass and the reasoning

* small additions to testing

* updated unicode test for storage

* final update

* updates that fix user_agent tests

* test CI returns a longer user agent so flipping the order for this test should solve the issue

* addresses anna's comments

* re-recorded a test with a recording error

* removed list comprehension for python3.5 compatability

* fixing a testing bug

* track2_azure-mgmt-baremetalinfrastructure for CI run normally (Azure#13963)

* ci.yml for track2_azure-mgmt-baremetalinfrastructure to make CI run normally

* Removed unnecessary includes.

Co-authored-by: Mitch Denny <mitchdenny@outlook.com>

Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Co-authored-by: Chidozie Ononiwu <31145988+chidozieononiwu@users.noreply.github.com>
Co-authored-by: Aviram Hassan <41201924+aviramha@users.noreply.github.com>
Co-authored-by: Sean Kane <68240067+seankane-msft@users.noreply.github.com>
Co-authored-by: iscai-msft <43154838+iscai-msft@users.noreply.github.com>
Co-authored-by: Rakshith Bhyravabhotla <sabhyrav@microsoft.com>
Co-authored-by: Tong Xu (MSFT) <57166602+v-xuto@users.noreply.github.com>
Co-authored-by: KieranBrantnerMagee <kibrantn@microsoft.com>
Co-authored-by: Scott Beddall <45376673+scbedd@users.noreply.github.com>
Co-authored-by: Xiang Yan <xiangsjtu@gmail.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Charles Lowell <chlowe@microsoft.com>
Co-authored-by: tasherif-msft <69483382+tasherif-msft@users.noreply.github.com>
Co-authored-by: Matt Ellis <matell@microsoft.com>
Co-authored-by: Yijun Xie <48257664+YijunXieMS@users.noreply.github.com>
Co-authored-by: Adam Ling (MSFT) <adam_ling@outlook.com>
Co-authored-by: Sima Zhu <48036328+sima-zhu@users.noreply.github.com>
Co-authored-by: Laurent Mazuel <laurent.mazuel@gmail.com>
Co-authored-by: xichen <braincx@gmail.com>
Co-authored-by: xichen <xichen@microsoft.com>
Co-authored-by: changlong-liu <59815250+changlong-liu@users.noreply.github.com>
Co-authored-by: SDK Automation <sdkautomation@microsoft.com>
Co-authored-by: Rakshith Bhyravabhotla <rakshith.bhyravabhotla@gmail.com>
Co-authored-by: Andy Gee <andygee@gmail.com>
Co-authored-by: Andy Gee <angee@microsoft.com>
Co-authored-by: Piotr Jachowicz <pjachowi@gmail.com>
Co-authored-by: Kaihui (Kerwin) Sun <sunkaihuisos@gmail.com>
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
Co-authored-by: nickzhums <56864335+nickzhums@users.noreply.github.com>
Co-authored-by: turalf <tural.ferhadov@gmail.com>
Co-authored-by: Krista Pratico <krpratic@microsoft.com>
Co-authored-by: Srinath Narayanan <srnara@microsoft.com>
Co-authored-by: msyyc <70930885+msyyc@users.noreply.github.com>
Co-authored-by: Mitch Denny <mitchdenny@outlook.com>
xiafu-msft added a commit that referenced this pull request Oct 2, 2020
* [Storage][Generate]Generate Blob, Share, Datalake code

* regenerate code

* reverted undelete container changes (#13377)

* [ADLS]change api version

* Added Last Access Time Feature (#13433)

* converted BlobItemInternal to BlobProperties and added the attribute in model's BlobProperties

* removed none default

* added unit test for get properties with lat

* more unit tests

* fixed failing test

* added docstrings and addeed extra test

* Update sdk/storage/azure-storage-blob/azure/storage/blob/_models.py

Co-authored-by: Xiaoxi Fu <49707495+xiafu-msft@users.noreply.github.com>

Co-authored-by: Xiaoxi Fu <49707495+xiafu-msft@users.noreply.github.com>

* [Swagger]regenerate swagger for file-share

* Added SMB Multichannel protocol (#13795)

* added smb multichannel feature

* added protocol properties to desarialize

* added two more classes

* better docstrings

* added test and everything is working

* added async

* passing tests

* fixed var names

* Merge master again (#13967)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 916 (#13374)

* Update Language Settings File (#13389)

* [Storage]Revert equal sign in async (#13501)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 930 (#13384)

* fix authorization header on asyncio requests containing url-encoded chars (#13346)

* fix authorization header on asyncio requests containing url-encoded-able characters (=, ! etc)

* edit all authentication files and add a test

Co-authored-by: xiafu <xiafu@microsoft.com>

* If match headers (#13315)

* verifying match conditions are correct, adding tests for other conditions

* changes to verify etag is working, think it's being ignored by service

* another change to test file, passing the correct etag to the service, it returns a 204 so no error is produced

* renaming vars

* testing for if-match conditions to make sure they are properly parsed and sent to service

* reverting header back

* testing update to reflect new response in create_entity

* [text analytics] link opinion mining in the readme (#13493)

* modify bing id docstring to mention Bing Entity Search more (#13509)

* Tableitem metadata (#13445)

* started collecting metadata, issue with asynciterator not being iterable

* simplifying code, improving testing

* issue with one recording, reran and seems good to go

* fixing doc strings

* fixed two tests, skipped another one. AsyncPageIterator is missing something to make it an iterator

* new recording and skipping one test until can figure out AsyncPageIterator issue

* adding asserts

* add test for date and table_name, api_version is not returned unless full_metadata is specified

* [EventGrid] Receive Functions (#13428)

* initial changes

* comments

* str

* static

* from_json

* Update sdk/eventgrid/azure-eventgrid/azure/eventgrid/_helpers.py

* Fix storage file datalake readme and samples issues (#12512)

* Fix storage file datalake readme issues

* Update README.md

* Make doc-owner recommended revisions to eventhub readme and samples. (#13518)

Move auth samples into samples and link from readme rather than line.
Add snippets to the top of samples lacking any verbiage.
Whitespace improvements in client auth sample
Explicit samples for connstr auth to link from new terse readme auth section.

* [ServiceBus] Clean up README prior to P6 with doc-owner recommendations. (#13511)

* Remove long-form samples from readme authentication section and move them into formal samples with URIs. (This by recommendation of docs folks, and supported via UX interview with user stating that the README was getting long in the teeth with this section being less critical)

* Live pipeline issues (#13526)

* changes to the tests that reflects the new serialization of EntityProperties, failing on acl payloads and sas signed identifiers

* re-generated code, fixed a couple test methods, solved the media type issue for AccessPolicies, re-recorded tests because of changes in EntityProperty

* updating a recording that slipped through

* 404 python erroring sanitize_setup. should not be (#13532)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 946 (#13533)

* update release date for Sep (#13371)

* update release date for Sep

* fix typo

* update release date for Sep (#13370)

* update changelog (#13369)

* [text analytics] add --pre suffix to pip install (#13523)

* Change prerelease versioning (#13500)

- Use a instead of dev
- Fix dev to alpha for regression test
- Clean-up some devops feed publishing steps

* add test for opinion in diff sentence (#13524)

* don't want to exclude mgmt. auto-increments are fine here (#13549)

* [keyvault] fix include_pending param and 2016-10-01 compatibility (#13161)

* Add redirect_uri argument to InteractiveBrowserCredential (#13480)

* Sync eng/common directory with azure-sdk-tools for PR 955 (#13553)

* Increment package version after release of azure_storage_file_datalake (#13111)

* Increment package version after release of azure_storage_file_share (#13106)

* Clean up "?" if there is no query in request URL (#13530)

* cleaned up the ? in source urls

* [devtool]turn primary_endpoint and key from unicode into str

Co-authored-by: xiafu <xiafu@microsoft.com>

* Update readme samples (#13486)

* updated each sample to use env variables, more informative print statements

* updating readme with more accurate links, still missing a few

* grammar fixes

* added a readme, and async versions for auth and create/delete operations

* added more files (copies for async), and a README to the samples portion

* basic commit, merging others into one branch

* put async into another folder

* more README updates to align with .NET

* initial draft of README and samples

* initial comments from cala and kate

* caught a text analytics reference, thanks kate

* caught a text analytics reference, thanks kate

* reverting version

* fixing two broken links

* recording for test_list_tables that got left out

* fixing broken links

* another attempt at broken links

* changing parentheses to a bracket per kristas recommendation

* commenting out one test with weird behavior

* found an actual broken link

* lint fixes

* update to readme and samples per cala, issy, and kates recs. added examples to sphinx docs that were not there.

* added a quote around odata filter, thanks Chris for the catch

* updating a few more broken links

* adding an aka.ms link for pypi

* two more broken links

* reverting a change back

* fixing missing END statements, removing link to nowhere, correcting logger

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 926 (#13368)

* [keyvault] add scope enum (#13516)

* delete connection_string in recorded tests (#13557)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 823 (#13394)

* Add sample docstrings to eg samples (#13572)

* docstrings

* Apply suggestions from code review

* few more docstrings

* decode

* add licesnse

* [EventGrid] README.md updates (#13517)

* Fix a link which did not have a target
* Use "Event Grid" instead of "Eventgrid" or "EventGrid" in prose

* Cloud Event Abstraction (#13542)

* initial commit

* oops

* some changes

* lint

* test fix

* sas key

* some more cahgnes

* test fix

* 2.7 compat

* comments

* extensions

* changes

* Update sdk/eventgrid/azure-eventgrid/azure/eventgrid/_models.py

* remove test

* Increment version for storage releases (#13096)

* Increment package version after release of azure_storage_blob

* Fix a docstring problem (#13579)

* SharedTokenCacheCredential uses MSAL when given an AuthenticationRecord (#13490)

* [EventHub] add SAS token auth capabilities to EventHub (#13354)

* Add SAS token support to EventHub for connection string via 'sharedaccesssignature'
* Adds changelog/docs/samples/tests, for now, utilize the old-style of test structure for sync vs async instead of preparer until import issue is resolved.

* [Evengrid] Regenrate code gen  (#13584)

* Regenrate swagger

* fix import

* regen

* Update Language Settings file (#13583)

* Added blob exists method  (#13221)

* added feature and unit tests

* fixed failing test issue

* added async method and more unit tests

* ffixed passed parameters

* fixed python 27 issue with kwargs

* reset commit

* Update _blob_client_async.py

removed unused import, fixed linter

* Update sdk/storage/azure-storage-blob/azure/storage/blob/aio/_blob_client_async.py

Co-authored-by: Xiaoxi Fu <49707495+xiafu-msft@users.noreply.github.com>

* Update sdk/storage/azure-storage-blob/azure/storage/blob/_blob_client.py

Co-authored-by: Xiaoxi Fu <49707495+xiafu-msft@users.noreply.github.com>

* fixed failing tests

* fixed linting/import order

Co-authored-by: Xiaoxi Fu <49707495+xiafu-msft@users.noreply.github.com>

* [keyvault] administration package readme (#13489)

* add release date to changelog (#13590)

* Sync eng/common directory with azure-sdk-tools repository (#13589)

* VisualStudioCodeCredential raises CredentialUnavailableError when configured for ADFS (#13556)

* Implement full vault backup and restore (#13525)

* Identity release notes (#13585)

* [DataLake][Rename]Rename with Sas (#12057)

* [DataLake][Rename]Rename with Sas

* small fix

* recordings

* fix pylint

* fix pylint

* fix pylint

* Update sdk/storage/azure-storage-file-datalake/azure/storage/filedatalake/aio/_data_lake_directory_client_async.py

* [Schema Registry + Avro Serializer] 1.0.0b1 (#13124)

* init commit

* avro serializer structure

* adding avro serializer

* tweak api version and fix a typo

* test template

* avro serializer sync draft

* major azure sr client work done

* add sample docstring for sr

* avro serializer async impl

* close the writer

* update avro se/de impl

* update avro serializer impl

* fix apireview reported error in sr

* srav namespace, setup update

* doc update

* update doc and api

* impl, doc update

* partial update according to laruent's feedback

* be consistent with eh extension structure

* more update code according to feedback

* update credential config

* rename package name to azure-schemaregistry-avroserializer

* fix pylint

* try ci fix

* fix test for py27 as avro only accept unicode

* first round of review feedback

* remove temp ci experiment

* init add conftest.py to pass py2.7 test

* laurent feedback update

* remove dictmixin for b1, update comment in sample

* update api in avroserializer and update test and readme

* update test, docs and links

* add share requirement

* update avro dependency

* pr feedback and livetest update

* Win py27 issue (#13595)

* change to the processor to check for BOM at the beginning

* changes to serialize and deserialize to be consistent with py2/3 strings/unicode types. new recordings for all that, changes to tests for everything, and a bug in sas file

* forgot to add a unicode explicitly, re-recorded some tests because of that

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 969 (#13591)

* [ServiceBus] Expose internal amqp message properties via AMQPMessage wrapper object on Message (#13564)

* Expose internal amqp message properties via AMQPMessage wrapper object on Message.  Add test, changelog notes and docstring.  (Note: Cannot rename old message as uamqp relies on the internal property name.  Should likely be adapted.)

Co-authored-by: Adam Ling (MSFT) <adam_ling@outlook.com>

* Update doc link in README.md (#13618)

* Bump template version (#13580)

* Increment package version after release of azure_appconfiguration (#13620)

* Increment package version after release of azure_search_documents (#13622)

* Increment package version after release of azure_core (#13621)

* Fix data nspkg (#13623)

* Add data nspkg to CI (#13626)

* Update EH and SB code owner (#13633)

* Rename ServiceBusManagementClient to ServiceBusAdministrationClient (#13597)

* Rename ServiceBusManagementClient to ServiceBusAdministrationClient

* Increment package version after release of azure_keyvault_certificates (#13630)

* Add administration package to Key Vault pipeline (#13631)

* fixed the long description, addition to changelog (#13637)

* fixed the long description, addition to changelog

* addresssing izzy's comments

* [EventGrid] Fix lint errors (#13640)

* Fix lint errors

* comments

* * Remove `is_anonymous_accessible` from management entities. (#13628)

* Remove `support_ordering` from `create_queue` and `QueueProperties`
* Remove `enable_subscription_partitioning` from `create_topic` and `TopicProperties`
* update tests/changelog

* Eventgrid Prepare for Release (#13643)

* Release preparation

* shared

* codeowners

* [Schema Registry] Fix docstring and docs (#13625)

* fix docstring and docs

* update codeowner and ci config

* update init in serializer

* update readme

* update sr dependecy in avro serializer

* update module __init__.py

* revert dependcy to see if it helps doc generetion

* [ServiceBus] Replace get_*_deadletter_receiver functions with a sub_queue parameter (#13552)

* Remove get_*_deadletter_receiver functions and add a sub_queue parameter to get_*_receiver functions taking enum SubQueue to specify the target subqueue.
Adjusts tests, docs accordingly.

Co-authored-by: Adam Ling (MSFT) <adam_ling@outlook.com>

* [Storage]Fix a permission bug and add enable test for list blob with metadata (#13260)

* Fix permission bug

* test list blobs with metadata

* add snapshot to get_blob_properties() response

* fix test

* [ChangeFeed]Unify cursor and add live mode (#13243)

* [ChangeFeed]Unify cursor and add live mode

* make the token into a str

* address comments

* 1. add a while True for sample
2. make the list of shards in cursor to a dict in internal code
3. test list 3-shard events in multiple times generate same results as
   list all events at once
4. Java is using sequential list, so it could give 1 shard cursor even
   there are 3 shards, the test makes sure python is working with 1 shard
   cursor.

* make end_time/start_time and continuation_token mutual exclusive

* update dependency version

* make all '' to "" in cursor

* fix pylint and update changelog

* fix blob pylint

* added playback mode only marker (#13636)

* Update changelog (#13641)

* added changelogs

* added prs/issues in the changelogs

* Update CHANGELOG.md (#13657)

* - Rename `entity_availability_status` to `availability_status` (#13629)

- Make it unsettable in create_* mgmt operations. (as it is readonly)
- adjust tests, docs etc.

* init resourcemover ci (#13666)

Co-authored-by: xichen <xichen@microsoft.com>

* Sdk automation/track2 azure mgmt keyvault (#13662)

* Generated from c273efbfeb4c2c2e0729579114947c91ab747daa

add tag

* version and test

Co-authored-by: SDK Automation <sdkautomation@microsoft.com>

* Increment package version after release of azure_keyvault_administration (#13651)

* Increment package version after release of azure_identity (#13652)

* [SchemaRegistry] small fix in setup.py (#13677)

* [SchemaRegistry] Pin avro serializer dependency version (#13649)

* Increment package version after release of azure_data_tables (#13642)

* Increment version for eventhub releases (#13644)

* Increment package version after release of azure_eventhub

* Increment package version after release of azure_eventhub_checkpointstoreblob

* Increment package version after release of azure_eventhub_checkpointstoreblob_aio

* Add parameters to function (#13653)

* [ServiceBus] Consistency review changes as detailed in issue #12415. (#13160)

* Consistency review changes as detailed in issue #12415.
* significant amount of renames, parameter removal, mgmt shim class building, and a few added capabilities in terms of renew_lock retval and receive_deferred param acceptance.
* Update mgmt test recordings

Co-authored-by: Adam Ling (MSFT) <adam_ling@outlook.com>

* [SchemaRegistry] Re-enable links check (#13689)

* Release sdk resourcemover (#13665)

* Generated from b7867a975ec9c797332d735ed8796474322c6621

fix schemas parameter

* init ci and version

Co-authored-by: SDK Automation <sdkautomation@microsoft.com>
Co-authored-by: xichen <xichen@microsoft.com>

* [ServiceBus] Support SAS token-via-connection-string auth, and remove ServiceBusSharedKeyCredential export (#13627)

- Remove public documentation and exports of ServiceBusSharedKeyCredential until we chose to release it across all languages.
- Support for Sas Token connection strings (tests, etc)
- Add safety net for if signature and key are both provided in connstr (inspired by .nets approach)

Co-authored-by: Rakshith Bhyravabhotla <rakshith.bhyravabhotla@gmail.com>

* [text analytics] default to v3.1-preview.2, have it listed under enum V3_1_PREVIEW (#13708)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 982 (#13701)

* Remove locale from docs links (#13672)

* [ServiceBus] Set 7.0.0b6 release date in changelog (#13715)

* [ServiceBus] Sample Fix (#13719)

* fix samples

* revert duration

* Increment package version after release of azure_schemaregistry_avroserializer (#13682)

* Increment package version after release of azure_schemaregistry (#13679)

* Revert the changes of relative links (#13681)

* KeyVaultBackupClient tests (#13709)

* Release sdk automanage (#13693)

* add automanage ci

* auto generated sdk

* add init

Co-authored-by: xichen <xichen@microsoft.com>

* Replace UTC_Now() workaround with MSRest.UTC (#13498)

* use msrest.serialization utc instead of custom implementation

* update reference for utc

Co-authored-by: Andy Gee <angee@microsoft.com>

* Increment package version after release of azure_servicebus (#13720)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 965 (#13578)

* Test preparer region config loader (and DeleteAfter fixes) (#12924)

* Initial implementation of region-loading-from-config, and opting in for SB and EH to support canary region specification.

* Truncate DeleteAfter at milliseconds to hopefully allow it to get picked up by engsys.  Add get_region_override to __init__ exports.

* Provide better validation semantics for the get_region_override function. (empty/null regions is invalid)

* Sync eng/common directory with azure-sdk-tools for PR 973 (#13645)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 973

* Update update-docs-metadata.ps1

* Update update-docs-metadata.ps1

Co-authored-by: Sima Zhu <48036328+sima-zhu@users.noreply.github.com>

* Update Tables Async Samples Refs (#13764)

* remove dependency install from azure-sdk-tools

* update tables samples w/ appropriate relative URLs

* undo accidental commit

* admonition in table service client not indented properly

* Internal code for performing Key Vault crypto operations locally (#12490)

* Abstract auth to the dev feeds. additionally, add pip auth (#13522)

* abstract auth to the dev feeds. additionally, add pip auth
* rename to yml-style filename formatting

* [EventHubs] Make __init__.py compatible with pkgutil-style namespace (#13210)

* Make __init__.py compatible with pkgutil-style namespace

Fixes #13187

* fix pylint and add license info

Co-authored-by: Yunhao Ling <adam_ling@outlook.com>

* Raise msal-extensions dependency to ~=0.3.0 (#13635)

* add redacted_text to samples (#13521)

* adds support for enums by converting to string before sending on the … (#13726)

* adds support for enums by converting to string before sending on the wire

* forgot about considerations for python2 strings/unicode stuff

* Allow skip publish DocMS or Github IO for each artifact (#13754)

* Update codeowners file for Azure Template (#13485)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 999 (#13791)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 1000 (#13792)

* regenerated with new autorest version (#13814)

* removed try/except wrapper on upsert method, added _process_table_error instead of create call (#13815)

fixes #13678

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 974 (#13650)

* [EventHubs] Update extensions.__ini__.py to the correct namespace module format (#13773)

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 895 (#13648)

* unskip aad tests (#13818)

* [text analytics] don't take doc # in json pointer to account for now bc of service bug (#13820)

* Increment package version after release of azure_eventgrid (#13646)

* [T2-GA] Appconfiguration (#13784)

* generate appconfiguration track2 ga version

* fix changelog

* fix version

* fix changelog

* generate keyvault track2 ga version (#13786)

* generate monitor track2 ga version (#13804)

* generate eventhub track2 ga version (#13805)

* generate network track2 ga version (#13810)

* generate compute track2 ga version (#13830)

* [T2-GA] Keyvault (#13785)

* generate keyvault track2 ga version

* fix changelog

* Update CHANGELOG.md

Co-authored-by: changlong-liu <59815250+changlong-liu@users.noreply.github.com>

* CryptographyClient can decrypt and sign locally (#13772)

* update release date (#13841)

* add link to versioning story in samples (#13842)

* GeneralNameReplacer correctly handles bytes bodies (#13710)

* [ServiceBus] Update relative paths in readme/migration guide (#13417)

* update readme paths
* update for urls

Co-authored-by: Andy Gee <angee@microsoft.com>

* Replaced relative link with absolute links and remove locale (#13846)

Replaced relative link with absolute links and remove locale

* Enable the link check on aggregate-report (#13859)

* use azure-mgmt-core 1.2.0 (#13860)

* [T2-GA] Resource (#13833)

* ci.yml (#13862)

* remove azure-common import from azure-devtools resource_testcase (#13881)

* move import from azure-common to within the track1 call (#13880)

* Synapse regenerated on 9/1 with autorest 5.2 preview (#13496)

* Synapse regenerated on 9/1 with autorest 5.2 preview

* 0.3.0

* ChangeLog

* Update with latest autorest + latest Swagger 9/16

* use autorest.python 5.3.0 (#13835)

* use autorest.python 5.3.0

* codeowner

* 20200918 streamanalytics (#13861)

* generate

* add init.py

* Small set of non-blocking changes from b6. (#13690)

- adds EOF whitespace
- renames _id to _session_id
- Adjusts docstring type

Closes #13686

* Add placeholder yml file for pipeline generation

* Bump Storage-Blob Requires to Range (#13825)

* bump dependency versions

* update upper bound appropriately

* revert unecessary changes

* updated formatting to eliminate trailing comma. these setup.py don't do that

* bump shared_requirements

* [ServiceBus] mode  (ReceiveMode) parameter needs better exception behavior (#13531)

* mode parameter needs better exception behavior, as if it is mis-typed now it will return an AttributeError further down the stack without useful guidance (and only does so when creating the handler, as well).  Will now return a TypeError at initialization.
* Add note of AttributeError->TypeError behavior for receive_mode misalignment to changelog.

* [SchemaRegistry] Samples for EH integration (#13884)

* add sample for EH integration

* add samples to readme and tweak the code

* add descriptions

* mention SR and serializer in EH

* small tweak

* Add communication service mapping

* Update docs to reflect Track 2 Python SDK status (#13813)

* Update python mgmt libraries message

* Update and rename mgmt_preview_quickstart.rst to mgmt_quickstart.rst

* Update python_mgmt_migration_guide.rst

* Update index.rst

* Update README.md

* Update README.md

* Update README.md

* KeyVaultPreparer passes required SkuFamily argument (#13845)

* Add code owners for Azure Communication Services (#13946)

* Resolve Failing SchemaRegistry Regressions (#13817)

* make the wheel retrieval a little bit more forgiving

* add 1.0.0b1 to the omission

* update version exclusion

* add deprecate note to v1 of form recognizer (#13945)

* add deprecate note to v1 of form recognizer

* update language, add back to ci.yml

* Additional Fixes from GA-ed Management Packages (#13914)

* add adal to dev_reqs for storage package

* add msrestazure to the dev_reqs for azure-common

* azure-loganalytics and azure-applicationinsights are both still track1. have to add msrestazure to the dev_reqs as azure-common requires it

* remove import of azure-common from the tests

* bump the version for azure-mgmt-core.

* crypto (#13950)

* Added partition key param for querying change feed (#13857)

* initia; changes for partitionkey for query changefeed

* Added test

* updated changelog

* moved partition_key to kwargs

* Sync eng/common directory with azure-sdk-tools repository for Tools PR 1022 (#13885)

* Update testing (#13821)

* changes for test_table.py

* fixed up testing, noting which tests do not pass and the reasoning

* small additions to testing

* updated unicode test for storage

* final update

* updates that fix user_agent tests

* test CI returns a longer user agent so flipping the order for this test should solve the issue

* addresses anna's comments

* re-recorded a test with a recording error

* removed list comprehension for python3.5 compatability

* fixing a testing bug

* track2_azure-mgmt-baremetalinfrastructure for CI run normally (#13963)

* ci.yml for track2_azure-mgmt-baremetalinfrastructure to make CI run normally

* Removed unnecessary includes.

Co-authored-by: Mitch Denny <mitchdenny@outlook.com>

Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Co-authored-by: Chidozie Ononiwu <31145988+chidozieononiwu@users.noreply.github.com>
Co-authored-by: Aviram Hassan <41201924+aviramha@users.noreply.github.com>
Co-authored-by: Sean Kane <68240067+seankane-msft@users.noreply.github.com>
Co-authored-by: iscai-msft <43154838+iscai-msft@users.noreply.github.com>
Co-authored-by: Rakshith Bhyravabhotla <sabhyrav@microsoft.com>
Co-authored-by: Tong Xu (MSFT) <57166602+v-xuto@users.noreply.github.com>
Co-authored-by: KieranBrantnerMagee <kibrantn@microsoft.com>
Co-authored-by: Scott Beddall <45376673+scbedd@users.noreply.github.com>
Co-authored-by: Xiang Yan <xiangsjtu@gmail.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Charles Lowell <chlowe@microsoft.com>
Co-authored-by: tasherif-msft <69483382+tasherif-msft@users.noreply.github.com>
Co-authored-by: Matt Ellis <matell@microsoft.com>
Co-authored-by: Yijun Xie <48257664+YijunXieMS@users.noreply.github.com>
Co-authored-by: Adam Ling (MSFT) <adam_ling@outlook.com>
Co-authored-by: Sima Zhu <48036328+sima-zhu@users.noreply.github.com>
Co-authored-by: Laurent Mazuel <laurent.mazuel@gmail.com>
Co-authored-by: xichen <braincx@gmail.com>
Co-authored-by: xichen <xichen@microsoft.com>
Co-authored-by: changlong-liu <59815250+changlong-liu@users.noreply.github.com>
Co-authored-by: SDK Automation <sdkautomation@microsoft.com>
Co-authored-by: Rakshith Bhyravabhotla <rakshith.bhyravabhotla@gmail.com>
Co-authored-by: Andy Gee <andygee@gmail.com>
Co-authored-by: Andy Gee <angee@microsoft.com>
Co-authored-by: Piotr Jachowicz <pjachowi@gmail.com>
Co-authored-by: Kaihui (Kerwin) Sun <sunkaihuisos@gmail.com>
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
Co-authored-by: nickzhums <56864335+nickzhums@users.noreply.github.com>
Co-authored-by: turalf <tural.ferhadov@gmail.com>
Co-authored-by: Krista Pratico <krpratic@microsoft.com>
Co-authored-by: Srinath Narayanan <srnara@microsoft.com>
Co-authored-by: msyyc <70930885+msyyc@users.noreply.github.com>
Co-authored-by: Mitch Denny <mitchdenny@outlook.com>

* Recursive acl (#13476)

* [Storage][Datalake]Added supoort for recursive acl operations

* [DataLake]Recursive ACL

* re-record

* re-record

* rename progress_callback to progress_hook

* re-record

Co-authored-by: zezha-msft <zezha@microsoft.com>

* [Storage]API Review Comments (#14019)

* [Storage]API Review Comments

* move new_name for undelete_container to kwargs

* [Storage][Blob][QuickQuery]Arrow Format (#13750)

* [Storage][Blob][DataLake]Quick Query Arrow Format

* fix pylint

* fix pylint

* fix pylint

* fix pylint

* Set expiry (#12642)

* [DataLake][SetExpiry]Set Expiry of DataLake File

* address comments

* use datalake set_expiry operation

* add serialize rfc1123 and fix pylint

* fix pylint

* remove return type

* Added get file range with the prevsharesnapshot parameter (#13507)

* added feature

* fixed var name

* added unit test that is still being worked on

* added unit test for get file range with snapshot

* added recording of the unit test

* added async unit test recording

* [Swagger][FileShare]Regenerate for Clear Range Change

* added a deserialize method

* recoreded new tests and added deserialize

* rerecorded

* recorded some more

* changed tests for sync

* recorded async tests

* added more docstrings

* linter

* added additional api

* linter

* unused import linter

Co-authored-by: xiafu <xiafu@microsoft.com>
Co-authored-by: Xiaoxi Fu <49707495+xiafu-msft@users.noreply.github.com>

* [Datalake][Exception]Throw DataLakeAclChangeFailedError (#14129)

* [Datalake][Exception]Throw DataLakeAclChangeFailedError

* fix pylint

* fix pylint

* Share Lease Feature (#13567)

* added needed parameters for shares

* added async methods

* added more methods for interacting with the API

* fixed small mistake with elif

* added tests and access conditions

* added more tests for leases

* fixed tests

* async changes

* added await

* corrected import

* fixed async imports and wrote all tests

* linting

* share renaming

* added file lease sample

* added sample for share

* fixed samples

* added docs

* removed checks

* lease change

* lease change

* added correct lease durations

* removed spacing

* version correction

* fixed snapshot

* added snapshot tests

* added snapshot tests

* changed version

* test

* test

* more docstrings

* fixed docstrings

* more docstring changes

* removed etag

* added exta check on test

* fixed tests

* added more tests for file

* added tests for list shares

* unused import

* changed method signitures

* fixed kwargs

* linter

Co-authored-by: Xiaoxi Fu <49707495+xiafu-msft@users.noreply.github.com>

* removed changelog feature

* [DelegationSas]support directory sas & add feature for delegation sas (#14206)

* [DelegationSas]support directory sas & add feature for delegation sas

* add doc string

Co-authored-by: tasherif-msft <69483382+tasherif-msft@users.noreply.github.com>
Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Co-authored-by: Chidozie Ononiwu <31145988+chidozieononiwu@users.noreply.github.com>
Co-authored-by: Aviram Hassan <41201924+aviramha@users.noreply.github.com>
Co-authored-by: Sean Kane <68240067+seankane-msft@users.noreply.github.com>
Co-authored-by: iscai-msft <43154838+iscai-msft@users.noreply.github.com>
Co-authored-by: Rakshith Bhyravabhotla <sabhyrav@microsoft.com>
Co-authored-by: Tong Xu (MSFT) <57166602+v-xuto@users.noreply.github.com>
Co-authored-by: KieranBrantnerMagee <kibrantn@microsoft.com>
Co-authored-by: Scott Beddall <45376673+scbedd@users.noreply.github.com>
Co-authored-by: Xiang Yan <xiangsjtu@gmail.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Charles Lowell <chlowe@microsoft.com>
Co-authored-by: Matt Ellis <matell@microsoft.com>
Co-authored-by: Yijun Xie <48257664+YijunXieMS@users.noreply.github.com>
Co-authored-by: Adam Ling (MSFT) <adam_ling@outlook.com>
Co-authored-by: Sima Zhu <48036328+sima-zhu@users.noreply.github.com>
Co-authored-by: Laurent Mazuel <laurent.mazuel@gmail.com>
Co-authored-by: xichen <braincx@gmail.com>
Co-authored-by: xichen <xichen@microsoft.com>
Co-authored-by: changlong-liu <59815250+changlong-liu@users.noreply.github.com>
Co-authored-by: SDK Automation <sdkautomation@microsoft.com>
Co-authored-by: Rakshith Bhyravabhotla <rakshith.bhyravabhotla@gmail.com>
Co-authored-by: Andy Gee <andygee@gmail.com>
Co-authored-by: Andy Gee <angee@microsoft.com>
Co-authored-by: Piotr Jachowicz <pjachowi@gmail.com>
Co-authored-by: Kaihui (Kerwin) Sun <sunkaihuisos@gmail.com>
Co-authored-by: Wes Haggard <Wes.Haggard@microsoft.com>
Co-authored-by: nickzhums <56864335+nickzhums@users.noreply.github.com>
Co-authored-by: turalf <tural.ferhadov@gmail.com>
Co-authored-by: Krista Pratico <krpratic@microsoft.com>
Co-authored-by: Srinath Narayanan <srnara@microsoft.com>
Co-authored-by: msyyc <70930885+msyyc@users.noreply.github.com>
Co-authored-by: Mitch Denny <mitchdenny@outlook.com>
Co-authored-by: zezha-msft <zezha@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Storage Storage Service (Queues, Blobs, Files)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants